Hi Peter,
You can definitely do this. But can you clarify where you ideally want to display videos? Is it inside pages, or forums, or in labels on the course page, or all of those places? And can you clarify what sort of links you are trying to use?
Generally Moodle uses filters to display videos in videojs. So it sees a link to something that looks like a video, and it tries to turn it into a video player. You can also embed from the html editor a video directly if you know the video file url. When I looked at MS Stream, for embedding it had a screenshot like this:
Thats not a link to a video, thats iframe code. In the html editor you can't easily paste iframe code, but you might use the plugin Generico (or its cousin VideoEasy) to might make that easier.