Learn Azure Communication Services Day 14 – Showing Video and Screen Share Content
This blog post is part of a series called “Learn ACS“, all about Microsoft Azure Communication Services. The series covers a high-level overview of capabilities and considerations, then dives into the development detail of using ACS in your application. Find the rest of the posts in the series at learnACS.dev.
Yesterday we looked at Call.remoteParticipants as a way to build and manage a participant list, and today we’re going to continue with using that array to get access to the video streams of each participant.
Video and Content Share streams are provided for each participant in a call or meeting. There isn’t a single “mixed” stream of participant video. This is either a good thing or a bad thing, depending on what you are trying to do. It gives the greatest flexibility for the widest range of applications because as a developer you have the choice of what video feeds to display when. However, if you’re looking for single video feed to represent the active speaker, that’s something you will need to implement yourself. (totally doable using the isSpeaking property of the remoteParticipant but outside the scope of this blog post).
From a developer perspective, both video and content-sharing streams are the same. They are both streams in the remoteParticipant.videoStreams array, they have the same properties, methods and events and you work with them in the same way. (Painting the stream onto the page is exactly the same as for the local video stream we covered in Day 10.)
Each RemoteVideoStream has a isAvailable property and a isAvailableChanged event you can use to know when there’s something to display. In fact, the only way to tell video and content streams apart is by the mediaStreamType property: for video, the value is “Video” and for content, it’s “ScreenSharing”.
Putting together a production-ready solution that incorporates both video and screen sharing can be a little tricky because it’s necessary to subscribe to the availability of both streams for each participant, as well as keeping track of participants leaving and joining to make sure you add new event handlers for new people. For that reason, today’s sample is deliberately simplified and does NOT use any events. Instead, there’s a Refresh button that takes the current state of all participants and display any available streams. It’s based on joining a Teams meeting to make it easier to test.
Here’s the code: don’t forget, on Line 19, replace the placeholder text with the full URL of your Azure Function created in Day 3, including the code parameter:
index.html
| <!DOCTYPE html> | |
| <html> | |
| <head> | |
| <title>Learn ASC Day 14 - Showing Video and Screen Share Content </title> | |
| </head> | |
| <body> | |
| <h2>Learn ASC Day 14 - Showing Video and Screen Share Content</h2> | |
| <p>Call state <span style="font-weight: bold" id="call-state">-</span></p> | |
| <input id="destination-group-input" type="text" placeholder="Microsoft Teams meeting join URL (https://teams.microsoft.com/l/meetup-join/....)" | |
| style="margin-bottom:1em; width: 600px;" /> | |
| <div> | |
| <button id="connect-button" type="button" disabled="false"> | |
| Connect with Video | |
| </button> | |
| <button id="disconnect-button" type="button" disabled="true"> | |
| Disconnect | |
| </button> | |
| <button id="startvideo-button" type="button" disabled="true"> | |
| Start Video | |
| </button> | |
| <button id="stopvideo-button" type="button" disabled="true"> | |
| Stop Video | |
| </button> | |
| <button id="refreshRemoteMedia-button" type="button" disabled="true"> | |
| Refresh Remote Media | |
| </button> | |
| </div> | |
| <div id="remoteContainer" style="display:flex;"> | |
| <div id="remotevideo" style="width: 40%;height: 30vw;"></div> | |
| <div id="remotescreenshare" style="width: 40%;height: 30vw;"></div> | |
| </div> | |
| <div id="selfVideo" style="width:20%; height:20%;"></div> | |
| <hr/> | |
| <p style="font-family: sans-serif">Hi, I'm Tom! I hope you found this code sample useful. This sample code comes from a series of blog posts about Azure Communication Services. Links for all blog posts can be found at <a href="https://learnacs.dev" target="_blank" rel="noopener">learnACS.dev</a>. I blog at <a href="https://blog.thoughtstuff.co.uk">thoughtstuff.co.uk</a>. You can also <a href="https://www.youtube.com/c/TomMorganTS?sub_confirmation=1" target="_blank" rel="noopener">subscribe to my YouTube channel</a> for videos about ACS (and much more!). </p> | |
| <h4>Disclaimer: This is a sample. It’s not meant for you to take and use without fully understanding what it’s doing. It’s definitely not meant for production use. You should understand the risks of hosting your own ACS instance and associated web-based entry point on the public internet before proceeding. If you end up sharing your access tokens, or there’s a bug in the code and you end up with a huge hosting bill, or find yourself unwittingly hosting other people’s rooms, you’re on your own. This sample code is provided under the <a href="https://opensource.org/licenses/MIT">MIT license</a>, which you should read in full (it’s 21 LOC).</h4> | |
| <script src="./bundle.js"></script> | |
| </body> | |
| </html> |
client.js
| import { CallClient, CallAgent, DeviceManager, LocalVideoStream, Renderer } from "@azure/communication-calling"; | |
| import { AzureCommunicationTokenCredential } from '@azure/communication-common'; | |
| const connectButton = document.getElementById('connect-button'); | |
| const disconnectButton = document.getElementById('disconnect-button'); | |
| const callStateElement = document.getElementById('call-state'); | |
| const destinationGroupElement = document.getElementById('destination-group-input'); | |
| const startVideoButton = document.getElementById('startvideo-button'); | |
| const stopVideoButton = document.getElementById('stopvideo-button'); | |
| const refreshRemoteMediaButton = document.getElementById('refreshRemoteMedia-button'); | |
| const remoteVideoCollection = document.getElementById('remotevideo'); | |
| const remoteScreenShareCollection = document.getElementById('remotescreenshare'); | |
| let call; | |
| let callAgent; | |
| let callClient; | |
| let localVideoStream; | |
| let localVideoRender; | |
| async function init() { | |
| callClient = new CallClient(); | |
| //get an access token to use | |
| const response = await fetch('YOUR ACS TOKEN ISSUING WEB FUNCTION URL HERE (WITH THE CODE). SEE DAY 3'); | |
| const responseJson = await response.json(); | |
| const token = responseJson.value.item2.token; | |
| const tokenCredential = new AzureCommunicationTokenCredential(token); | |
| callAgent = await callClient.createCallAgent(tokenCredential); | |
| connectButton.disabled = false; | |
| //get all the cameras, then choose the first one | |
| const deviceManager = await callClient.getDeviceManager(); | |
| const videoDevices = await deviceManager.getCameras(); | |
| const videoDeviceInfo = videoDevices[0]; | |
| localVideoStream = new LocalVideoStream(videoDeviceInfo); | |
| } | |
| init(); | |
| connectButton.addEventListener("click", () => { | |
| const destinationToCall = { meetingLink: destinationGroupElement.value}; | |
| const callOptions = {videoOptions: {localVideoStreams:[localVideoStream]}}; | |
| call = callAgent.join(destinationToCall,callOptions); | |
| call.on('stateChanged', () => { | |
| callStateElement.innerText = call.state; | |
| }) | |
| showLocalFeed(); | |
| // toggle button states | |
| disconnectButton.disabled = false; | |
| connectButton.disabled = true; | |
| startVideoButton.disabled = false; | |
| stopVideoButton.disabled = false; | |
| refreshRemoteMediaButton.disabled = false; | |
| }); | |
| disconnectButton.addEventListener("click", async () => { | |
| await call.hangUp(); | |
| // toggle button states | |
| disconnectButton.disabled = true; | |
| connectButton.disabled = false; | |
| callStateElement.innerText = '-'; | |
| }); | |
| startVideoButton.addEventListener("click", async () => { | |
| await call.startVideo(localVideoStream); | |
| showLocalFeed(); | |
| }); | |
| stopVideoButton.addEventListener("click", async () => { | |
| await call.stopVideo(localVideoStream); | |
| hideLocalFeed(); | |
| }); | |
| refreshRemoteMediaButton.addEventListener("click", async () => { | |
| remoteVideoCollection.innerHTML = remoteScreenShareCollection.innerHTML = ""; //start again | |
| call.remoteParticipants.forEach(function(participant) | |
| { | |
| SetUpRemoteParticipant(participant); | |
| }); | |
| }); | |
| async function showLocalFeed() { | |
| localVideoRender = new Renderer(localVideoStream); | |
| const view = await localVideoRender.createView(); | |
| document.getElementById("selfVideo").appendChild(view.target); | |
| }; | |
| async function hideLocalFeed() { | |
| localVideoRender.dispose(); | |
| document.getElementById("selfVideo").innerHTML = ""; | |
| }; | |
| async function SetUpRemoteParticipant(participant) { | |
| let videoStream = participant.videoStreams.find(function (s) { return s.mediaStreamType === "Video" }); | |
| let screenShareStream = participant.videoStreams.find(function (s) { return s.mediaStreamType === "ScreenSharing" }); | |
| if (videoStream.isAvailable) { | |
| RenderParticipantStream(videoStream, remoteVideoCollection); | |
| } | |
| if (screenShareStream.isAvailable) { | |
| RenderParticipantStream(screenShareStream, remoteScreenShareCollection); | |
| } | |
| }; | |
| async function RenderParticipantStream(stream, collection) | |
| { | |
| let renderer = new Renderer(stream); | |
| const view = await renderer.createView({scalingMode: "Fit"}); | |
| collection.appendChild(view.target); | |
| } |
Testing it out
Similar to previous days, this one is relatively straightforward to test out. As before, use a command-line to run this command then browse to the sample application in your browser (usually http://localhost:8080):
npx webpack-dev-server --entry ./client.js --output bundle.js --debug --devtool inline-source-map
Try joining a Teams meeting with at least one participant that is showing video and/or content. Once the ACS client has connected to the meeting, click the Refresh Remote Media button.
What’s the code doing?
As I said earlier, this sample is deliberately simplistic in not using any events. Instead, everything happens on the click of the Refresh button (line 78). I build up a collection of divs that contain the stream content each time, tearing down the previous ones before I start. Then, for each participant, I identify both the video and content-sharing streams (lines 99/100). These streams are always there, even if no video or content sharing is happening. I look at the isAvailable property and if it’s true, render the stream by creating a view and appending it to one of the divs that hold the stream content.
Today, we showed video and content sharing from other remote participants. Tomorrow is all about Chat. Links for all blog posts can be found at learnACS.dev. You can also subscribe to my YouTube channel for videos about ACS (and much more!).
Written by Tom Morgan
Tom is a Microsoft Teams Platform developer and Microsoft MVP who has been blogging for over a decade. Find out more.
Related posts:
Learn Azure Communication Services Day 12 – Showing Screen Share Content
Learn Azure Communication Services Day 13 – Managing the Participant List
Learn Azure Communication Services Day 9 – Muting and Unmuting
Learn Azure Communication Services Day 10 – Showing and Hiding Video
Learn Azure Communication Services Day 11 – Changing Audio & Video Devices
i.Renderer is not a constructor