Local stream mixing

Procedure

1. Obtain the video effect manager instance.

   const videoEffectManager = pushClient.getVideoEffectManager();

   For more information, see the getVideoEffectManager API.

2. Enable the local stream mixing mode.

   By default, the Push SDK supports only one video stream. After you enable local stream mixing, you can capture multiple video streams and one audio stream. You can also adjust the position and size of these video streams as needed.

   Enable this mode before you capture audio and video.

   videoEffectManager.enableMixing(true);

   For more information, see the enableMixing API.

3. Set the mixed stream output parameters.

   You can set parameters such as the resolution and frame rate for the mixed output stream.

   You can call setVideoQuality to set the output parameters before you enable local stream mixing. Do not call this method after local stream mixing is enabled.

   videoEffectManager.setMixingConfig({
     videoWidth: 1280,
     videoHeight: 720,
     videoFramerate: 30
   })

   For more information, see the setMixingConfig API.

4. Capture multiple streams.

   You can capture multiple camera streams, screen sharing streams, and one audio stream. The `streamId` that is returned after capture is required to configure the layout.

   const cameraStreamId = await pushClient.startCamera();
   const screenStreamId = await pushClient.startScreenCapture();
   const micStreamId = await pushClient.startMicrophone();

   For more information, see the startCamera, startMicrophone, and startScreenCapture APIs.

5. Set the video layout.

   You can adjust the size, position, and level of different video streams.

   This example shows one screen sharing stream and one camera stream. The screen sharing stream fills the entire video, and the camera stream is in the upper-left corner. Note that the values of `x` and `y` are half the width and height of the video source because the positioning origin of the video source is at its center, not its upper-left corner.

   videoEffectManager.setLayout([
     { streamId: screenStreamId, x: 1280/2, y: 720/2, width: 1280, height: 720, zOrder: 1 },
     { streamId: cameraStreamId, x: 320/2, y: 180/2, width: 320, height: 180, zOrder: 2 },
   ]);

   For more information, see the setLayout API.

6. Start stream ingest.

   You can ingest the locally mixed stream to the remote server.

   pushClient.startPush('artc://domain/appName/streamName');

   For more information, see the startPush API.

7. Mix audio streams.

   In mixing mode, you can mix the audio from the microphone and screen sharing into a single audio stream and set their volumes separately. The following example captures audio from the microphone and screen sharing and reduces the microphone volume by half. For more information, see the setVolume API.

   // Enable the microphone
   const micStreamId = await pushClient.startMicrophone();
   // Enable screen sharing and capture audio
   const screenStreamId = await pushClient.startScreenCapture(true);
   // Get the audio effect manager
   const audioEffectManager = pushClient.getAudioEffectManager();
   // Set the microphone volume
   audioEffectManager.setVolume(50, micStreamId);

   On some devices, stream mixing may produce poor results, such as noise or interruptions. This can be caused by the microphone's noise suppression or auto gain control. To achieve better mixing results, you can disable the 3A audio algorithms as shown in the following code. For more information, see the startMicrophone API.

   const micStreamId = await pushClient.startMicrophone({
     echoCancellation: false, autoGainControl: false, noiseSuppression: false
   });