Implement screen sharing on Harmony

Publish screen and camera streams

If your scenario requires publishing both the camera stream and the screen sharing stream, follow these steps:

• Call publishLocalVideoStream(true) to enable camera stream publishing. This is the default behavior and can be omitted.

• After joining the channel, call startScreenShare to start screen capture and publish the screen sharing stream.

mAliRtcEngine.publishLocalVideoStream(true);

Set the screen stream encoder configuration

To customize the encoding properties of the screen sharing video stream, call setScreenShareEncoderConfiguration. You can configure properties such as resolution, frame rate, bitrate, GOP, and video rotation.

The following table describes the configuration parameters.

The following code shows an example:

private aliRtcScreenShareConfig: AliRtcVideoEncoderConfiguration | undefined;

this.aliRtcScreenShareConfig = new AliRtcVideoEncoderConfiguration();
this.aliRtcScreenShareConfig.dimensions.width = 720
this.aliRtcScreenShareConfig.dimensions.height = 1080
this.aliRtcScreenShareConfig.frameRate = this.screenShareEncoderConfig.fps;
this.aliRtcScreenShareConfig.bitrate = this.screenShareEncoderConfig.bitrate;
this.aliRtcScreenShareConfig.keyFrameInterval = this.screenShareEncoderConfig.gop;
this.aliRtcScreenShareConfig.forceStrictKeyFrameInterval = this.screenShareEncoderConfig.forceGOP ? 1 : 0;
// Set the screen share encoder configuration
this.rtcEngine.setScreenShareEncoderConfiguration(this.aliRtcScreenShareConfig);

View the remote screen stream

When a user starts screen sharing, other users in the channel receive an onRemoteTrackAvailableNotify callback notification. Through this callback, the client can detect real-time changes to a remote user's audio and video streams, including when a camera video stream or a screen sharing video stream is published or stopped. The application can then use the callback parameters to dynamically create or remove corresponding rendering views to display and manage the remote screen sharing content.

The AliRtcVideoTrack enum in the onRemoteTrackAvailableNotify callback indicates the state of the remote video stream and has the following values:

• AliRtcVideoTrackNo (0): No video stream. The remote user is not publishing a video stream.

• AliRtcVideoTrackCamera (1): Camera video stream only.

• AliRtcVideoTrackScreen (2): Screen sharing video stream only.

• AliRtcVideoTrackBoth (3): The remote user is publishing both camera and screen sharing video streams.

Your application must check this enum to adjust the UI dynamically, for example, by showing or hiding views for the camera and screen streams.

this.rtcEventListener.onRemoteTrackAvailableNotify((userId: string, audioTrack: AliRtcAudioTrack,
videoTrack: AliRtcVideoTrack) => {
console.info(`Remote A/V stream available: userId=${userId}, audioTrack=${audioTrack}, videoTrack=${videoTrack}`);
// If the video track is Screen or Both, call setRemoteViewConfig to configure the display for the screen sharing stream.
if (videoTrack === AliRtcVideoTrack.AliRtcVideoTrackCamera) {
  this.viewRemoteVideo(userId, AliRtcVideoTrack.AliRtcVideoTrackCamera);
  this.removeRemoteVideo(userId, AliRtcVideoTrack.AliRtcVideoTrackScreen);
} else if (videoTrack === AliRtcVideoTrack.AliRtcVideoTrackScreen) {
  this.viewRemoteVideo(userId, AliRtcVideoTrack.AliRtcVideoTrackScreen);
  this.removeRemoteVideo(userId, AliRtcVideoTrack.AliRtcVideoTrackCamera);
} else if (videoTrack === AliRtcVideoTrack.AliRtcVideoTrackBoth) {
  this.viewRemoteVideo(userId, AliRtcVideoTrack.AliRtcVideoTrackCamera);
  this.viewRemoteVideo(userId, AliRtcVideoTrack.AliRtcVideoTrackScreen);
} else if (videoTrack === AliRtcVideoTrack.AliRtcVideoTrackNo) {
  this.removeAllRemoteVideo(userId);
}
});



// View the remote video
private viewRemoteVideo(uid: string, videoTrack: AliRtcVideoTrack): void {

// Check if the same stream already exists
const existingIndex = this.remoteVideoStreams.findIndex(
  s => s.uid === uid && s.videoTrack === videoTrack
);

if (existingIndex >= 0) {
  return;
}

// Create a new remote video stream object
const newStream: RemoteVideoStream = {
  uid,
  videoTrack,
  canvas: new AliRtcVideoCanvas(),
  xcomponentController: new XComponentController(),
  surfaceId: ''
};

// Add to the array and trigger a UI update
this.remoteVideoStreams = [...this.remoteVideoStreams, newStream];


// Immediately try to set up the video
setTimeout(() => {
  this.setupRemoteVideo(newStream);
}, 100);
}

// Set up the remote video
private setupRemoteVideo(stream: RemoteVideoStream): void {
if (!this.rtcEngine) {
  console.error('ARTC engine not initialized');
  return;
}

try {
  // Get the surfaceId of the XComponent
  stream.surfaceId = stream.xcomponentController.getXComponentSurfaceId();
  if (!stream.surfaceId) {
    console.warn(`Failed to get surfaceId: ${stream.uid}_${stream.videoTrack}`);
    return;
  }

  // Configure the video canvas
  if (!stream.canvas) {
    stream.canvas = new AliRtcVideoCanvas();
  }

  stream.canvas.surfaceId = stream.surfaceId;
  stream.canvas.renderMode = AliRtcRenderMode.AliRtcRenderModeAuto;
  stream.canvas.mirrorMode = AliRtcRenderMirrorMode.AliRtcRenderMirrorModeAllNoMirror;

  // Key step: Set the remote view configuration
  this.rtcEngine.setRemoteViewConfig(
    stream.canvas,
    this.componentController,
    stream.uid,
    stream.videoTrack
  );
  // Make sure to subscribe to this video stream
  if (stream.videoTrack === AliRtcVideoTrack.AliRtcVideoTrackCamera) {
    this.rtcEngine.subscribeRemoteVideoStream(stream.uid, stream.videoTrack, true);
  } else if (stream.videoTrack === AliRtcVideoTrack.AliRtcVideoTrackScreen) {
    // Special handling for the screen sharing stream
    this.rtcEngine.subscribeRemoteVideoStream(stream.uid, stream.videoTrack, true);
    console.info(`Subscribed to screen sharing stream: ${stream.uid}`);
  }

} catch (error) {
}
}

Start screen capture

Call startScreenShare to start capturing the device's screen and publishing the capture to the channel. Configure the following parameter based on your use case:

1. mode: The screen sharing mode. You can choose to share audio only, video only, or both.

Note: When your application calls this method, the system displays an authorization pop-up asking for permission to capture the screen. The user must grant this permission before screen sharing can begin.

this.rtcEngine.startScreenShare(mode);

Stop screen capture

Call stopScreenShare to stop capturing and publishing the screen and to release the associated resources.

this.rtcEngine.stopScreenShare();