Web co-streaming development guide

RTC co-streaming solution

The Live SDK uses ApsaraVideo Real-time Communication (RTC) for its co-streaming solution. This helps customers achieve ultra-low latency live interactions with many participants. The general flow is as follows:

• Before co-streaming: The streamer uses an RTC co-streaming ingest URL to ingest a stream. Viewers pull the stream from a CDN for playback.

• During co-streaming: A viewer switches to a co-streaming viewer, uses an RTC co-streaming ingest URL to ingest a stream, and uses an RTC co-streaming playback URL to pull the streamer's ultra-low latency stream. The streamer uses an RTC co-streaming playback URL to pull the co-streaming viewer's ultra-low latency stream.

• After co-streaming: The co-streaming viewer switches back to a regular viewer, stops RTC stream ingest and playback, and switches to pulling the stream from the CDN.

Notes

• Before you perform the operations described in this topic, you must configure the Web SDK Demo. For more information, see Try the Web Demo.

• In interactive mode, the ultra-low latency streams from the streamer or co-streaming viewers can be played only using the AlivcLivePlayer object. AlivcLivePlayer currently supports only the playback of ultra-low latency streams that have the artc://live.aliyun.com/play/ prefix.

• This topic describes how to use the web-based co-streaming feature. For more information about how to integrate the web co-streaming SDK, see Integrate the web co-streaming SDK.

  Important

  To ensure compatibility with the latest version of Chrome, the web co-streaming SDK was upgraded on March 15, 2024. If you have integrated an earlier version of the web co-streaming SDK, you must upgrade the SDK as soon as possible to prevent service interruptions. For more information about how to integrate the new SDK, see Integrate the web co-streaming SDK.

Step 1: Enable the co-streaming service

To use the co-streaming feature, you must first create a co-streaming application and configure a playback domain name. For more information, see Quick start for co-streaming.

Step 2: Generate co-streaming ingest and playback URLs

You can generate URLs in the console or by customizing them. These URLs include the ingest and playback URLs for the streamer and co-streaming viewers, and the CDN playback URL for regular viewers.

Method 1: Console Generation

If you want to quickly generate URLs for testing, you can use the URL generator in the console. For more information, see Co-streaming URL generator.

Method 2: Custom concatenation

The token in the URLs generated in the console is a temporary token for testing purposes. For use in a production environment, you can customize the ingest and playback URLs for better security. The token in a custom URL is calculated using the SHA256 encryption algorithm based on your SdkAppID, AppKey, channel ID, UserID, and Timestamp. This prevents attackers from forging a token to hijack your cloud service traffic. For more information about how to customize URLs, see Ingest and playback URLs for streamers and co-streaming viewers and CDN playback URL for regular viewers.

Step 3: Create a stream ingest object

The streamer and co-streaming viewers must perform this operation.

Create an AlivcLivePusher stream ingest object

Create an AlivcLivePusher stream ingest engine object and set the callbacks.

const pusher = new window.AlivcLivePush.AlivcLivePusher();
await pusher.init({
  resolution: window.AlivcLivePush.AlivcResolutionEnum.RESOLUTION_540P,
  fps: window.AlivcLivePush.AlivcFpsEnum.FPS_30,
  // audioOnly: true, for audio-only ingest scenarios. Not required for other scenarios.
});
pusher.info.on('pushstatistics', _stat => {
  // TODO... Handle stream ingest statistics.
  // console.log(_stat);
});
pusher.error.on('system', error => {
  console.log(error);
});
pusher.network.on('connectionlost', () => {
  console.log('connectionlost');
});

Ingest a stream in interactive co-streaming mode

Use the co-streaming ingest URL to ingest the stream.

await pusher.startPreview(videoElement);
await pusher.startPush("artc://live.aliyun.com/push/6375?timestamp=1661596947&token=XXX&userId=7135&sdkAppId=XXX"); // Ingest URL for the streamer or co-streaming viewer

Step 4: Create an AliPlayer for CDN playback

Regular viewers must perform this operation.

If the playback client needs to use AliPlayer for CDN playback, you must first obtain the CDN playback URL for regular viewers. For more information, see Step 2: Generate co-streaming ingest and playback URLs.

We recommend that you replace the RTMP playback URL with an HTTP-FLV URL. When you generate URLs in the ApsaraVideo Live console, both RTMP and HTTP-FLV URLs are created. These two protocols deliver the same content but use different network protocols. HTTP is the primary protocol on the Internet. CDNs, ISPs, and intermediate network devices are optimized for HTTP. The default HTTP ports 80 and 443 are also commonly whitelisted and less likely to be blocked. In contrast, RTMP is an older protocol. Its default port 1935 may be blocked by firewalls or other devices, which can cause connection issues. Therefore, in a complex network environment, HTTP-FLV offers better stability and performance, such as less stuttering and lower latency, than RTMP.

const player = new window.Aliplayer(
  {
    id: videoId, // video element ID
    source: url, // The playback URL. This can be a third-party streaming URL or a stream pulling URL from ApsaraVideo Live.
    isLive: true, // Specifies whether the stream is a live stream.
  },
  function (player: any) {
    console.log('The player is created.', player);
  }
);

Step 5: Viewer co-streaming

Streamer A ingests a stream over RTC

Streamer A starts to ingest a stream. Call the AlivcLivePusher stream ingest engine to start ingesting the stream of Streamer A. Sample code:

await pusher.startPreview(videoElement); // You can get videoElement using Document-related APIs.
await pusher.startPush("artc://live.aliyun.com/push/6375?timestamp=1661596947&token=XXX&userId=7135&sdkAppId=XXX"); // Ingest URL for the streamer or co-streaming viewer

Regular viewers play Streamer A's stream from CDN

All viewers watch the live stream of Streamer A. Call AliPlayer to start playing the stream of Streamer A. Sample code:

const player = new window.Aliplayer(
  {
    id: videoId, // video element ID
    source: 'rtmp://test.alivecdn.com/live/streamId?auth_key=XXX', // The playback URL. This can be a third-party streaming URL or a stream pulling URL from ApsaraVideo Live.
    isLive: true, // Specifies whether the stream is a live stream.
  },
  function (player: any) {
    console.log('The player is created.', player);
  }
);

Viewer D initiates co-streaming

• Viewer D creates an AlivcLivePusher object and starts to ingest a stream using the interactive ingest URL of Viewer D. Sample code:

  const pusher = new window.AlivcLivePush.AlivcLivePusher();
  pusher.init({
    resolution: window.AlivcLivePush.AlivcResolutionEnum.RESOLUTION_540P,
    fps: window.AlivcLivePush.AlivcFpsEnum.FPS_30,
    // audioOnly: true, for audio-only ingest scenarios. Not required for other scenarios.
  });
  pusher.info.on('pushstatistics', _stat => {
    // TODO... Handle stream ingest statistics.
    // console.log(_stat);
  });
  pusher.error.on('system', error => {
    console.log(error);
  });
  pusher.network.on('connectionlost', () => {
    console.log('connectionlost');
  });
  await pusher.startPreview(videoElement);  // You can get videoElement using Document-related APIs.
  await pusher.startPush("artc://live.aliyun.com/push/6375?timestamp=1661596947&token=XXX&userId=7135&sdkAppId=XXX"); // Ingest URL for the streamer or co-streaming viewer

• Streamer A obtains the interactive playback URL for co-streaming Viewer D, creates an AlivcLivePlayer object, and pulls the real-time stream of Viewer D. Sample code:

  const player = new window.AlivcLivePush.AlivcLivePlayer();
  // The artc:// URL is the playback URL for Viewer D's live stream.
  const playInfo = await player.startPlay("artc://live.aliyun.com/play/6375?timestamp=1661596947&token=XXX&userId=7135&sdkAppId=XXX", videoElement); 
  
  playInfo.on('userleft', () => {
    // Handle the event when Viewer D leaves.
  });
  
  playInfo.on('canplay', () => {
    // Handle the event when the video is ready to play.
  }

• At the same time, co-streaming Viewer D obtains the interactive playback URL for Streamer A, creates an AlivcLivePlayer object, and pulls the real-time stream of Streamer A. Sample code:

  const player = new window.AlivcLivePush.AlivcLivePlayer();
  // The artc:// URL is the playback URL for Streamer A's live stream.
  const playInfo = await player.startPlay("artc://live.aliyun.com/play/6375?timestamp=1661596947&token=XXX&userId=7135&sdkAppId=XXX", videoElement);
  
  playInfo.on('userleft', () => {
    // Handle the event when Streamer A leaves.
  });
  
  playInfo.on('canplay', () => {
    // Handle the event when the video is ready to play.
  }

Update the stream mix after co-streaming starts

To ensure that regular viewers B and C can see co-streaming Viewer D, Streamer A must initiate a stream mixing operation. This operation mixes the streams of Streamer A and co-streaming Viewer D into a single stream for CDN playback.

Streamer A calls the `setLiveMixTranscodingConfig` method to start a cloud stream mixing (transcoding) task and sets the objects to be mixed. The resolution, frame rate, and other parameters of the mixed video stream are based on the AlivcLivePushConfig settings that are specified when Streamer A created the AlivcLivePusher engine. For example, if Streamer A set the resolution to 720p in AlivcLivePushConfig when the AlivcLivePusher engine was created, the resolution of the mixed video stream is also 720p.

const resolution = pusher.getResolution();
const width = window.AlivcLivePush.AlivcResolution.GetResolutionWidth(resolution);
const height = window.AlivcLivePush.AlivcResolution.GetResolutionHeight(resolution);

const mixStreams = [];
const transcodingConfig = new window.AlivcLivePush.AlivcLiveTranscodingConfig();
const anchorMixStream = new window.AlivcLivePush.AlivcLiveMixStream(
  userA, // Change to the user ID of Streamer A
  0,
  0,
  width,
  height,
  1
);
// Add Streamer A's stream to the stream mixing list
mixStreams.push(anchorMixStream);

const audienceMixStream = new window.AlivcLivePush.AlivcLiveMixStream(
  userD, // Change to the user ID of co-streaming Viewer D
  100,
  200,
  200,
  300,
  2
);
// Add co-streaming Viewer D's stream to the stream mixing list
mixStreams.push(audienceMixStream);

transcodingConfig.mixStreams = mixStreams;
await pusher.setLiveMixTranscodingConfig(transcodingConfig);

This way, regular viewers can see the co-streaming interaction between Streamer A and co-streaming Viewer D.

Co-streaming Viewer D ends co-streaming

• Co-streaming Viewer D ends the co-streaming session. Call the `destroy` method of the AlivcLivePusher stream ingest engine to stop ingesting the stream and destroy the engine. Sample code:

  await pusher.destroy();
  pusher = null;

• At the same time, viewer D must call the stop playback interface on the AlivcLivePlayer stream pulling engine before destroying the engine. Sample code:

  await player.destroy();
  player = null;

• Co-streaming Viewer D switches back to a regular viewer and starts CDN playback as described in Step 4: Create an AliPlayer for CDN playback.

• Streamer A calls the `destroy` method of the AlivcLivePlayer stream pulling engine to stop playback and destroy the engine. Sample code:

  await player.destroy();
  player = null;

  At the same time, if Streamer A no longer needs to mix streams and regular viewers need to see only the stream of Streamer A, call the `setLiveMixTranscodingConfig` method and pass an empty value to switch to bypass mode.

  Warning

  If the streamer is still in the channel but no longer needs to mix streams, you must pass an empty value to cancel the stream mixing. After you initiate stream mixing, the cloud stream mixing module starts to work. If you do not cancel the stream mixing task in a timely manner, you may be charged for unnecessary billing.

  pusher.setLiveMixTranscodingConfig();