Quick integration

1. Initialize the SDK

1. Implement the EventHandler class. This class handles stream pulling callbacks. When data is received, the callback functions in this class are triggered.

2. Create an instance of EventHandler.

3. Call the CreateAliRTCEngine function in AliRTCEngineInterface.h to create an AliRTCEngine instance. During this process, the EventHandler instance is registered as the callback object for the engine. Then, call the methods of the AliRTCEngine instance to configure stream ingest and pulling.

4. If the AliRTCEngine instance fails to be created, destroy the EventHandler instance.

The following is an example:

// Implement the EventHandle class.
...

// Initialize the SDK.
AliRTCSdk::Central::FakeLinuxEngineEventHandler *engineEventHandler = new AliRTCSdk::Central::FakeLinuxEngineEventHandler();
std::string extra = "";
AliRTCSdk::Central::AliRTCEngineInterface *linuxEngine = AliRTCSdk::Central::CreateAliRTCEngine(engineEventHandler, 42000, 45000, logPath.c_str(), nullptr, false, extra.c_str());

if (linuxEngine == nullptr) {
    delete engineEventHandler;
    engineEventHandler = nullptr;
    return;
}

The parameters of the CreateAliRTCEngine function are as follows:

• EngineEventHandlerInterface * eventHandler: The callback object that processes callback logic.

• int lowPort: The lower limit of the port range. A port between lowPort and highPort is randomly selected for inter-process communication.

• int highPort: The upper limit of the port range.

• const char * logPath: The path where log files are saved when the SDK is running.

• const char * coreServicePath: The actual path of AliRtcCoreService.

• bool h5mode: The H5 compatibility mode. You can set this to false.

• const char * extra: A JSON string passed for extra SDK configurations.

2. Join a channel

First, understand the authentication feature. For more information, see Token-based authentication.

2.1 Multi-parameter API for joining a channel

1. Obtain authInfo to complete authentication with a token.

2. Configure JoinChannelConfig. For now, use the default configurations.

3. Call JoinChannel.

// Get authInfo.
AliRTCSdk::Central::AuthInfo authInfo;
authInfo.appid = "";
authInfo.channel = "";
authInfo.userid = "";
authInfo.username = "";
authInfo.nonce = "";
authInfo.token = "";
authInfo.timestamp = 1583582330; // Sample

int gslbCount = /* */;
authInfo.gslb_count = gslbCount;
const char *gslbArray[gslbCount];
if (gslbCount > 0)
{
    for(int i = 0; i < gslbCount; i++)
    {
        gslbArray[i] = "";
    }
    authInfo.gslb = gslbArray;
}

int agentCount = /* */;
authInfo.agent_count = agentCount;
const char *agentArray[agentCount];
if (agentCount > 0)
{
    for (int i = 0; i < agentCount; i++)
    {
        agentArray[i] = "";
    }
    authInfo.agent = agentArray;
}

// Configure JoinChannelConfig.
AliRTCSdk::Central::JoinChannelConfig config;

// Enable automatic stream ingest. You can also use manual stream ingest mode, where you join a channel first and then start stream ingest.
config.publishMode = AliRTCSdk::Central::PublishAutomatically;

// Disable automatic recording if it is not needed.
config.subscribeMode = AliRTCSdk::Central::SubscribeManually;

// Automatically subscribe to remote video streams after joining the channel. For more information, see AliRTCSdk::Central::VideoFormat.
config.subscribeVideoFormat = 1;

// Automatically subscribe to remote audio streams after joining the channel. For more information, see AliRTCSdk::Central::AudioFormat.
config.subscribeAudioFormat = 2;
// To subscribe to the mixed audio stream of all remote users, set config.subAudioMode = 1.

// Set the channel profile. You usually need to manually set it to interactive mode.
config.channelProfile = AliRTCSdk::Central::ChannelProfileInteractiveLive;

// Specify whether to ingest the camera stream. Set to false to not ingest the camera video stream.
linuxEngine->PublishLocalVideoStream(true);

// Specify whether to ingest the audio stream. Set to false to not ingest the audio stream.
linuxEngine->PublishLocalAudioSrtream(true);

// Set the encoding configuration for the camera video stream.
AliRTCSdk::Central::AliEngineVideoEncoderConfiguration videoConfig;
linuxEngine->SetVideoEncoderConfiguration(videoConfig);
  
// Enable YUV input and use the camera stream for ingest.
linuxEngine->SetExternalVideoSource(true, AliRTCSdk::Central::VideoSourceCamera, AliRTCSdk::Central::RenderModeAuto);

// Enable PCM input.
// The second parameter is the sample rate of the PCM data. Set it as needed.
// The third parameter is the number of channels for the PCM data. Set it as needed.
linuxEngine->SetExternalAudioSource(true, 16000, 2);

// Set the user mode: streamer or viewer.
// Viewer (AliEngineClientRoleLive): Only pulls streams, does not ingest streams. Remote users are not notified when a viewer goes online.
// Streamer (AliEngineClientRoleInteractive): Can ingest streams. Remote users are notified when a streamer goes online, regardless of whether they are ingesting a stream.
linuxEngine->SetClientRole(AliRTCSdk::Central::AliEngineClientRoleInteractive);

// Join the channel.
linuxEngine->JoinChannel(authInfo, config);

2.2 Single-parameter API for joining a channel

1. Obtain the token and basic information such as the username and channel number.

2. Configure JoinChannelConfig.

3. Call the overloaded JoinChannel.

// Get the basic information required to join the channel.
const char* token = "";    // Passed in by the service.
const char* channel = "";  // Channel number
const char* userid = "";   // User ID
const char* username = ""; // Username
// Configure JoinChannelConfig.
AliRTCSdk::Central::JoinChannelConfig config;
// ..... Configuration options are the same as above.
linuxEngine->JoinChannel(token, channel, userid, username, config);

Note

Do not call JoinChannel repeatedly.

The demo shows how to use GenerateToken to simulate an app server generating parameters for joining a channel. This API is provided to simplify development and testing.

In a production environment, obtain the token by interacting with your app server to prevent your AppKey from being leaked.

3. Manually ingest a stream

The following APIs set whether to ingest audio and video streams.

   /**
    * @brief Specifies whether to ingest the local video (camera) stream.
    * @param enabled Specifies whether to enable or disable local video stream ingest.
    - true: Enable video stream ingest.
    - false: Disable video stream ingest.
    * @return
    - 0: The setting is successful.
    - <0: The setting failed. An error code is returned.
    * @note By default, the SDK ingests the video stream. You can also call this API before joining a channel to change the default value. The setting takes effect after you successfully join the channel.
    */
    virtual int PublishLocalVideoStream(bool enabled) = 0;

   /**
    * @brief Specifies whether to ingest the local audio stream.
    * @param enabled Specifies whether to enable or disable local audio stream ingest.
    - true: Enable audio stream ingest.
    - false: Disable audio stream ingest.
    * @return
    - 0: The setting is successful.
    - <0: The setting failed. An error code is returned.
    * @note By default, the SDK ingests the audio stream. You can also call this API before joining a channel to change the default value. The setting takes effect after you successfully join the channel.
    */
    virtual int PublishLocalAudioStream(bool enabled) = 0;

4. Ingest a YUV video stream

/**
 * @brief Enables an external video input source.
 * @param enable true: enable, false: disable.
 * @param useTexture Specifies whether to use texture mode. Currently, only false is supported.
 * @param type The stream type.
 * @note After enabling, use the PushExternalVideoFrame API to input video data.
 */
virtual int SetExternalVideoSource(bool enable, bool useTexture, AliRTCSdk::Central::VideoSource sourceType) = 0;

/**
 * @brief Inputs an external video.
 * @param frame The frame data.
 * @param type The stream type.
 * @param For supported input video types, see VideoDataFormat.
 */
virtual int PushExternalVideoFrame(AliRTCSdk::Central::VideoDataSample *frame, AliRTCSdk::Central::VideoSource sourceType) = 0;

5. Ingest a PCM audio stream

/**
 * @brief Sets whether to enable stream ingest from an external audio source.
 * @param enable true: enable, false: disable.
 * @param sampleRate The sample rate, such as 16k or 48k.
 * @param channelsPerFrame The number of channels per frame.
 * @return A value greater than or equal to 0 indicates success. A value less than 0 indicates failure.
 * @note You can call SetExternalAudioPublishVolume to set the volume for audio stream ingest.
 */
virtual int SetExternalAudioSource(bool enable, unsigned int sampleRate,  unsigned int channelsPerFrame) = 0;

/**
 * @brief Inputs external audio data for stream ingest.
 * @param audioSamples The audio data.
 * @param sampleLength The length of the audio data.
 * @param timestamp The timestamp.
 * @return A value less than 0 indicates failure. If ERR_AUDIO_BUFFER_FULL is returned, retry the delivery after an interval equal to the data delivery time.
 */
virtual int PushExternalAudioFrameRawData(const void* audioSamples, unsigned int sampleLength, long long timestamp) = 0;

/**
 * @brief Sets the mixing volume for the ingested external audio stream.
 * @param vol The volume. The range is 0 to 100.
 */
virtual int SetExternalAudioPublishVolume(int volume) = 0;

/**
 * @brief Gets the mixing volume for the ingested external audio stream.
 * @return vol The volume.
 */
virtual int GetExternalAudioPublishVolume() = 0;

6. Manually subscribe to a stream

/**
 * @brief Stops or resumes subscribing to the audio stream of a specific remote user. This API is valid only when called within a channel.
 * @param uid The user ID. This is a unique identifier assigned by the app server.
 * @param sub Specifies whether to subscribe to the remote user's audio stream.
 * - true: Subscribe to the specified user's audio stream.
 * - false: Stop subscribing to the specified user's audio stream.
 * @return
 * - 0: Success.
 * - A non-zero value: Failure.
 */
virtual int SubscribeRemoteAudioStream(const char* uid, bool sub) = 0;

/**
* @brief Stops or resumes subscribing to the video stream of a remote user. This API is valid only when called within a channel.
* @param uid The user ID. This is a unique identifier assigned by the app server.
* @param track The video stream type.
* - AliEngineVideoTrackNo: Invalid parameter. The setting has no effect.
* - AliEngineVideoTrackCamera: Camera stream.
* - AliEngineVideoTrackScreen: Screen sharing stream.
* - AliEngineVideoTrackBoth: Camera stream and screen sharing stream.
* @param sub Specifies whether to subscribe to the remote user's video stream.
* - true: Subscribe to the specified user's video stream.
* - false: Stop subscribing to the specified user's video stream.
* @return
* - 0: The setting is successful.
* - <0: The setting failed.
* @note
*/
virtual int SubscribeRemoteVideoStream(const char* uid, AliRTCSdk::Central::VideoTrack videoTrack, bool sub) = 0;

7. Data callbacks

7.1 Audio

If you set audioFormat to AudioFormatPcmBeforMixing, the OnSubscribeAudioFrame callback function of the EventHandler instance is triggered when an audio frame is received.

• uid: Indicates the remote user from whom the audio frame is received. This helps distinguish between different subscribed audio streams.

• frame: The received audio frame in PCM format.

void OnSubscribeAudioFrame(const std::string& uid, const AliRTCSdk::Central::AudioFrame * frame)

If you select the AudioFormatMixedPcm mode, the OnSubscribeMixAudioFrame callback function of the EventHandler instance is triggered when an audio frame is received.

void OnSubscribeMixAudioFrame(const AliRTCSdk::Central::AudioFrame * frame)

7.2 Video

There are no mixed streams for video frame subscriptions. If you set videoFormat to VideoFormatH264, the OnRemoteVideoSample callback function of the EventHandler instance is triggered when a video frame is received.

• uid: Indicates the remote user from whom the video frame is received. This helps distinguish between different subscribed video streams.

• frame: The received video frame in YUV format.

void OnRemoteVideoSample(const char * uid, const AliRTCSdk::Central::VideoFrame * frame)

8. Leave a channel

/* Stop stream ingest */
linuxEngine->PublishLocalVideoStream(false);
linuxEngine->PublishLocalAudioStream(false);

To remain in the channel after stopping stream ingest, manually call this API. Leaving the channel directly also stops stream ingest.

linuxEngine->LeaveChannel();

9. Destroy the SDK

linuxEngine->Release();
linuxEngine = nullptr;

delete linuxEventHandler;
linuxEventHandler = nullptr;

1. Data Channel

Send a message using SendDataChannelMessage:

/**
 * @note Before sending a message, you must call linuxEngine->SetParameter("{\"data\":{\"enablePubDataChannel\":true,\"enableSubDataChannel\":true}}"); to open the Data Channel.
 */
std::string message = "This is data channel message";
AliRTCSdk::Central::AliEngineDataChannelMsg dataChannelMsg;
dataChannelMsg.data = (void*)message.c_str();
dataChannelMsg.dataLen = message.length();
dataChannelMsg.networkTime = t0; // Network time field. You can also use a custom value. This does not affect message sending and receiving.
dataChannelMsg.progress = 0; // Reserved field.
dataChannelMsg.type = AliRTCSdk::Central::AliEngineDataChannelProgress;

linuxEngine->SendDataChannelMessage(dataChannelMsg);

When a Data Channel message is received, a callback is triggered:

/**
 * @brief Obtains remote data from the Data Channel.
 * @param uid The ID of the remote user who sent the message.
 * @param msg The message from the remote user.
 */
virtual void OnDataChannelMsg(const char* uid, AliRTCSdk::Central::AliEngineDataChannelMsg& msg) {}

2. SEI

/**
 * @brief Sends Supplemental Enhancement Information (SEI). The maximum length is 4 × 1024 bytes. It is used for transmitting small amounts of business data.
 * @param message The content of the extension message. You can transmit 4 KB of data.
 * @param length The length of the extension message in bytes.
 * @param repeatCount The number of repetitions. This prevents message loss due to packet loss.
 * @param delay The delay before sending, in milliseconds.
 * @param isKeyFrame Specifies whether to add SEI only to keyframes.
 * @return 0: Success. A value less than 0 indicates failure. For example, -1 indicates an internal SDK error.
*/
int SendMediaExtensionMsg(const char* message,
                          size_t length,
                          int repeatCount,
                          uint32_t delay,
                          bool isKeyFrame);

Send a message using SendMediaExtensionMsg:

std::string seiMsg = "This is SEI message";
linuxEngine->SendMediaExtensionMsg(seiMsg.c_str(), seiMsg.length(), 3, 0 ,false);

The interval between keyframes is long. To send and receive SEI messages frequently, set isKeyFrame to false. To ensure that messages are delivered, set the number of repetitions to a value greater than 1.

When an SEI message is received, a callback is triggered:

/**
 * @brief Callback for receiving a media extension message.
 * @param uid The user ID of the sender.
 * @param message The content of the extension message.
 * @param size The length of the extension message.
 * @note When one client sends a message using {@link SendMediaExtensionMsg}, other clients receive the data through this callback.
 */
virtual void OnMediaExtensionMsgReceived(const char* userid, const char* message, size_t size) {}

1. Disable Audio Ranking

Audio Ranking is a feature for multi-user voice chat rooms that subscribes to only the few audio streams with the highest volume. This makes the sound clearer for listeners. This feature is enabled by default. If your scenario requires subscribing to all audio streams, use the following field to disable the Audio Ranking feature:

extraJson["user_specified_disable_audio_ranking"] = "true";

2. Enable AAC transcoding for audio subscriptions

For scenarios that require archiving pulled streams, the SDK can transcode subscribed remote audio to the AAC format and return it through the OnSubscribeAudioAac() callback function. You can use the items in AudioTranscodingCodec to select the format for the remote audio (PCM, AAC, or both). If you enable AAC transcoding, you can choose whether to resample the audio to reduce CPU and storage usage. If user_specified_audio_observer_resample_rate is set to 0, resampling is not performed.

extraJson["user_specified_audio_observer_codec"] = 
    (int)AliRTCSdk::Central::AudioTranscodingCodecBothPcmAndAac;
extraJson["user_specified_audio_observer_codec_format"] = 0; // Currently, only the ADTS format is supported.
extraJson["user_specified_audio_observer_resample_rate"] = 16000; // Resampling rate
extraJson["user_specified_audio_observer_codec_bitrate"] = 64000; // Target bitrate for transcoding

3. Directly get H.264 streams for video subscriptions

For scenarios that require archiving pulled streams, the SDK can directly get the H.264 stream of a remote video. You can use the items in VideoTranscodingCodec to select the format for the remote video (YUV, H.264 stream, or both).

extraJson["user_specified_video_observer_codec"] = 
    (int)AliRTCSdk::Central::VideoTranscodingCodecBothYuvAndH264;
extraJson["user_specified_video_observer_codec_format"] = 0; // Currently, only the Annex B format is supported.