C++

Import SDK

Unzip the Linux SDK package and open the Cpp folder. The directory structure is as follows:

├── Release
│   ├── include ## This directory contains header files to import
│   │   ├── AliRTCEngineCentralInterface.h ## Used for multi-process version
│   │   ├── AliRTCSdkDefineCentral.h ## Used for multi-process version
│   │   ├── AliRTCEngineInterface.h ## Used for single-process version
│   │   ├── AliRTCLinuxSdkDefine.h ## Used for single-process version
│   │   ├── AliRTCMediaPlayerInterface.h 
│   │   └── IAliRTCEngine.h
│   └── lib ## This directory contains SDK dynamic libraries to link
│       ├── AliRtcCoreService ## Used for multi-process version
│       ├── libAliRtcCentralEngine.so ## Used for multi-process version
│       ├── libAliRtcLinuxEngine.so ## Used for single-process version
│       └── libonnxruntime.so.1.16.3
└── Demo ## Simple example
    ├── CMakeLists.txt
    ├── fake_linux_event_listener.cc ## Contains callback logic for audio and video stream pulling
    ├── fake_linux_event_listener.h
    └── simple_main.cc ## The main demo body, including initialization, joining a channel, stream ingest, and other operations

Note:

• The Release directory contains header files to import (include) and SDK dynamic libraries to link (lib). APIs and data structures are declared in the header files within the include directory. Configure the correct dynamic library link address, for example: export LD_LIBRARY_PATH=./lib.

• The Demo directory provides a simple example. The file simple_main.cc contains the main demo logic, including initialization, joining a channel, and stream ingest. The file fake_linux_event_listener contains the callback logic for audio and video stream pulling.

Add the header file path and dynamic library link to your project’s CMakeLists.txt:

include_directories(path/to/Release/include)
link_directories(path/to/Release/lib)
target_link_libraries(your_target AliRtcCentralEngine)

Set the dynamic library search path before running:

export LD_LIBRARY_PATH=/path/to/Release/lib:$LD_LIBRARY_PATH

Implement Event Callback Class

Inherit from EngineEventHandlerInterface to implement event callbacks. These callbacks handle notifications from the SDK:

#include "AliRTCEngineCentralInterface.h"

class VideoCallEventHandler : public AliRTCSdk::Central::EngineEventHandlerInterface {
public:
    // Callback for join channel result
    void OnJoinChannelResult(int result, const char* channel, const char* userId) override {
        if (result == 0) {
            fprintf(stdout, "[OnJoinChannelResult] User %s joined channel %s successfully\n",
                    userId, channel);
        } else {
            fprintf(stdout, "[OnJoinChannelResult] Failed to join, error: %d\n", result);
        }
    }

    // Notification for remote user online
    void OnRemoteUserOnLineNotify(const char* uid) override {
        fprintf(stdout, "[OnRemoteUserOnLineNotify] uid: %s\n", uid);
    }

    // Notification for remote user audio/video track changes (handle remote stream subscription logic here)
    void OnRemoteTrackAvailableNotify(const char* uid,
                                      AliRTCSdk::Central::AudioTrack audioTrack,
                                      AliRTCSdk::Central::VideoTrack videoTrack) override {
        fprintf(stdout, "[OnRemoteTrackAvailableNotify] uid: %s, audio: %d, video: %d\n",
                uid, (int)audioTrack, (int)videoTrack);
    }

    // Receive remote mixed audio PCM frames (triggered when subscribeAudioFormat = AudioFormatMixedPcm)
    void OnSubscribeMixAudioFrame(const AliRTCSdk::Central::AudioFrame* frame) override {
        // Process received audio data here, such as writing to a file or sending to a playback device
    }

    // Receive remote video YUV frames (triggered when subscribeVideoFormat = VideoFormatYUV)
    void OnRemoteVideoSample(const char* uid, const AliRTCSdk::Central::VideoFrame* frame) override {
        // Process received video data here, such as writing to a file or sending to a display device
    }

    // TODO: Handle this. If an unrecoverable error occurs, rejoining the channel is recommended.
    void OnError(AliRTCSdk::Central::ERROR_CODE error_code) override {
        fprintf(stdout, "[OnError] error_code: 0x%X\n", error_code);
    }
};

Create and Initialize Engine

Call CreateAliRTCEngine to create an engine instance and register the event callback object. Then call methods of the AliRTCEngine instance to configure stream ingest and stream pulling.

If the AliRTCEngine instance fails to create, destroy the EventHandler instance.

Note: Each AliRTCEngine creation starts a process corresponding to a virtual user.

VideoCallEventHandler* eventHandler = new VideoCallEventHandler();

// extra: disable audio ranking to receive all remote audio streams
std::string extra = "{\"user_specified_disable_audio_ranking\":\"true\"}";

AliRTCSdk::Central::AliRTCEngineInterface* engine = AliRTCSdk::Central::CreateAliRTCEngine(
    eventHandler,
    42000, 45000,          // IPC port range for AliRtcCoreService
    "/tmp",                // Log file directory
    nullptr,               // AliRtcCoreService path; nullptr = same dir as executable
    false,                 // h5mode: set true for Web interoperability
    extra.c_str()
);

if (!engine) {
    fprintf(stderr, "Failed to create RTC engine\n");
    delete eventHandler;
    return -1;
}

The parameters of the CreateAliRTCEngine function are described below:

• EngineEventHandlerInterface * eventHandler: The callback object that handles event notifications.

• int lowPort: The lower bound of the port range. A port is auto-assigned from the range between lowPort and highPort for inter-process communication.

• int highPort: The upper bound of the port range.

• const char * logPath: The directory for SDK runtime log files.

• const char * coreServicePath: The path to `AliRtcCoreService`.

• bool h5mode: H5 compatibility mode. Set to false in most cases.

• const char * extra: A JSON-formatted string for additional SDK configuration.

Set Audio and Video Properties

Call SetClientRole to set the user role. Call SetVideoEncoderConfiguration to configure video encoding parameters.

// Call SetClientRole to set the user role to interactive mode (streamer), enabling both publishing and subscribing
engine->SetClientRole(AliRTCSdk::Central::AliEngineClientRoleInteractive);

// Call SetVideoEncoderConfiguration to set video encoding parameters
AliRTCSdk::Central::AliEngineVideoEncoderConfiguration videoConfig;
videoConfig.dimensions.width  = 720;
videoConfig.dimensions.height = 1280;
videoConfig.frameRate         = (AliRTCSdk::Central::AliEngineFrameRate)15;
videoConfig.bitrate           = 1200;
engine->SetVideoEncoderConfiguration(videoConfig);

Set Stream Ingest and Stream Pulling Properties

Configure audio and video publishing and subscribing behavior, and enable external audio and video source mode.

// Call PublishLocalVideoStream / PublishLocalAudioStream to enable local audio and video publishing
engine->PublishLocalVideoStream(true);
engine->PublishLocalAudioStream(true);

// Linux has no built-in camera/microphone. Call SetExternalVideoSource to enable an external video source,
// and input YUV frame data via PushExternalVideoFrame
engine->SetExternalVideoSource(true, AliRTCSdk::Central::VideoSourceCamera,
                               AliRTCSdk::Central::RenderModeAuto);

// Call SetExternalAudioSource to enable an external audio source, and input PCM frame data via PushExternalAudioFrameRawData
engine->SetExternalAudioSource(true, 16000 /* sample rate */, 1 /* channels */);

Configure the subscription mode when joining a channel (set in JoinChannelConfig). Two audio subscription formats are available:

AliRTCSdk::Central::JoinChannelConfig joinConfig;
joinConfig.channelProfile       = AliRTCSdk::Central::ChannelProfileInteractiveLive;
joinConfig.publishMode          = AliRTCSdk::Central::PublishAutomatically;   // Auto stream ingest
joinConfig.subscribeMode        = AliRTCSdk::Central::SubscribeAutomatically; // Auto subscribe
joinConfig.subscribeVideoFormat = AliRTCSdk::Central::VideoFormatYUV;         // Receive YUV video frames, triggering OnRemoteVideoSample

// There are two options for audio subscription format:
// AudioFormatMixedPcm: Receive mixed PCM for the entire channel, triggering OnSubscribeMixAudioFrame
// AudioFormatPcmBeforMixing: Receive individual PCM streams for each remote user, triggering OnSubscribeAudioFrame (including uid)
joinConfig.subscribeAudioFormat = AliRTCSdk::Central::AudioFormatMixedPcm;

Join Channel

Before you join a channel, understand token-based authentication. For details, see Token Authentication.

Note

Two overloaded versions of JoinChannel are available. The single-parameter JoinChannel is syntactic sugar built on top of the multi-parameter version. It removes the need to pass nonce and timestamp when joining, making it easier to use when you directly obtain a token.

JoinChannel provides two overloaded versions. We recommend using the single-parameter JoinChannel interface.

const char* token    = token.c_str(); // Base64 Token generated in step 3
const char* channel  = "your_channel_id";
const char* userid   = "your_user_id";
const char* username = "your_user_id";

// Call JoinChannel single-parameter overload to join the channel
engine->JoinChannel(token, channel, userid, username, joinConfig);

AliRTCSdk::Central::AuthInfo authInfo;
authInfo.appid     = "your_app_id";
authInfo.channel   = "your_channel_id";
authInfo.userid    = "your_user_id";
authInfo.username  = "your_user_id";
authInfo.nonce     = "";
authInfo.token     = "";          // SHA-256 hash value generated by calling CreateMultiParameterToken
authInfo.timestamp = ARTCTokenHelper::GetExpirationTimestamp();

// Call JoinChannel multi-parameter overload to join the channel
engine->JoinChannel(authInfo, joinConfig);

Push External Video Frames

Linux does not provide a built-in camera driver interface. Use PushExternalVideoFrame to input YUV video data to the SDK. The following example reads and pushes frame data in a loop from an I420-format YUV file. In production, replace this with camera driver or video decoder output.

```cpp
std::thread videoThread([&]() {
    const int    width     = 720;
    const int    height    = 1280;
    const int    fps       = 15;
    const size_t frameSize = width * height * 3 / 2; // I420

    FILE* fin      = fopen("/tmp/test_720p.yuv", "rb");
    void* buf      = malloc(frameSize);
    int   frameCount = 0;
    int64_t startTime = currentTimeMs();

    while (running) {
        size_t readSize = fread(buf, 1, frameSize, fin);
        if (readSize != frameSize) {
            fseek(fin, 0, SEEK_SET); // loop when file ends
            continue;
        }

        AliRTCSdk::Central::VideoDataSample sample;
        sample.data      = (unsigned char*)buf;
        sample.format    = AliRTCSdk::Central::VideoDataFormatI420;
        sample.width     = width;
        sample.height    = height;
        sample.strideY   = width;
        sample.strideU   = width / 2;
        sample.strideV   = width / 2;
        sample.dataLen   = frameSize;
        sample.timeStamp = frameCount * 1000 / fps;
        sample.rotation  = 0;

        int ret = engine->PushExternalVideoFrame(&sample, AliRTCSdk::Central::VideoSourceCamera);
        if (ret == 0x01070101) {
            // SDK buffer full, rewind and retry after a short delay
            long pos = ftell(fin);
            fseek(fin, pos - (long)readSize, SEEK_SET);
            usleep(100000);
            continue;
        }

        frameCount++;
        int64_t sleepTime = (int64_t)frameCount * 1000 / fps - (currentTimeMs() - startTime) - 1;
        if (sleepTime > 0) sleepMs(sleepTime);
    }

    free(buf);
    fclose(fin);
});
```

Push External Audio Frames

Linux does not provide a built-in microphone recording interface. Use PushExternalAudioFrameRawData to input PCM audio data to the SDK. The following example reads and pushes frame data in a loop from a PCM file (int16_t, 16 kHz, mono channel). In production, replace this with microphone driver or audio decoder output.

```cpp
std::thread audioThread([&]() {
    const int    sampleRate = 16000;
    const int    channels   = 1;
    const int    frameMs    = 20;  // 20ms per frame
    const size_t frameSize  = (sampleRate / 1000) * frameMs * sizeof(int16_t) * channels;

    FILE* fin = fopen("/tmp/test_16k_mono.pcm", "rb");
    void* buf = malloc(frameSize);
    int64_t totalSentMs = 0;
    int64_t startTime   = currentTimeMs();

    while (running) {
        size_t readSize = fread(buf, 1, frameSize, fin);
        if (readSize != frameSize) {
            fseek(fin, 0, SEEK_SET); // loop when file ends
            continue;
        }

        int ret = engine->PushExternalAudioFrameRawData(buf, (unsigned int)frameSize, totalSentMs);
        if (ret != 0) {
            // SDK buffer full, rewind and retry after a short delay
            long pos = ftell(fin);
            fseek(fin, pos - (long)readSize, SEEK_SET);
            usleep(20000);
            continue;
        }

        totalSentMs += frameMs;
        int64_t sleepTime = totalSentMs - (currentTimeMs() - startTime) - 1;
        if (sleepTime > 0) sleepMs(sleepTime);
    }

    free(buf);
    fclose(fin);
});
```

Handle Audio and Video Playback

Linux does not provide built-in audio and video playback devices. Remote audio and video data is delivered to the application layer through callback frames for processing, such as writing to a file, sending to a decoder, or connecting to a playback device.

Audio Playback

Based on the subscribeAudioFormat configuration in step 6, one of the following callbacks is triggered when remote audio frames are received:

// AudioFormatMixedPcm mode: Receive mixed PCM data for the entire channel from all remote users
void OnSubscribeMixAudioFrame(const AliRTCSdk::Central::AudioFrame* frame) override {
    // frame->data     PCM data pointer (int16_t)
    // frame->dataSize Data byte count
    // frame->sampleRate / frame->channel Sample rate and number of sound channels
    // Write to a file, send to an audio device, or decode for playback here
}

// AudioFormatPcmBeforMixing mode: Receive unmixed individual PCM data per user
void OnSubscribeAudioFrame(const std::string& uid,
                           const AliRTCSdk::Central::AudioFrame* frame) override {
    // uid identifies which remote user the frame is from
    // Process audio data separately per user here
}

Video Playback

When remote video frames are received, the OnRemoteVideoSample callback is triggered. The frame format is determined by subscribeVideoFormat in step 6:

void OnRemoteVideoSample(const char* uid,
                         const AliRTCSdk::Central::VideoFrame* frame) override {
    // uid identifies which remote user the frame is from
    // frame->data     YUV data pointer (I420 format)
    // frame->width / frame->height Resolution
    // Write to a file, send to a renderer, or video encoder here
}

Leave Channel and Destroy Engine

To release resources, stop stream ingest, leave the channel, and destroy the engine in sequence.

// Stop external push threads first
running = false;
videoThread.join();
audioThread.join();

// Call PublishLocalVideoStream(false) / PublishLocalAudioStream(false) to stop publishing
engine->PublishLocalVideoStream(false);
engine->PublishLocalAudioStream(false);

// Call LeaveChannel to leave the channel
engine->LeaveChannel();

// Call Release to destroy the engine (must be called after LeaveChannel)
engine->Release();
engine = nullptr;

delete eventHandler;
eventHandler = nullptr;