Quick integration

Preparations

Unzip the Linux SDK package and open the Go folder. The folder contains the following:

artc_linux_go/
├── alirtc
│   ├── AliRTCEngine.go
│   ├── AliRTCEngineImpl.go
│   ├── AliRTCLinuxSdkDefine.go
│   ├── go.mod
│   └── lib
│       ├── AliRtcCoreService
│       ├── libAliRtcLinuxEngine.so
│       └── libonnxruntime.so.1.16.3
├── Demo
├── demo.go
├── go.mod
└──	README.md

2. Core APIs

1. Initialize the SDK

Procedure:

1. Implement the EngineEventHandlerInterface class. This class contains the callbacks that ARTC requires. The callback functions in this class are triggered when data is received or the status changes.

2. Create an EngineEventListener instance.

3. Call the alirtc.CreateAliRTCEngine function to create an AliRTCEngine instance. This process registers the EngineEventListener instance as the callback object for the engine. Then, you can call the methods of the AliRTCEngine instance to configure stream ingest and stream pulling.

4. To ingest streams into multiple channels, create multiple AliRTCEngine instances. Each instance corresponds to a virtual user in a channel.

Example:

// Implement the EngineEventHandlerInterface class
...

// Initialize the SDK
eventHandler = &EngineEventListener{}
h5mode := false
coreServicePath := "/path/AliRtcCoreService" // Path of AliRtcCoreService
extraJobj := map[string]interface{}{
    "user_specified_disable_audio_ranking": "true",
}
extraBytes, _ := json.Marshal(extraJobj)
extra := string(extraBytes)
linuxEngine := alirtc.CreateAliRTCEngine(eventHandler, 42000, 45000, "/tmp", coreServicePath, h5mode, extra)

After the CreateAliRTCEngine function is called, an AliRtcCoreService process starts. The function parameters are as follows:

• eventHandler:EngineEventHandlerInterface: The callback object that handles callback logic.

• lowPort:int: The lower limit of the port range. A port between `lowPort` and `highPort` is randomly selected for inter-process communication.

• highPort:int: The upper limit of the port range.

• logPath:str: The path where log files are saved when the Linux SDK is running.

• coreServicePath:str: The path to the AliRtcCoreService file.

• h5mode:bool: The H5 compatibility mode. Set this parameter to `true` to communicate with web clients. In other scenarios, you can set it to `false`.

• extra:str: A JSON string for additional SDK configuration.

2. Join a channel

First, familiarize yourself with the Alibaba Cloud ARTC authentication process. For more information, see Token-based authentication.

Two versions of the channel-joining token are available. Use the single-parameter token. The Go interface provides only the API for this version.

Procedure:

1. Set basic information, such as the username and channel ID, and request the single-parameter token from your app server.

2. Configure JoinChannelConfig to set parameters, such as the stream ingest and stream pulling modes.

3. Call the JoinChannel member method to join the channel.

4. The OnJoinChannelResult function of the EngineEventListener instance is triggered to notify you of the result of joining the channel.

// -------- Obtain the basic information required to join a channel --------
authInfo := alirtc.AuthInfo{
    AppID:    "", // The App ID provided by your service.
    UserID:   "", // The username. It must be unique within the channel. If a user with the same ID joins, the previous user is removed from the channel.
    UserName: "", // The user's nickname. It can be the same as the UserID or a custom name.
    Channel:  "", // The channel ID.
}
// ...Request the single-parameter token from the app server.
authInfo.Token = ""      // The channel-joining token returned by the app server.

// -------- Configure JoinChannelConfig --------
joinConfig := alirtc.NewJoinChannelConfig()
joinConfig.ChannelProfile = alirtc.ChannelProfileInteractiveLive // Interactive live streaming mode
joinConfig.SubscribeAudioFormat = alirtc.AudioFormatPcmBeforMixing // Audio subscription format
joinConfig.SubscribeVideoFormat = alirtc.VideoFormatH264 // Video subscription format
joinConfig.IsAudioOnly = false // Audio-only mode. Typically set to false.
joinConfig.PublishAvsyncMode = alirtc.PublishAvsyncWithPts // A/V synchronization mode for stream ingest
joinConfig.SubscribeMode = alirtc.SubscribeAutomatically // Subscription mode
joinConfig.PublishMode = alirtc.PublishAutomatically // Stream ingest mode

// -------- Call JoinChannel to join the channel --------
linuxEngine.JoinChannel(authInfo.Token, authInfo.Channel, authInfo.UserID, authInfo.UserName, joinConfig)

JoinChannelConfig parameter descriptions:

1. Stream ingest mode: Supports automatic and manual modes. In automatic mode, audio and video stream ingest starts automatically after you join the channel. In manual mode, you must call an API to start stream ingest.

2. Subscription mode: Supports automatic and manual modes. In automatic mode, the SDK automatically subscribes to the audio and video streams of a streamer after the streamer joins the channel. In manual mode, you must call an API and specify the streamer's UID to subscribe.

   In scenarios where you subscribe to only audio or video, use SubscribeAudioAutoAndOnly or SubscribeCameraAutoAndOnly to reduce decoder CPU usage.

3. Audio subscription format: AudioFormatMixedPcm indicates a mixed stream. After you subscribe to the audio of all remote users in the channel, their audio is mixed into a single stream and returned in one audio data callback. AudioFormatPcmBeforeMixing indicates separate streams. The audio data of each remote user is returned in a separate callback.

4. Video subscription format: Set this to VideoFormatH264. The OnRemoteVideoSample callback is triggered when data is received. Otherwise, the callback is not triggered.

5. Audio/video synchronization mode: If you select PublishAvsyncWithPts, audio and video are synchronized based on the timestamp passed to the API during audio and video data ingest. Otherwise, the timestamp is ignored, and the data is ingested based on the API call frequency.

3. Manually start and stop stream ingest

The following APIs control whether to ingest audio and video streams. You must call these APIs to start stream ingest before you send data.

/**
 * @brief Specifies whether to ingest the local video (camera) stream.
 * @param enabled Specifies whether to start or stop ingesting the local video stream.
 - true: Start ingesting the video stream.
 - false: Stop ingesting the video stream.
 * @return
 - 0: The setting is successful.
 - <0: The setting failed. An error code is returned.
 * @note By default, the SDK is set to ingest the video stream. You can also call this interface before joining a channel to change the default value. The setting takes effect after you successfully join the channel.
**/
PublishLocalVideoStream(enabled bool) int

/**
 * @brief Specifies whether to ingest the local audio stream.
 * @param enabled Specifies whether to start or stop ingesting the local audio stream.
 - true: Start ingesting the audio stream.
 - false: Stop ingesting the audio stream.
 * @return
 - 0: The setting is successful.
 - <0: The setting failed. An error code is returned.
 * @note By default, the SDK is set to ingest the audio stream. You can also call this interface before joining a channel to change the default value. The setting takes effect after you successfully join the channel.
**/
PublishLocalAudioStream(enabled bool) int

If you have finished sending data but do not want to leave the channel immediately, you can call the two preceding APIs to stop stream ingest.

4. Ingest external video data

/**
 * @brief Enables an external video input source.
 * @param enable true: enable, false: disable.
 * @param type The stream type.
 * @note After enabling, use the PushExternalVideoFrame interface to input video data.
**/
SetExternalVideoSource(enable bool, sourceType VideoSource, renderMode RenderMode) int

/**
 * @brief Inputs an external video. Video input of 2K or higher resolution is not supported.
 * @param frame The frame data.
 * @param type The stream type.
 * @param The input video frame supports multiple types, such as YUV and RGB.
**/
PushExternalVideoFrame(frame *VideoDataSample, sourceType VideoSource) int

5. Ingest external audio data

If you do not call SetExternalAudioPublishVolume, the audio is ingested at the default volume of 100.

/**
 * @brief Sets whether to enable external audio input for stream ingest.
 * @param enable true: enable, false: disable.
 * @param sampleRate The sample rate, such as 16k or 48k.
 * @param channelsPerFrame The number of channels, such as 1 or 2.
 * @return A value of 0 or greater indicates success. A value less than 0 indicates failure.
 * @note You can call SetExternalAudioPublishVolume to set the volume for ingesting external audio.
**/
SetExternalAudioSource(enable bool, sampleRate, channelsPerFrame int) int

/**
 * @brief Inputs external audio data for stream ingest.
 * @param audioSamples The audio data.
 * @param sampleLength The length of the audio data.
 * @param timestamp The timestamp.
 * @return A value less than 0 indicates failure. If ERR_AUDIO_BUFFER_FULL is returned, retry delivering the data after an interval equal to the data delivery time.
**/
PushExternalAudioFrameRawData(audioSamples []byte, sampleLength int, timestamp int64) int

/**
 * @brief Sets the mixing volume for ingesting external audio.
 * @param vol The volume. Valid values: 0 to 100.
**/
SetExternalAudioPublishVolume(volume int) int

/**
 * @brief Gets the mixing volume for ingesting external audio.
 * @return vol The volume.
**/
GetExternalAudioPublishVolume() int

6. Manual subscription

In automatic subscription mode, you do not need to call these APIs to pull streams.

/**
 * @brief Stops or resumes subscribing to the audio stream of a specific remote user. This is called during a session and is invalid if called before a session.
 * @param uid The user ID, a unique identifier assigned by the app server.
 * @param sub Specifies whether to subscribe to the remote user's audio stream.
 * - true: Subscribe to the specified user's audio stream.
 * - false: Stop subscribing to the specified user's audio stream.
 * @return
 * - 0: Success.
 * - Non-zero: Failure.
**/
SubscribeRemoteAudioStream(uid string, sub bool) int

/**
 * @brief Stops or resumes subscribing to a remote user's video stream. This is called during a session and is invalid if called before a session.
 * @param uid The user ID, a unique identifier assigned by the app server.
 * @param track The video stream type.
 * - AliEngineVideoTrackNo: Invalid parameter. Setting this has no effect.
 * - AliEngineVideoTrackCamera: Camera stream.
 * - AliEngineVideoTrackScreen: Screen sharing stream.
 * - AliEngineVideoTrackBoth: Camera stream and screen sharing stream.
 * @param sub Specifies whether to subscribe to the remote user's video stream.
 * - true: Subscribe to the specified user's video stream.
 * - false: Stop subscribing to the specified user's video stream.
 * @return
 * - 0: The setting is successful.
 * - <0: The setting failed.
 * @note
**/
SubscribeRemoteVideoStream(uid string, videoTrack VideoTrack, sub bool) int

7. Data callbacks

7.1 Audio

If you select the AudioFormatPcmBeforeMixing mode, the OnSubscribeAudioFrame callback function of the EventHandler instance is triggered when an audio frame is received.

• uid: Indicates which remote user the received audio frame is from. This helps distinguish between different subscribed audio streams.

• frame: The received audio frame in PCM format.

/**
 * @brief Callback for the audio data of each remote user before mixing.
 * @details Audio data from a single remote user, corresponding to AliRTCSdk::Linux::AudioFormatPcmBeforMixing.
 * @param frame The audio data. For more information, see {@link AliRTCSdk::Linux::AudioFrame}.
**/
OnSubscribeAudioFrame(uid string, frame AudioFrame)

If you select the AudioFormatMixedPcm mode, the OnSubscribeMixAudioFrame callback function of the EventHandler instance is triggered when an audio frame is received.

/**
 * @brief Callback for locally subscribed audio data.
 * @details Audio data from all remote users, mixed and ready for playback, corresponding to AliRTCSdk::Linux::AudioFormatMixedPcm.
 * @param frame The audio data. For more information, see {@link AliRTCSdk::Linux::AudioFrame}.
**/
OnSubscribeMixAudioFrame(frame AudioFrame)

7.2 Video

When a video frame is received, the OnRemoteVideoSample callback function of the EventHandler instance is triggered.

• uid: Indicates which remote user the received video frame is from. This helps distinguish between different subscribed video streams.

• frame: The received video frame in YUV I420 format.

/**
 * @brief Callback for subscribed remote video data.
 * @param uid The user ID.
 * @param frame The raw video data.
 * @return
**/
OnRemoteVideoSample(uid string, frame VideoFrame)

8. Leave a channel

// Stop stream ingest
linuxEngine.PublishLocalVideoStream(false)
linuxEngine.PublishLocalAudioStream(false)

If you want to remain in the channel after you stop stream ingest, you must call this API manually. Leaving the channel also stops stream ingest.

linuxEngine.LeaveChannel()

9. Destroy the SDK

linuxEngine.Release()
linuxEngine = nil

3. Use the demo

In the demo.go file, replace ALIRTC_APPID and ALIRTC_APPKEY with your credentials, and specify the correct path for AliRtcCoreService. Then, run the go run demo.go command in the directory where demo.go is located.

If the following output is displayed in the terminal, it indicates that you have successfully joined the channel and started stream ingest.

# Successfully joined the channel
[Go] on join channel result. Channel: 123123, user: linux, result: 0

# Stream ingest successful
[Go] on audio publish state changed, oldState: 0, newState: 2
[Go] on video publish state changed, oldState: 0, newState: 2
[Go] on audio publish state changed, oldState: 2, newState: 3
[Go] on video publish state changed, oldState: 2, newState: 3

# Remote user 'web' is online
[Go] on remote user online: web

# Received audio and video streams from remote user 'web'
[Go] on audio subscribe state of web, oldState: 0, newState: 2
[Go] on video subscribe state of web, oldState: 0, newState: 2
[Go] on audio subscribe state of web, oldState: 2, newState: 3
[Go] on video subscribe state of web, oldState: 2, newState: 3

When you select the AudioFormatMixedPcm mode for audio subscription, an audio callback occurs when you join the channel, regardless of whether there are remote users in the channel. When you select the AudioFormatPcmBeforeMixing mode, an audio callback is triggered only when there are other users in the channel and they are ingesting an audio stream.

4. Message communication

In addition to audio and video interaction, the ARTC SDK supports real-time messaging for scenarios that require message interaction. You can send and receive messages through two types of channels: data channels and Supplemental Enhancement Information (SEI). Data channels are independent of the audio and video transmission channels. SEI relies on video data transmission. In most scenarios, use data channels.

1. Data Channel

Call SetParameter to enable the data channel: linuxEngine.SetParameter("{\"data\":{\"enablePubDataChannel\":true,\"enableSubDataChannel\":true}}").

Send a message using SendDataChannelMessage:

dataChannelMsg := alirtc.AliEngineDataChannelMsg{}
message := "This is data channel message"
dataChannelMsg.Data = []byte(message)
dataChannelMsg.DataLen = len(dataChannelMsg.Data)
dataChannelMsg.NetworkTime = 0 // Network time field. You can also set a custom value, which does not affect message sending and receiving.
dataChannelMsg.Progress = 0    // Reserved field.
dataChannelMsg.Type = alirtc.AliEngineDataChannelCustom

linuxEngine.SendDataChannelMessage(dataChannelMsg)

When a data channel message is received, a callback is triggered:

/**
 * @brief Obtains remote data from the data channel.
 * @param msg The message from the remote end.
**/
OnDataChannelMsg(uid string, msg AliEngineDataChannelMsg)

2. SEI

/**
 * @brief Sends supplemental enhancement information (SEI). The maximum length is 4 × 1024 bytes. This is used for transmitting small amounts of business data.
 * @param message The content of the extension message. You can transmit 4 KB of data.
 * @param length The length of the extension message in bytes.
 * @param repeatCount The number of repetitions, used to prevent message loss due to network packet loss.
 * @param delay The delay before sending, in milliseconds.
 * @param isKeyFrame Specifies whether to add SEI only on keyframes.
 * @return <0: Success, -1: Internal SDK error.
**/
SendMediaExtensionMsg(message []byte, length, repeatCount, delay int, isKeyFrame bool) int

Send a message using SendMediaExtensionMsg:

seiMsg := []byte("This is SEI message")
linuxEngine.SendMediaExtensionMsg(seiMsg, len(seiMsg), 3, 0, false)

Because the keyframe interval is long, you must set isKeyFrame to false to send and receive SEI messages at a high frequency. To ensure effective message delivery, set the number of repetitions to a value greater than 1.

When an SEI message is received, a callback is triggered:

/**
 * @brief Callback for receiving a media extension message.
 * @param uid The user ID of the sender.
 * @param message The content of the extension message.
 * @param size The length of the extension message.
 * @note When one end sends a message through {@link SendMediaExtensionMsg}, other ends receive the data through this callback.
**/
OnMediaExtensionMsgReceived(userid string, message []byte, size int)

5. Other features

1. Disable Audio Ranking

Audio Ranking is a feature for multi-user voice chat rooms that subscribes to only the few audio streams with the highest volume, which makes the sound clearer for the audience. This feature is enabled by default. If your scenario requires you to subscribe to all audio streams, use the following field to disable Audio Ranking:

{"user_specified_disable_audio_ranking" : "true"}

2. Enable AAC transcoding for subscribed audio

For scenarios that require archiving pulled streams, the SDK can transcode subscribed remote audio to the AAC format. You can use the options in AudioTranscodingCodec to select the format for obtaining remote audio: PCM, AAC, or both. If you enable AAC transcoding, you can also choose whether to resample the audio to reduce CPU and storage usage. If `user_specified_audio_observer_resample_rate` is set to 0, resampling is not performed.

{"user_specified_audio_observer_codec" : 
    AudioTranscodingCodec.AudioTranscodingCodecBothPcmAndAac.value}
{"user_specified_audio_observer_codec_format" : 0} // Only the ADTS format is currently supported.
{"user_specified_audio_observer_resample_rate" : 16000} // The resampling rate.
{"user_specified_audio_observer_codec_bitrate" : 64000} // The target bitrate for transcoding.

3. Directly obtain H.264 streams for subscribed video

For scenarios that require archiving pulled streams, the SDK can directly obtain the H.264 stream of remote video. You can use the options in VideoTranscodingCodec to select the format for obtaining remote video: YUV, H.264 stream, or both.

{"user_specified_video_observer_codec" :
    VideoTranscodingCodec.VideoTranscodingCodecBothYuvAndH264.value}
{"user_specified_video_observer_codec_format" : 0} // Only the Annex B format is currently supported.