Quick integration

1. Specify the ELF file directory

The underlying SDK is implemented in C, and the Java interface manages multiple RTC engine instances through a multi-process approach. The alirtc_linux_java_multiprocess.jar in the libs directory bridges the Java layer with the C engine. The process startup requires the compiled ELF file libs/AliRtcCoreService. Therefore, you must correctly specify the path to this file to ensure that subsequent operations can be performed properly.

String coreServicePath = "/mnt/AliRTCSDK_Linux-v6.8.2/Java/libs/AliRtcCoreService";

2. Initialize the SDK

1. Implement the AliRTCLinuxEngineListener interface to receive engine callbacks

2. Create an instance of the AliRTCLinuxEngineListener implementation class

3. Call AliRTCLinuxEngine.createInstance to create an engine instance and attach the Listener to the engine

Sample code:

//Initialize the SDK
AliRTCLinuxEngineListener engineEventHandler = new EngineListener();
boolean h5mode = true; // H5 compatibility mode must be enabled for interoperability with web clients
String extra = "";
AliRTCLinuxEngine linuxEngine = AliRTCLinuxEngine.createInstance(engineEventHandler, 0, 0, "/tmp", "", h5mode, extra);

3. Configure parameters and join a channel

1. Create authInfo and obtain the required parameters for joining a channel

2. Configure JoinChannelConfig. You can use the default settings

3. Call JoinChannel

// Obtain authInfo
AliRTCLinuxEngine.AuthInfo authInfo = new AliRTCLinuxEngine.AuthInfo();
authInfo.appid = "";
authInfo.channel = "";
authInfo.userid = "";
authInfo.username = "";
authInfo.nonce = "";
authInfo.token = "";
authInfo.timestamp = 1591350597; // sample

int gslbCount = 1;
authInfo.gslb_count = gslbCount;
String[] gslbArray = new String[gslbCount];
if (gslbCount > 0) {
    for (int i = 0; i < gslbCount; i++) {
        gslbArray[i] = "https://******";
    }
    authInfo.gslb = gslbArray;
}

int agentCount = 0; // Set the agent based on your requirements. You can choose not to set it
authInfo.agent_count = agentCount;
String[] agentArray = new String[agentCount];
if (agentCount > 0) {
    for (int i = 0; i < agentCount; i++) {
        agentArray[i] = "https://******";
    }
    authInfo.agent = agentArray;
}

// Initialize the configurations that are required to join the channel
AliRTCLinuxEngine.JoinChannelConfig joinConfig = new AliRTCLinuxEngine.JoinChannelConfig();

joinConfig.channelProfile = AliRTCLinuxEngine.ChannelProfile.ChannelProfileInteractiveLive;
// Set whether to subscribe to audio data and the subscription method
joinConfig.subscribeAudioFormat = AliRTCLinuxEngine.AudioFormat.AudioFormatPcmBeforeMixing;
// Set whether to subscribe to video data
joinConfig.subscribeVideoFormat = AliRTCLinuxEngine.VideoFormat.VideoFormatH264;

// Audio and video synchronization options: choose to send data immediately or synchronize based on timestamps
joinConfig.publishAvsyncMode = AliRTCLinuxEngine.PublishAvsyncMode.PublishAvysncWithPts;

// Automatic subscription [Set manual subscription based on your requirements]
joinConfig.subscribeMode = AliRTCLinuxEngine.SubscribeMode.SubscribeAutomatically;
// Enable automatic stream ingest [You can set manual stream ingest based on your requirements]
joinConfig.publishMode = AliRTCLinuxEngine.PublishMode.PublishAutomatically; 

// Set whether to publish camera streams [You can configure this based on your requirements]
linuxEngine.publishLocalVideoStream(true);
// Set whether to publish audio streams [You can configure this based on your requirements]
linuxEngine.publishLocalAudioStream(true);

// Set camera stream encoding options [You can configure this based on your requirements]
AliRTCLinuxEngine.AliEngineVideoEncoderConfiguration videoConfig = new AliRTCLinuxEngine.AliEngineVideoEncoderConfiguration();

linuxEngine.setVideoEncoderConfiguration(videoConfig);

// Enable YUV input, use camera streams for publishing, and specify the fill mode
linuxEngine.setExternalVideoSource(true, AliRTCLinuxEngine.VideoSource.VideoSourceCamera, AliRTCLinuxEngine.RenderMode.RenderModeFill);

// Publish media tracks in the PCM format
// The second parameter specifies the sampling rate of audio in PCM video tracks. Specify a value based on your requirements
// The third parameter specifies the number of audio channels in PCM video tracks. Specify a value based on your requirements
linuxEngine.setExternalAudioSource(true, 16000, 2);

// Set the role for joining the channel
linuxEngine.setClientRole(AliRTCLinuxEngine.AliEngineClientRole.AliEngineClientRoleInteractive);

// Join a channel
linuxEngine.joinChannel(authInfo, joinConfig);

3. Stream ingest

After setting the external video source with setExternalVideoSource, use the following method to push each original video frame (I420 format).

/**
 * @param frame The frame data
 * @param source The stream type
 * @brief Import external video data
 * @note For supported input video types, see VideoDataFormat
 */
public abstract int pushExternalVideoFrame(VideoDataSample frame, VideoSource source);

Similarly for audio frames:

/**
 * @param audioSamples The audio data
 * @param sampleLength The length of the audio data
 * @param timestamp    The timestamp
 * @return A value of less than 0 indicates failure. If ERR_AUDIO_BUFFER_FULL is returned, try again after the specified data delivery interval elapses
 * @brief Import external audio data for stream ingest
 */
public abstract int pushExternalAudioFrameRawData(byte[] audioSamples, int sampleLength, long timestamp);

4. Subscription

You need to ensure that subscription is enabled when setting audioFormat and videoFormat.

Callback APIs allow you to observe whether remote users are online and whether they are publishing streams.

/**
 * @brief Callback for when a remote user (in communication mode) or (in interactive mode, streamer role) joins the channel
 *
 * @param uid User ID, a unique identifier assigned by the App server
 * @note Callback behavior in interactive mode
 * - Streamers can receive join channel callbacks from each other
 * - Viewers can receive join channel callbacks from streamers
 * - Streamers cannot receive join channel callbacks from viewers
 */
void onRemoteUserOnLineNotify(String uid);

/**
 * @brief Callback for changes in remote users' audio and video streams
 * @details This callback is triggered in the following scenarios
 * - When a remote user changes from not publishing to publishing streams (including audio and video)
 * - When a remote user changes from publishing to not publishing streams (including audio and video)
 * - In interactive mode, when calling {@link AliEngine::SetClientRole} to switch to the streamer role {@link AliEngineClientRoleInteractive} and setting stream publishing, this callback is triggered
 * @param uid userId, a unique identifier assigned by the App server
 * @param audioTrack Audio stream type, see {@link AliEngineAudioTrack}
 * @param videoTrack Video stream type, see {@link AliEngineVideoTrack}
 * @note This callback is only triggered for users in communication mode and streamers in interactive mode
 */
void onRemoteTrackAvailableNotify(String uid, AliRTCLinuxEngine.AudioTrack audioTrack, AliRTCLinuxEngine.VideoTrack videoTrack);

(1) Video callback: Each remote user's audio stream subscription is separate, distinguished by uid

/**
 * @brief Callback for subscribed remote video data
 * @param uid User ID
 * @param frame Raw video data
 * @return
 */
void onRemoteVideoSample(String uid, AliRTCLinuxEngine.VideoFrame frame);

(2) Mixed audio callback: All subscribed remote users' audio data is mixed into one stream and returned via callback

/**
 * @brief Callback for local subscribed mixed audio data
 * @details Audio data from all remote users mixed and ready for playback. This callback is triggered when {@link IAliEngineMediaEngine::SubscribeAudioData} subscription type is AliEngineAudiosourceSub
 * @param frame Audio data, see {@link AliEngineAudioRawData}
 */
void onSubscribeMixedAudioFrame(AliRTCLinuxEngine.AudioFrame frame);

(3) Separate audio stream callback: Different remote users' audio data is distinguished by uid

/**
 * @brief Callback for local subscribed audio data
 * @details Audio data from a single remote user mixed, distinguished by uid. This callback is triggered when {@link IAliEngineMediaEngine::SubscribeAudioData} subscription type is AliEngineAudiosourceSub
 * @param frame Audio data, see {@link AliEngineAudioRawData}
 */
void onSubscribeAudioFrame(String uid, AliRTCLinuxEngine.AudioFrame frame);

5. Leave a channel

If you want to stop stream ingest without leaving the channel, call the following methods manually.

linuxEngine.publishLocalVideoStream(0);
linuxEngine.publishLocalAudioStream(0);

If you want to leave the channel directly, you can call the following interface to automatically stop stream ingest.

linuxEngine.leaveChannel();

6. Destroy the SDK

linuxEngine.destroy();
linuxEngine = null;

1. Extra field configuration

The extra field passed when creating the engine is in JSON format and can be used to set certain options to help you use other capabilities. For example, you can set it to the following content.

Disable Audio Ranking
user_specified_disable_audio_ranking":"true"

Support audio AAC transcoding

-> Audio callback mode
"user_specified_audio_observer_codec":
    AliRTCLinuxEngine.AudioTranscodingCodec.AudioTranscodingCodecAac.getValue()

-> AAC format, currently only adts is supported
"user_specified_audio_observer_codec_format": 0 // adts

-> If resample_rate is non-zero, resampling is performed. This specifies the resampling rate
"user_specified_audio_observer_resample_rate":16000

-> Target bitrate for AAC transcoding
"user_specified_audio_observer_codec_bitrate":64000

Support direct access to H264 bitstream

-> Video callback mode
"user_specified_video_observer_codec":
    AliRTCLinuxEngine.VideoTranscodingCodec。VideoTranscodingCodecH264.getValue()

-> H264 format, currently only annexb is supported
"user_specified_video_observer_codec_format":0 // annexb

2. Message sending and receiving: DataChannel or SEI

In addition to audio and video data, you can also use the RTC SDK for real-time message interaction. The SDK supports two types of real-time message sending and receiving: Data Channel and SEI (Supplemental Enhancement Information). The difference is that the former is independent of audio and video channels and is the recommended choice, while the latter depends on video data transmission.

To enable DataChannel, call: engineIns.setParameter("{\"data\":{\"enablePubDataChannel\":true,\"enableSubDataChannel\":true}}");

• Sending messages via Data Channel:

/**
  * @brief Send messages through dataChannel
  * @param controlMsg The message to be sent
  * @return
  * - 0: successful
  * - A value other than 0: failure
  */
int 
sendDataChannelMsg(
  AliRtcDataChannelMsg controlMsg
);

• Receiving messages via Data Channel:

void 
onDataChannelMsg(
  String uid, 
  AliRTCLinuxEngine.AliRtcDataChannelMsg msg
)

• Sending messages via SEI:

/**
  * @brief Send an SEI message. The message can be up to 4 KB in length. It is used for a small amount of data transmission
  * @param message The content of the message, which can be up to 4 KB in size
  * @param repeatCount The number of retries. This parameter can be used to prevent message loss caused by packet loss
  * @param delay The delay for sending the message. Unit: milliseconds
  * @param isKeyFrame Specifies whether to insert SEI into only key frames
  */
int 
sendMediaExtensionMsg(
  byte[] message, int repeatCount, 
  int delay, boolean isKeyFrame
);

• Receiving messages via SEI:

void 
onMediaExtensionMsgReceived(
  String uid, byte[] msg
)