custom audio playback-ApsaraVideo Live(LIVE)-阿里云帮助中心

ARTC allows you to bypass its built-in audio playback module and manage audio rendering with your own player, giving you full control over how received audio data is processed and played.

Overview

By default, ARTC includes a field-tested audio playback module that meets most playback requirements. However, if you already have a custom audio playback module or need to post-process audio data before playback, you can use the custom audio playback feature to disable internal playback logic and handle audio rendering yourself.

You can then pass the audio data to your playback device for processing and playback.

Sample code

Android: Android/ARTCExample/AdvancedUsage/src/main/java/com/aliyun/artc/api/advancedusage/CustomAudioCaptureAndRender/CustomAudioRenderActivity.java.

iOS: iOS/ARTCExample/AdvancedUsage/CustomAudioRender/CustomAudioRenderVC.swift.

Harmony: Harmony/ARTCExample/entry/src/main/ets/pages/advancedusage/CustomAudioRenderPage.ets.

Prerequisites

Before using custom audio playback, make sure you meet the following requirements:

You can develop a complete audio playback module, or you already have a third-party or in-house audio player.
You understand basic audio processing concepts such as PCM data format, sample rate, and number of channels.
You have integrated the ARTC SDK and implemented basic audio and video call features.
Your use case requires bypassing the ARTC internal playback module. Otherwise, use the default playback solution to ensure stability and compatibility.

Implementation

1. Disable internal playback

To use custom audio playback, disable the SDK's internal playback. We recommend doing this when creating the engine instance by calling getInstance and passing the extras parameter. The relevant parameter is as follows:

user_specified_use_external_audio_player: specifies whether to use an external player.

"TRUE": uses an external player and disables internal playback.
"FALSE": uses the default internal playback.

Android

String extras = "{\"user_specified_use_external_audio_player\":\"TRUE\"}";
mAliRtcEngine = AliRtcEngine.getInstance(this, extras);

iOS

// Create and initialize the engine.
var customAudioPlayConfig: [String: String] = [:]
// Use an external player, which disables the SDK's internal playback.
customAudioPlayConfig["user_specified_use_external_audio_player"] = "TRUE"
// Serialize to JSON.
guard let jsonData = try? JSONSerialization.data(withJSONObject: customAudioPlayConfig, options: []),
let extras = String(data: jsonData, encoding: .utf8) else {
    print("JSON serialization failed")
    return
}
let engine = AliRtcEngine.sharedInstance(self, extras:extras)

Mac

NSString * extras = @"{\"user_specified_use_external_audio_player\":\"TRUE\"}";
self.engine = [AliRtcEngine sharedInstance:self extras:extras];

Windows

std::string extras = "{\"user_specified_use_external_audio_player\":\"TRUE\"}";
mAliRtcEngine = AliRtcEngine.getInstance(this, extras.c_str());

Harmony

// Create an RTC engine instance and set it to use an external player.
const extras = '{"user_specified_use_external_audio_player":"TRUE"}';
this.rtcEngine = AliRtcEngine.getInstance(extras, this.context);

2. Register audio frame callback

Call enableAudioFrameObserver to register the audio frame callback with the following parameters:

enable: a boolean that enables or disables the data callback.
audioSource: the type of data source for the callback. For custom audio playback, set this to AliRtcAudioSourcePlayback to receive data from the playback stage through the onPlaybackAudioFrame callback.
config: the desired audio parameters for the callback, including sample rate and number of channels.

Android

// Set the callback configuration.
AliRtcEngine.AliRtcAudioFrameObserverConfig config = new AliRtcEngine.AliRtcAudioFrameObserverConfig();
config.sampleRate = AliRtcAudioSampleRate_48000;
config.channels = 1;
// Register to enable the playback data callback.
mAliRtcEngine.enableAudioFrameObserver(true, AliRtcAudioSourcePlayback, config);

iOS

// Set the callback configuration.
var observerConfig: AliRtcAudioFrameObserverConfig = AliRtcAudioFrameObserverConfig()
observerConfig.sampleRate = ._Unknown
observerConfig.channels = .monoAudio
// Register to enable the playback data callback.
let audioSource: AliRtcAudioSource = .playback
engine.enableAudioFrameObserver(true, audioSource: audioSource, config: observerConfig)

Mac

AliRtcAudioFrameObserverConfig* config = [[AliRtcAudioFrameObserverConfig alloc] init];
config.channels = audioNumChannel;
config.sampleRate = AliRtcAudioSampleRate_48000;
config.mode = AliRtcAudioFrameObserverOperationModeReadOnly;
[self.engine enableAudioFrameObserver:TRUE audioSource:AliRtcAudioSourcePlayback config:config];

Windows

// Set the callback configuration.
AliEngineAudioFrameObserverConfig config ;
config.sampleRate = AliRtcAudioSampleRate_48000;
config.channels = AliEngineMonoAudio;
// Register to enable the playback data callback.
mAliRtcEngine->EnableAudioFrameObserver(true, AliEngineAudioSourcePlayback, config);

Harmony

const config: AliRtcAudioFrameObserverConfig = new AliRtcAudioFrameObserverConfig();
config.sampleRate = AliRtcAudioSampleRate.AliRtcAudioSampleRate_48000;
config.channels = AliRtcAudioNumChannel.AliRtcMonoAudio;
this.rtcEngine.enableAudioFrameObserver(true, AliRtcAudioSource.AliRtcAudioSourcePlayback, config);

3. Process PCM data in the callback

In the callback, process the received PCM data and pass it to your playback device.

Android

@Override
public boolean onPlaybackAudioFrame(AliRtcEngine.AliRtcAudioFrame frame) {
    // Implement your custom logic to play the audio data received in the callback.

    return true;
}

iOS

func onPlaybackAudioFrame(_ frame: AliRtcAudioFrame) -> Bool {
    // Implement your custom logic to play the audio data received in the callback.
    
    return true
}

Mac

- (BOOL)onPlaybackAudioFrame:(AliRtcAudioFrame* _Nonnull)frame {
    // Implement your custom logic to play the audio data received in the callback.
    return true;
}

Windows

bool CTutorialDlg::OnPlaybackAudioFrame(AliEngineAudioRawData audioRawData) {
	// Implement your custom logic to play the audio data received in the callback.
	return true;
}

Harmony

rtcAudioFrameObserver.onPlaybackAudioFrame((frame: AliRtcAudioFrame) => {
  // Playback data (post-mixing) callback.
  console.info('onPlaybackAudioFrame');
  return true;
});

4. Disable the observer

Android

mAliRtcEngine.enableAudioFrameObserver(false, AliRtcAudioSourcePlayback, config)

iOS

engine.enableAudioFrameObserver(false, audioSource: audioSource, config: observerConfig)

Mac

[self.engine enableAudioFrameObserver:false audioSource:AliRtcAudioSourcePlayback config:config];

Windows

mAliRtcEngine->EnableAudioFrameObserver(false, AliEngineAudioSourcePlayback, config);

Harmony

this.rtcEngine.enableAudioFrameObserver(false, AliRtcAudioSource.AliRtcAudioSourcePlayback, config);

5. (Optional) Dynamically enable or disable internal playback

To dynamically enable or disable the SDK's internal playback during a call, call the setParameter method.

Android

/* Dynamically disable internal playback. */
String parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}";
mAliRtcEngine.setParameter(parameter);

/* Dynamically enable internal playback. */
String parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}"; 
mAliRtcEngine.setParameter(parameter);

iOS

// Dynamically disable internal playback.
engine.setParameter("{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}")
// Dynamically enable internal playback.
engine.setParameter("{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}")

Mac

// Dynamically disable internal playback.
[self.engine setParameter:"{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}"];
// Dynamically enable internal playback.
[self.engine setParameter:"{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}"];

Windows

/* Dynamically disable internal playback. */
std::string parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}";
mAliRtcEngine->SetParameter(parameter.c_str());

/* Dynamically enable internal playback. */
std::string parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}"; 
mAliRtcEngine->SetParameter(parameter.c_str());

Harmony

/* Dynamically disable internal playback. */
this.rtcEngine.setParameter("{"audio":{"enable_system_audio_device_play":"FALSE"}}");

/* Dynamically enable internal playback. */
this.rtcEngine.setParameter("{"audio":{"enable_system_audio_device_play":"TRUE"}}");