iOS and Mac-Real-Time Communication(RTC)-阿里云帮助中心

The RTC SDK provides the function to obtain audio data. You can process the obtained audio data based on your actual needs. By reading this article, you can learn how to obtain audio data.

Scenarios

You can convert audio data from a local publisher or subscriber to text by using the Alibaba Cloud speech recognition service. The implementation process is as follows:

Alibaba Cloud RTC sends audio data to the audio recognition SDK.
The audio recognition SDK sends audio data to the audio recognition service for real-time speech processing and returns recognition results.
The audio recognition SDK provides recognition results for users.

For more information, see Intelligent Speech Interaction.

Architecture

Call sequence diagram

Interface and usage

Obtain callback data by calling the interface subscribeAudioData, obtain audio data from the callback interface onAudioSampleCallback, and use the corresponding data source according to the business scenario.

The onAudioSampleCallback interface parameters are as follows:

Parameter	Type	Description
audioSource	AliRtcAudioSource	The type of the bare audio data source.
audioSample	AliRtcAudioDataSample *	The bare audio data.

Speech data processing

RTC obtains audio data as follows:

Set Join Mode to Music Mode.

Interface method:

+ (instancetype)sharedInstance:(id<AliRtcEngineDelegate>)delegate extras:(NSString *)extras;

Set the extras parameter to Music Mode. For more information, seeextras parameter configuration.

{
  "user_specified_scene_mode" : "SCENE_MUSIC_MODE",
}

Note extras is a JSON string. If the extras configuration of other key values is used, it can coexist.

The sample code for music mode is as follows:

NSMutableDictionary *extrasDic = [[NSMutableDictionary alloc] init];
[extrasDic setValue:@"ENGINE_BASIC_QUALITY_MODE" forKey:@"user_specified_engine_mode"];
[extrasDic setValue:@"SCENE_MUSIC_MODE" forKey:@"user_specified_scene_mode"];
NSError *parseError = nil;
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:extrasDic options:NSJSONWritingPrettyPrinted error:&parseError];
NSString *extras = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];
AliRtcEngine *engine = [AliRtcEngine sharedInstance:self extras:extras];

You can call this operation to obtain the bare data of the audio publisher.

To register audio bare data, call the following interfaces:
```
- (void)subscribeAudioData:(AliRtcAudioSource)audioSource;
```
Note This interface must be called after the audio stream is successfully published to take effect. After you call this operation, you can start to subscribe to audio bare data, which is returned through the callback operation.
Stop obtaining audio bare data and call the following interfaces:
```
- (void)unSubscribeAudioData:(AliRtcAudioSource)audioSource;
```

Use the callback to obtain the corresponding audio bare data. The callback interface is as follows:

Note After you call the subscribeAudioData operation, use this callback to obtain the corresponding audio bare data.

{
// Start the subscription (after the audio stream is successfully published)
[self.engine subscribeAudioData:(AliRtcAudiosourcePub)];
}

// Callback of the subscribed audio data.
- (void)onAudioSampleCallback:(AliRtcAudioSource)audioSource audioSample:(AliRtcAudioDataSample *)audioSample {

    if (audioSample.dataPtr == 0) {
        // No audio subscription data.
        return;
    }

    if (audioSource == AliRtcAudiosourcePub) {
        // Pub data callback
    }

    // Obtain the buffer and bufferSize.
    int bufferSize = audioSample.numOfSamples * audioSample.bytesPerSample * audioSample.numOfChannels;
    void *audioSampleBufferPtr = NULL;

    if(bufferSize) {
        audioSampleBufferPtr = malloc(bufferSize);
        if(audioSampleBufferPtr) {
            memcpy(audioSampleBufferPtr, (void *)audioSample.dataPtr, bufferSize);
        }
    }

    // Write the file (the initialization and destruction of the FILE object are omitted here, please write it externally)    
    fwrite(audioSampleBufferPtr, 1, bufferSize, file);

    if (audioSampleBufferPtr) {
        free(audioSampleBufferPtr);
        audioSampleBufferPtr = NULL;
    }
}