iOS and Mac-Real-Time Communication(RTC)-阿里云帮助中心

By reading this article, you can learn how to output audio and video data.

Output video data

Register naked video data.
The interface method is as follows:
```
- (void)registerVideoSampleObserver;
```
Note After you call this operation, you can start to subscribe to the bare video data. The bare data is given through the callback operation.
To stop obtaining bare audio data, call the following operation:
```
- (void)unregisterVideoSampleObserver;
```

Obtain the naked data of a video by using a callback.

The callback interface method is as follows:

/**
* @brief Callback for the locally collected video data subscribed to.
* @param videoSource video source
* @param videoSample video sample
*/
- (void)onCaptureVideoSample:(AliRtcVideoSource)videoSource videoSample:(AliRtcVideoDataSample *)videoSample;

/**
* @brief The callback for the subscribed remote video data.
* @param uid user id
* @param videoSource video source
* @param videoSample video sample
*/
- (void)onRemoteVideoSample:(NSString *)uid videoSource:(AliRtcVideoSource)videoSource videoSample:(AliRtcVideoDataSample *)videoSample;

Note

After you call the registerVideoSampleObserver operation, use the following two callbacks to obtain the corresponding video bare data.

The onCaptureVideoSample is a preview data callback. You can receive a data stream after the preview starts.
The onRemoteVideoSample is a stream-pulling data callback. You can receive the data stream after the subscribe is successfully pulled.

The preview video is written to a local yuv file.

{
// Start the subscription.
[_engine registerVideoSampleObserver];
}

- (void)onCaptureVideoSample:(AliRtcVideoSource)videoSource videoSample:(AliRtcVideoDataSample *)videoSample {

    [self dumpVideoYuvData:@"" videoSource:videoSource videoSample:videoSample];
}

- (void)onRemoteVideoSample:(NSString *)uid videoSource:(AliRtcVideoSource)videoSource videoSample:(AliRtcVideoDataSample *)videoSample {
    [self dumpVideoYuvData:uid videoSource:videoSource videoSample:videoSample];
}

- (void)dumpVideoYuvData:(NSString *)uid videoSource:(AliRtcVideoSource)videoSource videoSample:(AliRtcVideoDataSample *)videoSample {
    // Create a serial queue.
       dispatch_queue_t testqueue = dispatch_queue_create("subVideo", NULL);

       // Synchronously execute the task.
       dispatch_sync(testqueue, ^{
           if(_dumpYUVVideoData || _remoteYUVVideoData){

               if (videoSource != AliRtcVideosourceCameraLargeType) {
                   return;
               }

               if (!_subDataMutableDic[uid]) {
                   NSString *time = [self getCurrentTimes];

                   NSString *filePath = [NSString stringWithFormat:@"Documents/yuv/%@/",uid];
                   if ([uid isEqualToString:@""]) {
                       filePath = [NSString stringWithFormat:@"Documents/yuv/self/"];
                   }

                   NSString *fileName = [NSString stringWithFormat:@"sub_%@_video_%d*%d_%@.yuv",uid,videoSample.width,videoSample.height,time];
                   if ([uid isEqualToString:@""]) {
                       fileName = [NSString stringWithFormat:@"pub_%@_video_%d*%d_%@.yuv",uid,videoSample.width,videoSample.height,time];
                   }

                   NSString * dataPath = [SubFilePath getSubDataFilePath:filePath FileName:fileName];

                   subYUVFile = fopen([dataPath UTF8String], "ab");

                   SubDataModel *cache = [[SubDataModel alloc] init];
                   cache.filePath = dataPath;
                   cache.outputFile = subYUVFile;
                   [_subDataMutableDic setObject:cache forKey:uid];
               }

               SubDataModel *cache = _subDataMutableDic[uid];
               subYUVFile = cache.outputFile;

               uint8_t *outputData = NULL;
               int32_t w = videoSample.width;
               int32_t h = videoSample.height;

               if (outputData == NULL) {
                   outputData = (uint8_t*)malloc(w*h*3/2);
               }

               if(videoSample.type == AliRtcBufferType_Raw_Data) {

                   if(videoSample.dataYPtr && videoSample.dataUPtr &&  videoSample.dataVPtr) {
                       memcpy(outputData, (uint8_t*)videoSample.dataYPtr,w*h*3/2);
                       memcpy(outputData + w*h, (uint8_t*)videoSample.dataUPtr, w*h/4);
                       memcpy(outputData + w*h*5/4, (uint8_t*)videoSample.dataVPtr, w*h/4);
                   }

               }else if(videoSample.type == AliRtcBufferType_CVPixelBuffer) {
                   if(videoSample.pixelBuffer) {
                       CVPixelBufferRef newBuffer = videoSample.pixelBuffer;
                       if(newBuffer) {
                           CVReturn cvRet = CVPixelBufferLockBaseAddress(newBuffer, 0);
                            if ( cvRet != kCVReturnSuccess ) {
                                return;
                            }
                            void*  src_y = CVPixelBufferGetBaseAddressOfPlane(newBuffer, 0);
                            void*  src_uv = CVPixelBufferGetBaseAddressOfPlane(newBuffer, 1);

                            size_t src_y_stride = CVPixelBufferGetBytesPerRowOfPlane(newBuffer, 0);
                            size_t src_uv_stride = CVPixelBufferGetBytesPerRowOfPlane(newBuffer, 1);

                            uint8_t *destData = outputData;

                            for (int row = 0; row < h; ++row) {
                                memcpy(destData, (uint8_t*)src_y + row * src_y_stride,w);
                                destData += w;
                            }

                            for (int row = 0; row < (h + 1) / 2; ++row) {
                                memcpy(destData, (uint8_t*)src_uv + row * src_uv_stride,w);
                                destData += w;
                            }

                            CVPixelBufferUnlockBaseAddress(newBuffer, 0);
                       }
                   }
               }

               fwrite(outputData, 1, w*h*3/2, subYUVFile);

               if (outputData) {
                   free(outputData);
                   outputData = NULL;
               }
           }
       });
}

Output audio bare data

Set Join Mode to Music Mode.

Interface method:

+ (instancetype)sharedInstance:(id<AliRtcEngineDelegate>)delegate extras:(NSString *)extras;

You can set extras parameters to control whether it is in music mode.

{
  "user_specified_scene_mode" : "SCENE_MUSIC_MODE",
}

Note extras is a JSON string. If the extras configuration of other key values is used, it can coexist.

The following example shows the sample code for music mode.

NSMutableDictionary *extrasDic = [[NSMutableDictionary alloc] init];
[extrasDic setValue:@"ENGINE_BASIC_QUALITY_MODE" forKey:@"user_specified_engine_mode"];
[extrasDic setValue:@"SCENE_MUSIC_MODE" forKey:@"user_specified_scene_mode"];
NSError *parseError = nil;
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:extrasDic options:NSJSONWritingPrettyPrinted error:&parseError];
NSString *extras = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];

AliRtcEngine *engine = [AliRtcEngine sharedInstance:self extras:extras];

Obtains the audio ingest bare data.

Registers audio bare data.
The interface method is as follows:
```
- (void)subscribeAudioData:(AliRtcAudioSource)audioSource;
```
Note This interface must be called after the audio stream is successfully published to take effect. After this interface is called, you can start to subscribe to the audio bare data, and the bare data is returned through the callback interface.
Parameter description:
Parameter Description
AliRtcAudioSource
AliRtcAudiosourcePub: data at the ingest end.
AliRtcAudiosourceSub: pull data on the stream end.
To stop obtaining bare audio data, call the following operations:
```
- (void)unSubscribeAudioData:(AliRtcAudioSource)audioSource;
```

Obtain the corresponding audio bare data through the callback.

The callback interface is as follows:

- (void)onAudioSampleCallback:(AliRtcAudioSource)audioSource audioSample:(AliRtcAudioDataSample *)audioSample;

Note After you call the subscribeAudioData operation, use this callback to obtain the corresponding audio bare data.

The AliRtcAudioDataSample parameters are described as follows:

Parameter	Type	Description
dataPtr	long	Naked data.
numOfSamples	int	The number of audio sample points.
bytesPerSample	int	The number of quantized digits.
numOfChannels	int	The number of sound channels.
samplesPerSec	int	The sampling rate.

Stream audio bare data is written to a local pcm file.

{
// Start the subscription (after the audio stream is successfully published)
[self.engine subscribeAudioData:(AliRtcAudiosourcePub)];
}

// Callback of the subscribed audio data.
- (void)onAudioSampleCallback:(AliRtcAudioSource)audioSource audioSample:(AliRtcAudioDataSample *)audioSample {

    if (audioSample.dataPtr == 0) {
        // No audio subscription data.
        return;
    }

    if (audioSource == AliRtcAudiosourcePub) {
        // Pub data callback
    }

    // Obtain the buffer and bufferSize.
    int bufferSize = audioSample.numOfSamples * audioSample.bytesPerSample * audioSample.numOfChannels;
    void *audioSampleBufferPtr = NULL;

    if(bufferSize) {
        audioSampleBufferPtr = malloc(bufferSize);
        if(audioSampleBufferPtr) {
            memcpy(audioSampleBufferPtr, (void *)audioSample.dataPtr, bufferSize);
        }
    }

    // Write the file (the initialization and destruction of the FILE object are omitted here, please write it externally)    
    fwrite(audioSampleBufferPtr, 1, bufferSize, file);

    if (audioSampleBufferPtr) {
        free(audioSampleBufferPtr);
        audioSampleBufferPtr = NULL;
    }
}

Parameter	Description
AliRtcAudioSource	AliRtcAudiosourcePub: data at the ingest end. AliRtcAudiosourceSub: pull data on the stream end.