Advanced features for iOS-Mobile Platform as a Service(mPaaS)-阿里云帮助中心

This topic describes the advanced features provided by Mobile Real-Time Communication (MRTC) APIs on the iOS platform.

Rendering view related APIs

Implement the callback APIs related to rendering in ARTVCEngineDelegate.

Create a rendering view object.

// The rendering view is created and associated with the feed. When the business side receives the callback, this view will be added to the layout. // Sets its frame. 
-(void)didVideoRenderViewInitialized:(UIView*)renderView forFeed:(ARTVCFeed*)feed{
}

Render the first video frame.

-(void)didFirstVideoFrameRendered:(UIView*)renderView forFeed:(ARTVCFeed*)feed{
}

Stop rendering the video.

-(void)didVideoViewRenderStopped:(UIView*)renderView forFeed:(ARTVCFeed*)feed{
}

Unified error handling

All errors. A unified error callback API is used to return business operation results. The business side handles errors differently depending on the error code.
```
-(void)didEncounterError:(NSError *)error forFeed:(ARTVCFeed*)feed{
  // The business side handles errors differently depending on the error code. 
}
```

The errors are defined below:

typedef NS_ENUM(int,ARTVCErrorCode){
  /**
  bad parameters passed to API
  */
  ARTVCErrorCodeBadParameters                  = - 103,
  /**
   camera permission is denied by user
   without this permission,video call cann't be continued,please advise user enable camera permission in settings.
   */
  ARTVCErrorCodeCameraPermissionNOtAllowed     = -104,
  /**
  microphone permission is denied by user
  without this permission,video call cann't be continued,please advise user enable camera permission in settings.
  */
  ARTVCErrorCodeMicrophonePermissionNOtAllowed = -105,
  /**
  timeout happened,publish/subscribe cann't be finishied successfully
  */
  ARTVCErrorCodeTimeout                        = -108,
  /**
   you has already published or subsrcibed a feed .
  you cann't publish or subsrcibe the same feed once again when it has NOt been unpublished or unsubscribed.
  */
  ARTVCErrorCodeAlreadyPublishedOrSubsrcibed   = -111,
  /**
   you has NOt published or subsrcibed the feed .so you cann't do unpublish or unsubscribe operation.
  */
  ARTVCErrorCodeFeedHasNOtBeenPublishedOrSubsrcibed   = -119,
  /**
  internal  webrtc-relative error when doing publish/subscribe,for example,setting sdp failed,creating sdp failed,e.g.
  */
  ARTVCErrorCodeInternalError                  = -113,
  /**
   * current room has become invalid .most of all,it's because network's down,heartbeat abNOrmal.
   * if you wanna continue,you MUST call createRoom again to get a new valid room.
   */
  ARTVCErrorCodeCurrentRoomHasBecomeInvalid    = -114,
  /**
  server error hanppened.CreateRoom request failed,it's a server internal error.
  */
  ARTVCErrorCodeProtocolErrorCreateRoomFailed  = -115,
  /**
  server error hanppened.JoinRoom request failed,it's a server internal error.maybe the room you joined has been became invalid yet.
  */
  ARTVCErrorCodeProtocolErrorJoinRoomFailed    = -116,
  /**
  server error hanppened.Publish request failed,it's a server internal error.
  */
  ARTVCErrorCodeProtocolErrorPublishFailed     = -117,
  /**
  server error hanppened.Subscribe request failed,it's a server internal error.maybe the stream you subscribed has been unpunlished yet or some error else.
  */
  ARTVCErrorCodeProtocolErrorSubscribeFailed   = -118,
};

Switch between the front and rear cameras

The code for switching between the front and rear cameras is as follows:

[_artvcEgnie switchCamera];

Mute the remote video

The code for muting the remote video is as follows:

ARTVCFeed* feed = the remote feed to be muted;
[_artvcEgnie muteRemoteVideo:YES forFeed:feed];

Mute the microphone

The code for muting the microphone is as follows:

[_artvcEgnie muteMicrophone:YES];

Mute the remote audio

The code for muting the remote audio is as follows:

ARTVCFeed* feed = the remote feed to be muted;
[_artvcEgnie muteRemoteAudio:YES forFeed:feed];

Speaker mode of earpiece

Switch the mode:

[_artvcEgnie switchAudioPlayModeTo:ARTVCAudioPlayModeReceiver complete:nil];

Change notification:

-(void)didAudioPlayModeChangedTo:(ARTVCAudioPlayMode)audioPlayMode{
  NSString *toast = nil;
  switch (audioPlayMode) {
      case ARTVCAudioPlayModeSpeaker:{
          toast = @"Speaker mode";
      }
          break;
      case ARTVCAudioPlayModeReceiver:{
          toast = @"Earpiece mode";
      }
          break;
      case ARTVCAudioPlayModeHeadphone:{
          toast = @"Headphone mode";
      }
          break;
      case ARTVCAudioPlayModeBluetooth:{
          toast = @"Bluetooth mode";
      }
          break;
      case ARTVCAudioPlayModeInit:{
          toast = @"Unknown mode";
      }
          break;
  }

  [self showToastWith:toast duration:2.0];
}

Network change notification

The following is the code for returning network change notifications. For mobile networks, a window can pop up to notify users of network traffic risks.

-(void)didNetworkChangedTo:(APMNetworkReachabilityStatus)netStatus{
    if(netStatus == APMNetReachabilityStatusReachableViaWiFi){
        return {
    }
    [self showToastWith:[NSString stringWithFormat:@"Network is switched to:%@",[APMNetworkStatusManager stringOfNetworkStatus:netStatus]] duration:2.0];
}

Low bandwidth notification

The code for returning low bandwidth notifications is as follows:

-(void)didAvailabeSendBandwidthBecomeLow:(BOOL)isLow currentBandwidth:(double)bw forFeed:(ARTVCFeed*)feed{
    if(isLow){
        [self showToastWith:@"The current call quality is poor" duration:2.0];
    }
}

Snapshot taking

You can use the following code to take a snapshot of any stream:

ARTVCFeed* feed = the feed of which a snapshot is to be taken;
[_artvcEgnie snapshotForFeed:feed  complete:^(UIImage* image){
// Handle the snapshot as required.
}];

Query the debug information about the call quality

Return debug information of any stream through the callback API.

/**
 brief debug information is generated(including bitrate/cpu/codec,e.g.),you can show this on your debug information view.
 */
- (void)didBriefDebugInformationGenerated:(NSString*)debugInfo forFeed:(ARTVCFeed*)feed{
// If necessary, display the debug information by stream in the Debug window. 
}

Query the real-time call quality monitoring information

Query the real-time monitoring information of any stream through the callback API, including the bitrate, frame rate, and CPU.
```
-(void)didRealtimeStatisticGenerated:(ARTVCRealtimeStatisticSummary*)summary forFeed:(ARTVCFeed*)feed{
}
```

The returned data is as follows:

@interface ARTVCRealtimeStatisticSummary : NSObject
// connection stats googCandidatePair
/** The total sent bitrate, in bps*/
@property(NOnatomic,copy) NSString* totalSendBitrate;
/** The total received bitrate, in bps*/
@property(NOnatomic,copy) NSString* totalRecvBitrate;
/** The network latency in milliseconds*/
/** The network latency in milliseconds*/
@property(NOnatomic,copy) NSString* rtt;
// video send
/** The sent video bitrate, in bps*/
@property(NOnatomic,copy) NSString* videoSendBitrate;
/** The actual video frame rate that is sent */
@property(NOnatomic,copy) NSString* videoSendFps;
// video recv
/** The received video bitrate, in bps*/
@property(NOnatomic,copy) NSString* videoRecvBitrate;
/** The actual video frame rate that is received */
@property(NOnatomic,copy) NSString* videoRecvFps;
/** The sent audio bitrate, in bps*/
@property(NOnatomic,copy) NSString* audioSendBitrate;
/** The received audio bitrate, in bps*/
@property(NOnatomic,copy) NSString* audioRecvBitrate;
/** The packet loss rate when sending a video*/
@property(NOnatomic,copy) NSString* videoLossRate;
// audio send
/** The packet loss rate when sending an audio file*/
@property(NOnatomic,copy) NSString* audioLossRate;
/** cpu */
@property(NOnatomic,copy) NSString* cpu;
@end

Dynamically adjust the encoding resolution and camera FPS during a call (only for built-in cameras)

To set the resolution to 640 × 360 and use 15 FPS, use the following setting:
```
_artvcEgnine.videoProfileType = ARTVCVideoProfileType_640x360_15Fps;
```
For example, to increase the resolution to 960 × 540 and use 30 FPS due to business requirements, use the following setting:
```
_artvcEgnine.videoProfileType = ARTVCVideoProfileType_960x540_30Fps;
```
To restore the resolution to 640 × 360 and continue to use 15 FPS after specific business tasks are handled, use the following setting:
```
_artvcEgnine.videoProfileType = ARTVCVideoProfileType_640x360_15Fps;
```

Dynamically adjust the encoding resolution during a call (for all video sources)

In a custom ingest-stream, the resolution is 640 × 360. If you need to increase the resolution to 960 × 540 to meet the business requirements, you can use the following setting:
```
[ _artvcEgnine changeVideoProfileTo:ARTVCVideoProfileType_960x540_15Fps forVideoSource:ARTVCVideoSourceType_Custom];
```
The current resolution of the built-in camera is 640 × 360. If you need to increase the resolution to 960 × 540 to meet the business requirements, you can use the following setting:
```
[ _artvcEgnine changeVideoProfileTo:ARTVCVideoProfileType_960x540_30Fps forVideoSource:ARTVCVideoSourceType_Camera];
```

Customize a video ingest-stream

Limitations on customizing a video ingest-stream

Only the video frame in NV12 format is supported, and the input format is CVPixelBufferRef.
Currently, the caller must ensure that the FPS of the input source meets the requirements. We recommend that you use 15 FPS to 24 FPS.
Only the manual mode of stream ingestion is supported.

Process

The process for customizing a video ingest-stream is as follows:

// Set to the manual mode during the customization.
[ _artvcEgnine setAutoPublish:NO];
 // Create a custom ingest-stream class.
ARTVCCreateCustomVideoCaputurerParams* params = [[ARTVCCreateCustomVideoCaputurerParams alloc] init];
// If a SDK is required for rendering, set provideRenderView to YES. By default, the SDK does not render views for the business side. 
            params.provideRenderView = YES;


            self.customCapturer = [_artvcEgnine createCustomVideoCapturer:params];
            // Send data in CVPixelBufferRef format to the SDK based on a certain frequency. Only NV12 format is supported. 
            [self.customCapturer provideCustomVideoFramePeriodlyWith:CVPixelBufferRef];
// Actively call the ingest-stream.
ARTVCPublishConfig* config = [[ARTVCPublishConfig alloc] init];
            config.videoSource = ARTVCVideoSourceType_Custom;
            config.videoProfile = ARTVCVideoProfileType_640x360_15Fps;
            self.customPublishConfig = config;
            [_artvcEgnine publish:config];

Screen sharing

The code for sharing a screen is as follows:

// Set to the manual stream ingestion mode.
[ _artvcEgnine setAutoPublish:NO];
 // Start the screen sharing.
-(void)startScreenSharing{
    NSLog(@"start screen sharing");
    ARTVCCreateScreenCaputurerParams* screenParams = [[ARTVCCreateScreenCaputurerParams alloc] init];
    screenParams.provideRenderView = YES;
    [_artvcEgnine startScreenCaptureWithParams:screenParams complete:^(NSError* error){
        NSLog(@"start screen sharing finish,error:%@",error);
        if(error){
             // For specific code, see the error code part in the section about enabling screen capturing.

        }else{
           // Call the publish API to start stream ingestion after the screen capturing is successful.
            ARTVCPublishConfig* config = [[ARTVCPublishConfig alloc] init];
            config.videoSource = ARTVCVideoSourceType_Screen;
            config.audioEnable = NO;
            config.videoProfile = ARTVCVideoProfileType_1280x720_30Fps;
            [_artvcEgnine publish:config];
        }
    }];
}
// Stop screen sharing.
-(void)stopScreenSharing{
    NSLog(@"stop screen sharing");
    // Stop screen capturing.
    [_artvcEgnine stopScreenCapture];
    // Cancel the publishing of the screen sharing stream.
    ARTVCUnpublishConfig* config = [[ARTVCUnpublishConfig alloc] init];
    config.feed = self.screenLocalFeed;
    [_artvcEgnine unpublish:config];
}

Error code captured during screen capturing

The error code captured during screen capturing is as follows:

/**
    screen capture alrady under running, you cann't start it again before you call stop
    */
    ARTVCErrorCodeScreenCapturerAlreadyUnderRunning            = -1011,
    /**
    starting screen capture failed
    */
    ARTVCErrorCodeStartScreenCaptureFailed                     = -1012,
    /**
    start screen capture success,but may be encounter errors during the processing of the capure operation.
    */
    ARTVCErrorCodeScreenCaptureFailedInProcessing              = -1013,