iOS SDK

Initialize the SDK

Implement NeoNuiSdkDelegate and the recording module's ConvVoiceRecorderDelegate in the view controller. Retain the SDK instance, recorder instance, and audio buffer:

#import <nuisdk/NeoNui.h>
#import "audio/AudioController.h"

@interface SpeechRecognizerViewController () <NeoNuiSdkDelegate, ConvVoiceRecorderDelegate>
@property(nonatomic, strong) NeoNui *nui;
@property(nonatomic, strong) AudioController *audioController;
@property(nonatomic, strong) NSMutableData *recordedVoiceData;
@end

Use get_instance to obtain the singleton. If multiple instances are required, use [[NeoNui alloc] init]. Call nui_release before initializing the same instance again.

Configure the service URL:

NSString *gateway = @"wss://nls-gateway.cn-shanghai.aliyuncs.com:443/ws/v1";

Replace <Appkey> and <NLS Token> with the project Appkey and a valid token from your server. Initialize the SDK on a work queue:

_nui = [NeoNui get_instance];
_nui.delegate = self;
_recordedVoiceData = [NSMutableData data];
_audioController = [[AudioController alloc] init:only_recorder];
_audioController.delegate = self;

NSDictionary *initParams = @{
    @"app_key": @"<Appkey>",
    @"token": @"<NLS Token>",
    @"url": gateway,
    @"service_mode": @"4"
};
NSData *initData = [NSJSONSerialization dataWithJSONObject:initParams options:0 error:nil];
NSString *initJSON = [[NSString alloc] initWithData:initData encoding:NSUTF8StringEncoding];
NuiResultCode result = [_nui nui_initialize:initJSON.UTF8String
                                 logLevel:NUI_LOG_LEVEL_NONE
                                  saveLog:NO];
if (result != 0) {
    NSLog(@"nui_initialize failed: %d", result);
    return;
}

Starting with version 2.6.2, cloud-only features do not require workspace or device_id. For earlier versions that require workspace, set it to the resource path of Resources.bundle. The optional device_id helps locate issues and does not need to be an advertising identifier.

For debugging, set the log level to NUI_LOG_LEVEL_VERBOSE. When saveLog is enabled, set debug_path to a writable directory. Use save_wav to control whether debug audio is saved. Log files have no size limit. Clean them up regularly to avoid exhausting storage.

Set parameters

Set recognition parameters in JSON format. enable_intermediate_result is optional. Set it to YES if intermediate results are needed.

NSMutableDictionary *nlsConfig = [@{
    @"enable_intermediate_result": @YES,
    @"sample_rate": @16000,
    @"sr_format": @"pcm"
} mutableCopy];
NSDictionary *params = @{
    @"service_type": @(SERVICE_TYPE_ASR),
    @"nls_config": nlsConfig
};
NSData *paramsData = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];
NSString *paramsJSON = [[NSString alloc] initWithData:paramsData encoding:NSUTF8StringEncoding];
NuiResultCode result = [_nui nui_set_params:paramsJSON.UTF8String];
if (result != 0) {
    NSLog(@"nui_set_params failed: %d", result);
    return;
}

The 01B version does not include a local VAD module. The 029 version, which provides wake-up functionality, includes local VAD. To automatically detect the end of a sentence, add the following online VAD parameters after creating nlsConfig and before serializing it. Continue to use MODE_P2T when starting recognition:

nlsConfig[@"enable_voice_detection"] = @YES;
nlsConfig[@"max_start_silence"] = @10000;
nlsConfig[@"max_end_silence"] = @800;

For the meanings and values of parameters such as punctuation prediction, inverse text normalization, sample rate, and audio transmission format, see API reference.

If HTTPDNS is used, set the top-level direct_ip parameter to the resolved IP address. Use the top-level extend_config object to pass parameters that the service supports but that are not separately listed in the SDK.

Start recognition

NuiResultCode result = [_nui nui_dialog_start:MODE_P2T dialogParam:"{}"];
if (result != 0) {
    NSLog(@"nui_dialog_start failed: %d", result);
}

To update the Appkey or token for the current recognition task, include app_key or token in the dialogParam JSON string. If these temporary parameters are omitted from the next task, the values supplied during initialization are used.

Handle callbacks

Recording state and audio input

Start or stop recording based on the SDK audio state. Store audio from the recorder callback in the buffer, then supply it to the SDK through onNuiNeedAudioData.

- (void)onNuiAudioStateChanged:(NuiAudioState)state {
    if (state == STATE_OPEN) {
        @synchronized (self) {
            [_recordedVoiceData setLength:0];
        }
        [_audioController startRecorder];
    } else if (state == STATE_CLOSE || state == STATE_PAUSE) {
        [_audioController stopRecorder:NO];
    }
}

- (void)voiceDidFail:(NSError *)error {
    NSLog(@"Recording failed: %@", error.localizedDescription);
}

- (void)voiceRecorded:(unsigned char *)buffer Length:(int)len {
    @synchronized (self) {
        [_recordedVoiceData appendBytes:buffer length:(NSUInteger)len];
    }
}

- (int)onNuiNeedAudioData:(char *)audioData length:(int)len {
    @synchronized (self) {
        NSUInteger count = MIN(_recordedVoiceData.length, (NSUInteger)len);
        if (count == 0) return 0;
        [_recordedVoiceData getBytes:audioData length:count];
        [_recordedVoiceData replaceBytesInRange:NSMakeRange(0, count)
                                     withBytes:NULL length:0];
        return (int)count;
    }
}

Recognition results and errors

asr_result contains the recognition result or error information. Use task_id to help troubleshoot requests. A true finish value means the task has ended, either normally or because of an error.

- (void)onNuiEventCallback:(NuiCallbackEvent)event
                   dialog:(long)dialog
                kwsResult:(const char *)wuw
                asrResult:(const char *)asrResult
                 ifFinish:(BOOL)finish
                  retCode:(int)code {
    NSString *message = asrResult ? [NSString stringWithUTF8String:asrResult] : @"";
    if (event == EVENT_ASR_PARTIAL_RESULT || event == EVENT_ASR_RESULT) {
        NSLog(@"ASR result: %@", message);
    } else if (event == EVENT_ASR_ERROR || event == EVENT_MIC_ERROR) {
        NSLog(@"ASR error: %d, %@", code, message);
    }
    if (finish) {
        dispatch_async(dispatch_get_main_queue(), ^{
            // Update the UI to allow the next recognition task to start.
        });
    }
}

Stop recognition and release resources

For a normal stop, pass NO and wait for the final result and task completion callback. If the audio contains no valid speech, the final recognition text may be empty.

[_nui nui_dialog_cancel:NO];

If the result is no longer needed, call nui_dialog_cancel:YES to force cancellation and perform cleanup in the application. When the SDK is no longer needed, release resources and clear the delegates outside SDK callbacks:

[_audioController stopRecorder:NO];
[_nui nui_release];
_nui.delegate = nil;
_audioController.delegate = nil;

What do I do if NeoNui.h cannot be found?

Confirm that the SDK is added to and linked with the project. Import the header with #import <nuisdk/NeoNui.h>, not #import "nuisdk.framework/Headers/NeoNui.h". Dynamic libraries must also be set to Embed & Sign.

Can mobile SDKs be used in a private cloud?

Mobile SDKs can call public cloud ASR and TTS services and can also be used in a private cloud. Private cloud installation packages do not include mobile SDKs by default. Download them from the documentation for the corresponding service, such as the Android SDK and iOS SDK for real-time speech recognition.

Is background processing supported?

The SDK itself does not restrict foreground or background use. Background recording also depends on the iOS audio session and application configuration. To record in the background:

1. Add audio under Required background modes (UIBackgroundModes) in Info.plist. Its display value is App plays audio or streams audio/video using AirPlay.

2. Check the recording module's background handling. Do not stop recording or deactivate the audio session when the application enters the background. In the older NLSVoiceRecorder.m module, remove the AudioSessionSetActive(NO) call from _appResignActive.

The 2.7.4 demo uses the AudioController module and configures UIBackgroundModes=audio. Its background handler does not call AudioSessionSetActive(NO).

What do I do if a device reports no suitable image found?

First confirm that the linked library targets a physical device. Then try deleting the application from the device, cleaning build artifacts, and reinstalling it. Also check that the application and dynamic library signatures, certificates, and provisioning profiles match. If the enterprise certificate is confirmed to be invalid, create a valid certificate and provisioning profile, then sign and package the application again.

What do I do if EVENT_MIC_ERROR occurs?

Check microphone permission, whether another module is using the recorder, and whether onNuiNeedAudioData continuously supplies audio. This event occurs if no audio data is supplied for 2 consecutive seconds.

What do I do if a library targets both iOS and iOS Simulator?

If the build reports Building for iOS, but the linked and embedded framework ... was built for iOS + iOS Simulator, check whether an older mixed-platform Framework is used. Integrate nuisdk.xcframework from the SDK package instead, so Xcode selects the library for the build destination. Do not embed a simulator library in a device application.

What do I do if App Store reports Unsupported Architectures?

If an older Framework contains simulator architectures such as x86_64 or i386, inspect it with lipo -info <Framework-binary> and remove simulator architectures from the device distribution package. When using an XCFramework, archive for a device destination so Xcode selects the device library.

What do I do if an older project requires Legacy Build System?

For older Xcode projects that still provide Validate Workspace, try setting it to Yes and rebuilding. New projects should integrate the XCFramework provided in the current SDK package.