Mobile SDK guide

Updated at:

Use the NUI SDK for Android or iOS to upload local audio files to the FlashRecognizer service and obtain transcription results. This topic describes endpoints, the call sequence, request parameters, and response fields.

Prerequisites and limitations

  • Activate the commercial edition of the FlashRecognizer service. The service does not offer a free trial. For activation, billing, and concurrency limits, see the FlashRecognizer API reference.

  • Obtain a project Appkey and a valid NLS token. See Manage projects and Obtain a token, respectively. Configure the recognition language and model in the project. The selected model must match the language of the audio.

  • Supported audio formats are WAV, MP3, AAC, MP4, and OPUS. Files must not exceed 100 MB or 2 hours in duration.

Important

Keep AccessKey IDs and AccessKey secrets on the server. Do not embed them in app code or store them on mobile devices. Obtain an NLS token on the server, send it to the app, and update it before it expires. Initialization parameters contain credentials. Do not log the complete parameter string.

Obtain the SDK

Download an SDK package that supports audio file transcription from Choose and download a mobile SDK, and review the compatibility and compliance requirements on that page. The package includes FileTranscriberActivity.java for Android and FileTranscriberViewController.m for iOS.

For integration instructions and code examples, see Android SDK and iOS SDK.

Starting with SDK V2.6.2, cloud-only features do not require resource files to be copied or a workspace to be configured.

Endpoints

Set the service endpoint in the url initialization parameter. Mobile devices access the service over the Internet. VPC endpoints are for runtime environments with VPC network access in the corresponding region.

Access type

Region

URL

Internet

Shanghai

https://nls-gateway-cn-shanghai.aliyuncs.com/stream/v1/FlashRecognizer

Beijing

https://nls-gateway-cn-beijing.aliyuncs.com/stream/v1/FlashRecognizer

Shenzhen

https://nls-gateway-cn-shenzhen.aliyuncs.com/stream/v1/FlashRecognizer

VPC

Shanghai

http://nls-gateway-cn-shanghai-internal.aliyuncs.com/stream/v1/FlashRecognizer

Beijing

http://nls-gateway-cn-beijing-internal.aliyuncs.com/stream/v1/FlashRecognizer

Shenzhen

http://nls-gateway-cn-shenzhen-internal.aliyuncs.com/stream/v1/FlashRecognizer

Host is the domain name in the URL. For example, the Shanghai Internet endpoint uses nls-gateway-cn-shanghai.aliyuncs.com.

Note

ECS instances in Shanghai, Beijing, and Shenzhen can use the VPC endpoint in the same region. The classic network does not support internal access to the speech service. Use HTTP for VPC endpoints and set Host to the corresponding internal domain name. This access method does not incur Internet data transfer fees for the ECS instance.

Call sequence

The following diagram shows the interaction flow for the Android and iOS SDKs.

image
  1. Create an SDK instance and set the audio file transcription callback. Initialize the SDK with the Appkey, NLS token, service endpoint, and device ID.

  2. Set recognition parameters. You can configure nls_config through the parameter-setting interface or include it in the parameters used to start a task.

  3. Pass the absolute path of a local audio file and start the task. A return value of 0 indicates that the interface call succeeded. Recognition results arrive through EVENT_FILE_TRANS_RESULT. Errors are reported through EVENT_ASR_ERROR.

  4. Process the recognition result, and release SDK resources when they are no longer needed. Run synchronous calls, such as initialization, task startup, and resource release, off the UI thread.

The interfaces for each platform are listed below.

Android

Use a NativeNui instance and implement INativeFileTransCallback.

Operation

Interface

Initialize

initialize(callback, ticket, logLevel, saveLog)

Set parameters

setParams(params)

Start a task

startFileTranscriber(params, taskId). Use a byte[] of length 32 for taskId to receive the task ID.

Release resources

release()

Results and errors arrive through the following callback:

void onFileTransEventCallback(Constants.NuiEvent event, int resultCode,
        int finish, AsrResult asrResult, String taskId);

AsrResult.asrResult contains the recognition response JSON. resultCode is the SDK error code, and finish indicates whether the task has ended.

iOS

Use a NeoNui instance and set its delegate to an object that implements NeoNuiSdkDelegate.

Operation

Interface

Initialize

nui_initialize:logLevel:saveLog:

Set parameters

nui_set_params:

Start a task

nui_file_trans_start:taskId:. Use a char array of length 32 for taskId to receive the task ID.

Release resources

nui_release

Results and errors arrive through the following callback:

- (void)onFileTransEventCallback:(NuiCallbackEvent)nuiEvent
                     asrResult:(const char *)asr_result
                        taskId:(const char *)task_id
                      ifFinish:(BOOL)finish
                       retCode:(int)code;

asr_result contains the recognition response JSON. code is the SDK error code, and finish indicates whether the task has ended.

Parameters

Initialization parameters

Pass a JSON string during initialization. The following example uses the Shanghai Internet endpoint. Replace the Appkey, NLS token, and device ID with actual values.

{
  "url": "https://nls-gateway-cn-shanghai.aliyuncs.com/stream/v1/FlashRecognizer",
  "app_key": "<APPKEY>",
  "token": "<NLS_TOKEN>",
  "device_id": "<DEVICE_ID>"
}

Parameter

Type

Required

Description

url

String

Yes

The speech service endpoint.

app_key

String

Yes

The project Appkey from the console.

token

String

Yes

A valid, unexpired NLS token. Set it during initialization or update it through the parameter-setting interface.

device_id

String

Yes

A device ID that uniquely identifies the device.

Task parameters

Pass a JSON string when starting a task. file_path is the absolute path of a local file that the app can read. The following example recognizes a 16 kHz WAV file and enables word-level results.

{
  "file_path": "<AUDIO_FILE_PATH>",
  "nls_config": {
    "format": "wav",
    "sample_rate": 16000,
    "enable_word_level_result": true
  }
}

Parameter

Type

Required

Description

file_path

String

Yes

The absolute path of the audio file to recognize.

direct_ip

String

No

An IP address obtained by resolving the service domain name through DNS. You can pass the address to access the service.

nls_config

JsonObject

No

Recognition configuration. Pass it when starting the task, or configure it beforehand through the parameter-setting interface.

nls_config fields

Parameter

Type

Required

Description

format

String

Yes

The audio encoding format. Supported values: wav, mp3, aac, mp4, and opus. Configure it in nls_config.

sample_rate

Integer

No

The recognition model's sample rate. Supported values: 8000 for telephony and 16000 for non-telephony audio. The server upsamples or downsamples the audio to this rate.

enable_inverse_text_normalization

Boolean

No

Whether to enable inverse text normalization (ITN) to convert Chinese numerals to Arabic numerals. Default: false.

max_end_silence

Integer

No

The maximum trailing silence, in milliseconds. Default: 450.

customization_id

String

No

The ID of a custom model created through a POP API. Omitted by default.

vocabulary_id

String

No

The ID of a custom vocabulary. Omitted by default.

enable_word_level_result

Boolean

No

Whether to return word-level information. Default: false.

first_channel_only

Boolean

No

For stereo files, true recognizes only the first channel, and false recognizes both channels. When omitted, 8 kHz models process both channels, and 16 kHz models process only the first channel.

Set parameters and update the token

The parameter-setting interface also takes a JSON string. The following example updates the NLS token:

{
  "token": "<NEW_NLS_TOKEN>"
}

To configure nls_config beforehand, include the object at the top level of the parameter-setting JSON. You can then reuse the configuration when starting a task without passing it again.

Response

Recognition results arrive through EVENT_FILE_TRANS_RESULT. The following example shows a response with word-level results enabled. Some sentences are omitted.

{
  "task_id": "<TASK_ID>",
  "status": 20000000,
  "message": "SUCCESS",
  "flash_result": {
    "duration": 3834,
    "completed": true,
    "sentences": [
      {
        "text": "hello world, ",
        "words": [
          {
            "text": "hello ",
            "begin_time": "280",
            "end_time": "752",
            "punc": ""
          },
          {
            "text": "world",
            "begin_time": "752",
            "end_time": "988",
            "punc": ", "
          }
        ],
        "begin_time": 280,
        "end_time": 988,
        "channel_id": 0
      }
    ]
  }
}

Field

Description

task_id

The unique task ID.

status

The server status code. 20000000 indicates success.

message

The status message.

flash_result

The recognition result object.

flash_result fields

Field

Description

duration

The audio duration, in milliseconds.

completed

Whether recognition has completed.

sentences

An array of sentence-level recognition results.

sentences fields

Field

Description

text

The sentence-level recognition result.

begin_time

The sentence start offset in the audio, in milliseconds.

end_time

The sentence end offset in the audio, in milliseconds.

channel_id

The audio channel number, starting from 0.

words

An array of word-level information. Returned when enable_word_level_result is enabled.

words fields

Field

Description

text

The recognized word.

begin_time

The word start offset in the audio, in milliseconds.

end_time

The word end offset in the audio, in milliseconds.

punc

Punctuation after the word. Empty when there is no punctuation.

Error codes

For service and mobile SDK error codes and troubleshooting, see Error code reference.