Mobile SDK guide
Use the NUI SDK for Android or iOS to upload local audio files to the FlashRecognizer service and obtain transcription results. This topic describes endpoints, the call sequence, request parameters, and response fields.
Prerequisites and limitations
Activate the commercial edition of the FlashRecognizer service. The service does not offer a free trial. For activation, billing, and concurrency limits, see the FlashRecognizer API reference.
Obtain a project Appkey and a valid NLS token. See Manage projects and Obtain a token, respectively. Configure the recognition language and model in the project. The selected model must match the language of the audio.
Supported audio formats are WAV, MP3, AAC, MP4, and OPUS. Files must not exceed 100 MB or 2 hours in duration.
Keep AccessKey IDs and AccessKey secrets on the server. Do not embed them in app code or store them on mobile devices. Obtain an NLS token on the server, send it to the app, and update it before it expires. Initialization parameters contain credentials. Do not log the complete parameter string.
Obtain the SDK
Download an SDK package that supports audio file transcription from Choose and download a mobile SDK, and review the compatibility and compliance requirements on that page. The package includes FileTranscriberActivity.java for Android and FileTranscriberViewController.m for iOS.
For integration instructions and code examples, see Android SDK and iOS SDK.
Starting with SDK V2.6.2, cloud-only features do not require resource files to be copied or a workspace to be configured.
Endpoints
Set the service endpoint in the url initialization parameter. Mobile devices access the service over the Internet. VPC endpoints are for runtime environments with VPC network access in the corresponding region.
|
Access type |
Region |
URL |
|
Internet |
Shanghai |
|
|
Beijing |
|
|
|
Shenzhen |
|
|
|
VPC |
Shanghai |
|
|
Beijing |
|
|
|
Shenzhen |
|
Host is the domain name in the URL. For example, the Shanghai Internet endpoint uses nls-gateway-cn-shanghai.aliyuncs.com.
ECS instances in Shanghai, Beijing, and Shenzhen can use the VPC endpoint in the same region. The classic network does not support internal access to the speech service. Use HTTP for VPC endpoints and set Host to the corresponding internal domain name. This access method does not incur Internet data transfer fees for the ECS instance.
Call sequence
The following diagram shows the interaction flow for the Android and iOS SDKs.
Create an SDK instance and set the audio file transcription callback. Initialize the SDK with the Appkey, NLS token, service endpoint, and device ID.
Set recognition parameters. You can configure
nls_configthrough the parameter-setting interface or include it in the parameters used to start a task.Pass the absolute path of a local audio file and start the task. A return value of
0indicates that the interface call succeeded. Recognition results arrive throughEVENT_FILE_TRANS_RESULT. Errors are reported throughEVENT_ASR_ERROR.Process the recognition result, and release SDK resources when they are no longer needed. Run synchronous calls, such as initialization, task startup, and resource release, off the UI thread.
The interfaces for each platform are listed below.
Android
Use a NativeNui instance and implement INativeFileTransCallback.
|
Operation |
Interface |
|
Initialize |
|
|
Set parameters |
|
|
Start a task |
|
|
Release resources |
|
Results and errors arrive through the following callback:
void onFileTransEventCallback(Constants.NuiEvent event, int resultCode,
int finish, AsrResult asrResult, String taskId);
AsrResult.asrResult contains the recognition response JSON. resultCode is the SDK error code, and finish indicates whether the task has ended.
iOS
Use a NeoNui instance and set its delegate to an object that implements NeoNuiSdkDelegate.
|
Operation |
Interface |
|
Initialize |
|
|
Set parameters |
|
|
Start a task |
|
|
Release resources |
|
Results and errors arrive through the following callback:
- (void)onFileTransEventCallback:(NuiCallbackEvent)nuiEvent
asrResult:(const char *)asr_result
taskId:(const char *)task_id
ifFinish:(BOOL)finish
retCode:(int)code;
asr_result contains the recognition response JSON. code is the SDK error code, and finish indicates whether the task has ended.
Parameters
Initialization parameters
Pass a JSON string during initialization. The following example uses the Shanghai Internet endpoint. Replace the Appkey, NLS token, and device ID with actual values.
{
"url": "https://nls-gateway-cn-shanghai.aliyuncs.com/stream/v1/FlashRecognizer",
"app_key": "<APPKEY>",
"token": "<NLS_TOKEN>",
"device_id": "<DEVICE_ID>"
}
|
Parameter |
Type |
Required |
Description |
|
|
String |
Yes |
The speech service endpoint. |
|
|
String |
Yes |
The project Appkey from the console. |
|
|
String |
Yes |
A valid, unexpired NLS token. Set it during initialization or update it through the parameter-setting interface. |
|
|
String |
Yes |
A device ID that uniquely identifies the device. |
Task parameters
Pass a JSON string when starting a task. file_path is the absolute path of a local file that the app can read. The following example recognizes a 16 kHz WAV file and enables word-level results.
{
"file_path": "<AUDIO_FILE_PATH>",
"nls_config": {
"format": "wav",
"sample_rate": 16000,
"enable_word_level_result": true
}
}
|
Parameter |
Type |
Required |
Description |
|
|
String |
Yes |
The absolute path of the audio file to recognize. |
|
|
String |
No |
An IP address obtained by resolving the service domain name through DNS. You can pass the address to access the service. |
|
|
JsonObject |
No |
Recognition configuration. Pass it when starting the task, or configure it beforehand through the parameter-setting interface. |
nls_config fields
|
Parameter |
Type |
Required |
Description |
|
|
String |
Yes |
The audio encoding format. Supported values: |
|
|
Integer |
No |
The recognition model's sample rate. Supported values: |
|
|
Boolean |
No |
Whether to enable inverse text normalization (ITN) to convert Chinese numerals to Arabic numerals. Default: |
|
|
Integer |
No |
The maximum trailing silence, in milliseconds. Default: |
|
|
String |
No |
The ID of a custom model created through a POP API. Omitted by default. |
|
|
String |
No |
The ID of a custom vocabulary. Omitted by default. |
|
|
Boolean |
No |
Whether to return word-level information. Default: |
|
|
Boolean |
No |
For stereo files, |
Set parameters and update the token
The parameter-setting interface also takes a JSON string. The following example updates the NLS token:
{
"token": "<NEW_NLS_TOKEN>"
}
To configure nls_config beforehand, include the object at the top level of the parameter-setting JSON. You can then reuse the configuration when starting a task without passing it again.
Response
Recognition results arrive through EVENT_FILE_TRANS_RESULT. The following example shows a response with word-level results enabled. Some sentences are omitted.
{
"task_id": "<TASK_ID>",
"status": 20000000,
"message": "SUCCESS",
"flash_result": {
"duration": 3834,
"completed": true,
"sentences": [
{
"text": "hello world, ",
"words": [
{
"text": "hello ",
"begin_time": "280",
"end_time": "752",
"punc": ""
},
{
"text": "world",
"begin_time": "752",
"end_time": "988",
"punc": ", "
}
],
"begin_time": 280,
"end_time": 988,
"channel_id": 0
}
]
}
}
|
Field |
Description |
|
|
The unique task ID. |
|
|
The server status code. |
|
|
The status message. |
|
|
The recognition result object. |
flash_result fields
|
Field |
Description |
|
|
The audio duration, in milliseconds. |
|
|
Whether recognition has completed. |
|
|
An array of sentence-level recognition results. |
sentences fields
|
Field |
Description |
|
|
The sentence-level recognition result. |
|
|
The sentence start offset in the audio, in milliseconds. |
|
|
The sentence end offset in the audio, in milliseconds. |
|
|
The audio channel number, starting from |
|
|
An array of word-level information. Returned when |
words fields
|
Field |
Description |
|
|
The recognized word. |
|
|
The word start offset in the audio, in milliseconds. |
|
|
The word end offset in the audio, in milliseconds. |
|
|
Punctuation after the word. Empty when there is no punctuation. |
Error codes
For service and mobile SDK error codes and troubleshooting, see Error code reference.