Python SDK

The parameters and interfaces of the Paraformer real-time speech recognition Python SDK.

User guide: For model introduction and selection recommendations, see Real-time speech recognition - Fun-ASR/Paraformer.

Prerequisites

You have activated the service and Obtain an API key. Please Configure API key as an environment variable instead of hardcoding it in your code to prevent security risks caused by code leakage.

• Install the latest DashScope SDK.

Model list

Getting started

The core resource (client.audio.asr) provides non-streaming and bidirectional streaming call interfaces. You can select the appropriate method based on your requirements:

• Non-streaming call (recognize_file): Recognizes a local file. The SDK automatically sends the audio in chunks and drives the task to completion. This is suitable for processing pre-recorded audio.
• Bidirectional streaming call (recognize session): Recognizes an audio stream and outputs the results in real time. The audio stream can come from an external device, such as a microphone, or be read from a local file. This is suitable for scenarios that require immediate feedback.

Non-streaming call

This method submits a real-time speech-to-text task for a local file. The process is blocked until the complete transcription result is returned.

Call the recognize_file method of the core resource (client.audio.asr) with the request parameters and a local file path, and iterate the server events to obtain the RecognitionResult.

import os
import dashscope

# China (Beijing): Replace {WorkspaceId} with your actual workspace ID. The configuration varies by region.
client = dashscope.Client(
    # If you have not configured the API key in the environment variable, replace the following line with your API key: api_key="sk-xxx"
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    workspace="{WorkspaceId}",
    region="cn-beijing",
)

# Recognize a local audio file: recognize_file automatically sends the audio in chunks and drives the task to completion
for event in client.audio.asr.recognize_file(
        '{YOUR_AUDIO_FILE}',
        model='paraformer-realtime-v2',
        format='wav',
        sample_rate=16000,
        # The "language_hints" parameter is supported only by the paraformer-realtime-v2 model.
        parameters={'language_hints': ['zh', 'en']}):
    payload = event.get('payload') or {}
    sentence = (payload.get('output') or {}).get('sentence') or {}
    if sentence.get('text') and not sentence.get('heartbeat'):
        print('Recognition result: ', sentence['text'])

Bidirectional streaming call

This method submits a real-time speech-to-text task and streams real-time recognition results by iterating events.

1. Start streaming speech recognition

   Call the recognize method of the core resource (client.audio.asr) to create a recognition session with the request parameters. Enter the with context (which automatically calls the start method to establish the connection and submit the task) to start streaming speech recognition.

2. Streaming

   Repeatedly call the session's send_audio_frame method to send the binary audio stream read from a local file or a device (such as a microphone) to the server in segments. Sending and receiving must be performed concurrently: the receiver thread iterates server events through event iteration (events()/results()) to return the recognition results to the client in real time.

   We recommend that the duration of each audio segment sent is about 100 milliseconds, and the data size is between 1 KB and 16 KB.

3. End processing

   Call the session's stop method to stop speech recognition.

   This method only notifies the server that the input is complete; the receiver thread must continue to consume events() until the task-finished event to obtain the final recognition results and statistics. Errors at the WebSocket layer are raised as DashScopeError exceptions, which can be caught with try/except at the sending or receiving points.

import os
import signal  # for keyboard events handling (press "Ctrl+C" to terminate recording)
import sys

import dashscope
import pyaudio
from concurrent.futures import ThreadPoolExecutor

# China (Beijing): Replace {WorkspaceId} with your actual workspace ID. The configuration varies by region.
client = dashscope.Client(
    # If you have not configured the API key in the environment variable, replace the following line with your API key: api_key="sk-xxx"
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    workspace="{WorkspaceId}",
    region="cn-beijing",
)

mic = None
stream = None

# Set recording parameters
sample_rate = 16000  # sampling rate (Hz)
channels = 1  # mono channel
dtype = 'int16'  # data type
format_pcm = 'pcm'  # the format of the audio data
block_size = 3200  # number of frames per buffer

# Real-time speech recognition event receiver
def receive_events(rec):
    for event in rec.events():
        payload = event.get('payload') or {}
        sentence = (payload.get('output') or {}).get('sentence') or {}
        if sentence.get('text') and not sentence.get('heartbeat'):
            print('Recognition text: ', sentence['text'])
            if sentence.get('sentence_end'):
                print(
                    'Recognition sentence end, request_id:%s, usage:%s'
                    % (rec.request_id, rec.usage))

def signal_handler(sig, frame):
    print('Ctrl+C pressed, stop recognition ...')
    # Stop recognition
    recognition.stop()
    print('Recognition stopped.')
    print(
        '[Metric] requestId: {}, first package delay ms: {}, last package delay ms: {}'
        .format(
            recognition.request_id,
            recognition.first_package_delay,
            recognition.last_package_delay,
        ))
    # Forcefully exit the program
    sys.exit(0)

# main function
if __name__ == '__main__':
    print('Initializing ...')

    signal.signal(signal.SIGINT, signal_handler)
    print("Press 'Ctrl+C' to stop recording and recognition...")

    # Call recognition service, you can customize the recognition parameters, like model, format,
    # sample_rate
    with client.audio.asr.recognize(
            model='paraformer-realtime-v2',
            format=format_pcm,
            # 'pcm', 'wav', 'opus', 'speex', 'aac', or 'amr'. You can check the supported formats in the document.
            sample_rate=sample_rate,
            # 8000 or 16000 is supported.
            parameters={'semantic_punctuation_enabled': False},
    ) as recognition:
        # Open the microphone
        mic = pyaudio.PyAudio()
        stream = mic.open(format=pyaudio.paInt16,
                          channels=1,
                          rate=16000,
                          input=True)
        with ThreadPoolExecutor(max_workers=1) as pool:
            receiver = pool.submit(receive_events, recognition)
            try:
                # Send audio from the microphone until "Ctrl+C" is pressed
                while stream:
                    data = stream.read(3200, exception_on_overflow=False)
                    recognition.send_audio_frame(data)
                recognition.stop()
                receiver.result()
            finally:
                # close must run inside the pool context to unblock the receiver
                recognition.close()
        stream.stop_stream()
        stream.close()
        mic.terminate()
        stream = None
        mic = None

import os
import time
import dashscope
from concurrent.futures import ThreadPoolExecutor

# China (Beijing): Replace {WorkspaceId} with your actual workspace ID. The configuration varies by region.
client = dashscope.Client(
    # If you have not configured the API key in the environment variable, replace the following line with your API key: api_key="sk-xxx"
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    workspace="{WorkspaceId}",
    region="cn-beijing",
)

from datetime import datetime

def get_timestamp():
    now = datetime.now()
    formatted_timestamp = now.strftime("[%Y-%m-%d %H:%M:%S.%f]")
    return formatted_timestamp

# Real-time speech recognition event receiver
def receive_events(rec):
    for event in rec.events():
        payload = event.get('payload') or {}
        sentence = (payload.get('output') or {}).get('sentence') or {}
        if sentence.get('text') and not sentence.get('heartbeat'):
            print(get_timestamp() + ' Recognition text: ', sentence['text'])
            if sentence.get('sentence_end'):
                print(get_timestamp() +
                    'Recognition sentence end, request_id:%s, usage:%s'
                    % (rec.request_id, rec.usage))

with client.audio.asr.recognize(
        model='paraformer-realtime-v2',
        format='wav',
        sample_rate=16000,
        # The "language_hints" parameter is supported only by the paraformer-realtime-v2 model.
        parameters={'language_hints': ['zh', 'en']},
) as recognition:
    with ThreadPoolExecutor(max_workers=1) as pool:
        receiver = pool.submit(receive_events, recognition)
        try:
            audio_data: bytes = None
            f = open("{YOUR_AUDIO_FILE}", 'rb')
            if os.path.getsize("{YOUR_AUDIO_FILE}"):
                while True:
                    audio_data = f.read(3200)
                    if not audio_data:
                        break
                    else:
                        recognition.send_audio_frame(audio_data)
                    time.sleep(0.1)
            else:
                raise Exception(
                    'The supplied file was empty (zero bytes long)')
            f.close()
            recognition.stop()
            receiver.result()
        finally:
            # close must run inside the pool context to unblock the receiver
            recognition.close()

print(
    '[Metric] requestId: {}, first package delay ms: {}, last package delay ms: {}'
    .format(
        recognition.request_id,
        recognition.first_package_delay,
        recognition.last_package_delay,
    ))

Concurrent calls

In Python, because of the Global Interpreter Lock (GIL), only one thread can execute Python code at a time (although some performance-oriented libraries may remove this limitation). If you want to better utilize the computing resources of a multi-core computer, we recommend that you use multiprocessing or concurrent.futures.ProcessPoolExecutor. Multi-threading can significantly increase SDK call latency under high concurrency.

Request parameters

Request parameters are set through the recognize/recognize_file methods of the core resource (client.audio.asr): model, format, sample_rate, vocabulary_id, and disfluency_removal_enabled are named parameters, and the remaining parameters (such as language_hints and semantic_punctuation_enabled) are passed in through the parameters dictionary.

Key interfaces

Core resource (client.audio.asr)

Real-time speech recognition is exposed through the client.audio.asr resource: recognize creates a bidirectional streaming recognition session (Recognition), and recognize_file recognizes a local file in one shot.

def recognize(self,
              *,
              model: str,
              format: str = "pcm",
              sample_rate: int = 16000,
              vocabulary_id: str = None,
              disfluency_removal_enabled: bool = None,
              diarization_enabled: bool = None,
              parameters: Dict = None,
              input: Dict = None,
              phrase_id: str = None,
              ...) -> Recognition

def recognize_file(self,
                   path: str,
                   *,
                   model: str,
                   format: str = "pcm",
                   sample_rate: int = 16000,
                   ...) -> Iterator[Dict]

def start(self)

def send_audio_frame(self, buffer: bytes)

def stop(self)

def events(self) -> Iterator[Dict]
def results(self) -> Iterator[RecognitionResult]

def close(self)

rec.request_id
rec.first_package_delay
rec.last_package_delay

Event iteration (events()/results())

During a bidirectional streaming call, the server returns key process information and data to the client through events. You need to iterate the session's events() (raw event dicts) or results() (typed recognition result frames) in a receiver thread to process the information and data returned by the server.

# Iterate raw events: the recognized text is located in payload.output.sentence
def receive_events(rec):
    for event in rec.events():
        payload = event.get('payload') or {}
        sentence = (payload.get('output') or {}).get('sentence') or {}
        if sentence.get('text') and not sentence.get('heartbeat'):
            print('Recognized text: ', sentence['text'])


# Or iterate only recognition result frames (returns RecognitionResult, heartbeat frames filtered out)
def receive_results(rec):
    for result in rec.results():
        sentence = result.get_sentence()
        if sentence and 'text' in sentence:
            print('Recognized text: ', sentence['text'])

event["header"]["event"] == "task-started"

def events(self) -> Iterator[Dict]

def results(self) -> Iterator[RecognitionResult]

try:
    ...
except DashScopeError as e:
    print(e.code, e.message, e.request_id)

event["header"]["event"] == "task-finished"

Response results

Recognition result (RecognitionResult)

RecognitionResult represents a recognition result frame yielded by iterating the session's results(). It can be used in both bidirectional streaming calls and non-streaming calls. events() yields raw event dicts; use results() if you need this typed view.

def get_sentence(self) -> Union[Dict[str, Any], List[Any]]

result.request_id

def is_sentence_end(self) -> bool

Sentence (Sentence)

The members of the Sentence class are as follows:

Word timestamp information (Word)

The members of the Word class are as follows:

Error codes

If you encounter errors, see Error codes for troubleshooting.

If the issue persists, join the developer community to report your issue and provide the Request ID for further investigation.

More examples

For more examples, see GitHub.

FAQ

Features

Q: How to maintain a long connection with the server during prolonged silence?

Set the request parameter heartbeat to true and continuously send silent audio to the server.

Silent audio refers to audio files or data streams that contain no sound signal. Silent audio can be generated through various methods, such as using audio editing software like Audacity or Adobe Audition, or through command-line tools like FFmpeg.

Q: How to convert audio to a supported format?

You can use the FFmpeg tool. For more usage, refer to the FFmpeg official website.

# Basic conversion command (universal template)
# -i: Input file path. Example: audio.wav
# -c:a: Audio codec. Example: aac, libmp3lame, pcm_s16le
# -b:a: Bitrate (quality control). Example: 192k, 320k
# -ar: Sample rate. Example: 44100 (CD), 48000, 16000
# -ac: Number of channels. Example: 1 (mono), 2 (stereo)
# -y: Overwrite existing file (no value needed)
ffmpeg -i input_audio.ext -c:a codec_name -b:a bitrate -ar sample_rate -ac channels output.ext

# Example: WAV -> MP3 (preserve original quality)
ffmpeg -i input.wav -c:a libmp3lame -q:a 0 output.mp3
# Example: MP3 -> WAV (16-bit PCM standard format)
ffmpeg -i input.mp3 -c:a pcm_s16le -ar 44100 -ac 2 output.wav
# Example: M4A -> AAC (extract/convert Apple audio)
ffmpeg -i input.m4a -c:a copy output.aac  # Direct extraction without re-encoding
ffmpeg -i input.m4a -c:a aac -b:a 256k output.aac  # Re-encode for higher quality
# Example: FLAC lossless -> Opus (high compression)
ffmpeg -i input.flac -c:a libopus -b:a 128k -vbr on output.opus

Q: Does it support viewing the time range for each sentence?

Yes. The speech recognition results include the start and end timestamps for each sentence, which can be used to determine the time range of each sentence.

Q: How do I recognize a local file (recorded audio file)?

There are two ways to recognize a local file:

• Directly pass the local file path: This method returns the complete recognition result after the file is fully processed. It is not suitable for scenarios that require immediate feedback.

  Call the recognize_file method of the core resource (client.audio.asr) with the file path to directly recognize the recorded audio file. For more information, see Non-streaming call.

• Convert the local file into a binary stream for recognition: This method returns recognition results as a stream while the file is being processed. It is suitable for scenarios that require immediate feedback.

  You can use the send_audio_frame method of the recognition session to send a binary stream to the server for recognition. For more information, see bidirectional streaming call.

Troubleshooting

Q: What causes the failure to recognize speech (no recognition results)?

1. Check whether the audio format (format) and sample rate (sampleRate/sample_rate) in the request parameters are correctly set and comply with parameter constraints. The following are common error examples:

   • The audio file extension is .wav, but the actual format is MP3, and the request parameter format is set to wav (incorrect parameter setting).
   • The audio sample rate is 3600 Hz, but the request parameter sampleRate/sample_rate is set to 48000 (incorrect parameter setting).

   You can use the ffprobe tool to obtain the container, codec, sample rate, channel, and other information about the audio:

ffprobe -v error -show_entries format=format_name -show_entries stream=codec_name,sample_rate,channels -of default=noprint_wrappers=1 input.xxx

2. When using the paraformer-realtime-v2 model, check whether the language set in language_hints matches the actual language of the audio.

   For example: The audio is actually in Chinese, but language_hints is set to en (English).

3. If all the above checks pass, you can use custom hot words to improve recognition accuracy for specific words.

Q: How do I fix missing recognition results caused by an incorrect audio format parameter?

format is required and must match the actual audio format. For example, for an MP3 audio file with a .wav extension, set format='mp3'.

1. Check the actual audio format: Use ffprobe to inspect the audio container, codec, sample rate, and channels.

   ffprobe -v error -show_entries format=format_name -show_entries stream=codec_name,sample_rate,channels -of default=noprint_wrappers=1 your_audio_file
   

2. Set the request parameters: Set format to the actual audio format. Set sample_rate to the actual sample rate and ensure that it meets the requirements of the model you use.

3. Convert the audio format if needed: If you need WAV audio, use the following command to generate a PCM-encoded, 16 kHz, mono WAV file.

   ffmpeg -i input_file -c:a pcm_s16le -ar 16000 -ac 1 output.wav
   

   After conversion, set format='wav' and sample_rate=16000.