Language model customization

When to use language model customization

Pre-trained models handle everyday speech well but often misrecognize domain-specific terms. For example, an application that transcribes medical consultations may frequently confuse drug names, procedures, or clinical terms when using a general-purpose model. Training a custom language model on your domain text teaches the model the vocabulary and context specific to your field, improving recognition accuracy for specialized terms and high-frequency words.

Use language model customization when:

• Your application handles domain-specific speech (medical or industry-specific)

• The pre-trained scenario models don't recognize your product names, brand terms, or specialized terminology

• Recognition accuracy for high-frequency business-critical words needs improvement

How it works

Upload business-related text data to the self-learning platform. Select a base model that matches your scenario, then start training. The platform learns the vocabulary and context from your data. After training completes, publish the model so it applies to your speech recognition requests.

There are two ways to create and apply a self-learning model:

• Console: Train and manage self-learning models in the console. In the Project Feature Configuration section, click Switch Scenario and select your self-learning model. After you publish the model, it is automatically attached to your AppKey—no code changes required.

• POP API: Create a self-learning model using the POP API. Set the model ID in your client code by calling the interface in the SDK. The model takes effect only after the ID is set.

Train a model: end-to-end example

This example shows how to train a language model using a brief Alibaba company introduction. Download the sample training data.

Prepare your training data

Training data is the text you use to teach the model your domain vocabulary and context. The quality and format of your training data directly affect recognition accuracy.

Format requirements:

• Plain text, one sentence per line

• Use punctuation to separate sentences

Data quantity guidance:

• If a word or phrase is business-critical, include sentences containing it multiple times to reinforce recognition

The sample training data looks like this:

In September 1999, the 18 founders led by Jack Ma officially established Alibaba Group in an apartment in Hangzhou. The group's first website was Alibaba, an English-language global wholesale marketplace.
In October 1999, Alibaba Group raised five million USD from several investment institutions.
In October 1999, Alibaba Group raised five million USD from several investment institutions.
In January 2000, Alibaba Group raised twenty million USD from several investment institutions, including SoftBank.
In January 2000, Alibaba Group raised twenty million USD from several investment institutions, including SoftBank.
In September 2000, Alibaba Group held the first West Lake Summit on Swordmanship, gathering business and opinion leaders from the Internet industry to discuss important topics.

Key sentences appear more than once. Repeating sentences that contain business-critical words—such as "financing" and "the Internet"—reinforces the model's recognition of those terms.

Train the model

1. Select a base model that matches your scenario. This example uses the general-purpose model.

2. Format your training data as plain text, one sentence per line, with punctuation between sentences.

3. Submit your text data on the self-learning platform and start training. After training completes, the model recognizes vocabulary from your training data and delivers improved recognition results for your domain.