This topic describes how to use custom prompts. Custom prompts allow you to define your own prompts for a Large Language Model (LLM). This feature guides the LLM to complete various tasks that you define. If the standard AI model capabilities provided by Tongyi Tingwu do not meet your business needs, this feature provides more flexible control over the LLM. For example, you can generate summaries with different levels of granularity, extract key business information, or define business standards for the LLM to evaluate. A prompt provides context to an AI model. This context acts as a hint or guide that helps the model understand your intent and respond correctly.
Request parameters
Parameter Name | Type | Description |
CustomPromptEnabled | boolean | The default value is false. |
CustomPrompt | object | The object for custom prompt control parameters. |
CustomPrompt.Contents | list[] | A list of custom prompt parameters. The maximum length for a single submission is 3. |
CustomPrompt.Contents[i].Name | string | The custom name of the prompt. This name is used to match the output result. |
CustomPrompt.Contents[i].Model | string | The model for the prompt. Valid values are |
CustomPrompt.Contents[i].Prompt | string | The custom content of the prompt. This must include the |
CustomPrompt.Contents[i].TransType | string | The format of the chat: The transcription result includes speaker information. sentence-chat: The transcription result includes sentence numbers and speaker information. default: The transcription result is in plain text format. For more information, see Tag parameters. |
For descriptions of other request parameters, see the document for your scenario:
Token input limits for different models:
Model name | Input token limit |
tingwu-turbo | 28k |
tingwu-plus | 125k |
qwen-max | 28k |
If the transcription result of the input audio exceeds the token limit, the text is truncated. Only the text within the token limit is used as input for the LLM. You can check the CustomPrompt.Truncated parameter in the response to determine whether the text was truncated.
Example settings
{
"Input": {
...
},
"Parameters": {
"CustomPromptEnabled": true,
"CustomPrompt": {
"Contents": [
{
"Name": "split-summary-demo",
"Prompt": "Please summarize the following conversation by speaker:\n {Transcription}",
"Model": "tingwu-turbo",
"TransType": "chat"
},
{
"Name": "inspection-demo",
"Prompt": "Please check the conversation for any inappropriate language. If found, respond with 'Yes'. If not, respond with 'No'. The conversation is as follows:\n {Transcription}",
"Model": "tingwu-turbo",
"TransType": "default"
}
]
}
}
}Tag parameters
The {Transcription} tag in the prompt is replaced with text in a different format based on the value of TransType. For example, consider a scenario where a user submits an audio file with speaker diarization enabled. In the audio, the first speaker says, "What's the weather like in Beijing?" and the second speaker says, "What's the weather like in Shanghai?"
{
"TaskId":"10683ca4ad3f4f06bdf6e9dc*********",
"Transcription":{
"Paragraphs":[
{
"ParagraphId":"16987422100275*******",
"SpeakerId":"1",
"Words":[
{
"Id":10,
"SentenceId":1,
"Start":4970,
"End":5560,
"Text":"What's the weather like in Beijing?"
}
]
},
{
"ParagraphId":"16987422100276*******",
"SpeakerId":"2",
"Words":[
{
"Id":20,
"SentenceId":2,
"Start":4970,
"End":5560,
"Text":"What's the weather like in Shanghai?"
},
]
}
]
}
}In the prompt, the {Transcription} tag is replaced with text in different formats, as shown in the following table.
Tag name | Format type | Replacement result |
{Transcription} | default | What's the weather like in Beijing? What's the weather like in Shanghai? |
chat | Speaker 1:What's the weather like in Beijing?\nSpeaker 2:What's the weather like in Shanghai?\n | |
sentence-chat | [1] Speaker 1:What's the weather like in Beijing?\n[2] Speaker 2:What's the weather like in Shanghai?\n |
Code examples
#!/usr/bin/env python
#coding=utf-8
import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential
def create_common_request(domain, version, protocolType, method, uri):
request = CommonRequest()
request.set_accept_format('json')
request.set_domain(domain)
request.set_version(version)
request.set_protocol_type(protocolType)
request.set_method(method)
request.set_uri_pattern(uri)
request.add_header('Content-Type', 'application/json')
return request
def init_parameters():
root = dict()
root['AppKey'] = 'Enter the AppKey you created in the Tingwu console'
# Basic request parameters
input = dict()
input['SourceLanguage'] = 'cn'
input['TaskKey'] = 'task' + datetime.datetime.now().strftime('%Y%m%d%H%M%S')
input['FileUrl'] = 'Enter the URL of the audio file to test'
root['Input'] = input
# AI-related parameters. Set as needed.
parameters = dict()
# Summary control, including the following: full-text summary, speaker summary, and Q&A summary (Q&A review)
parameters['CustomPromptEnabled'] = True
customPrompt = dict()
customPrompt['Contents'] = [
{
"Name": "split-summary-demo",
"Prompt": "Please summarize the following conversation by speaker:\n {Transcription}",
"Model": "tingwu-turbo",
"TransType": "chat"
},
{
"Name": "inspection-demo",
"Prompt": "Please check the conversation for any inappropriate language. If found, respond with 'Yes'. If not, respond with 'No'. The conversation is as follows:\n {Transcription}",
"Model": "tingwu-turbo",
"TransType": "default"
}
]
parameters['CustomPrompt'] = customPrompt
root['Parameters'] = parameters
return root
body = init_parameters()
print(body)
# TODO: Set your AccessKeyId and AccessKeySecret using environment variables.
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)
request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'offline')
request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))package com.alibaba.tingwu.client.demo.aitest;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;
/**
* @author tingwu2023
*/
public class CustomPromptTest {
@Test
public void testCustomPrompt() throws ClientException {
CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
request.putQueryParameter("type", "offline");
JSONObject root = new JSONObject();
root.put("AppKey", "Enter the AppKey you created in the Tingwu console");
JSONObject input = new JSONObject();
input.fluentPut("FileUrl", "Enter the URL of the audio file to test")
.fluentPut("SourceLanguage", "cn")
.fluentPut("TaskKey", "task" + System.currentTimeMillis());
root.put("Input", input);
JSONObject parameters = new JSONObject();
parameters.put("CustomPromptEnabled", true);
JSONObject customPrompt = new JSONObject();
JSONArray contents = new JSONArray()
.fluentAdd(new JSONObject()
.fluentPut("Name", "split-summary-demo")
.fluentPut("Prompt", "Please summarize the following conversation by speaker:\n {Transcription}")
.fluentPut("Model", "tingwu-turbo")
.fluentPut("TransType", "chat"))
.fluentAdd(new JSONObject()
.fluentPut("Name", "inspection-demo")
.fluentPut("Prompt", "Please check the conversation for any inappropriate language. If found, respond with 'Yes'. If not, respond with 'No'. The conversation is as follows:\n {Transcription}")
.fluentPut("Model", "tingwu-turbo")
.fluentPut("TransType", "default"));
customPrompt.put("Contents", contents);
parameters.put("CustomPrompt", customPrompt);
root.put("Parameters", parameters);
System.out.println(root.toJSONString());
request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);
// TODO: Set your AccessKeyId and AccessKeySecret using environment variables.
DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
IAcsClient client = new DefaultAcsClient(profile);
CommonResponse response = client.getCommonResponse(request);
System.out.println(response.getData());
}
public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
// Create an API request and set its parameters.
CommonRequest request = new CommonRequest();
request.setSysDomain(domain);
request.setSysVersion(version);
request.setSysProtocol(protocolType);
request.setSysMethod(method);
request.setSysUriPattern(uri);
request.setHttpContentType(FormatType.JSON);
return request;
}
}Example output
{
"Code":"0",
"Data":{
"TaskId":"5a7343ad75e64z3da121ce65********",
"TaskStatus":"COMPLETED",
"Result":{
"Transcription":"http://speech-swap-hangzhou.oss-cn-hangzhou.aliyuncs.com/tingwu/output/1503864348104017/5a7343ad75e6493da121ce653f9389eb/5a7343ad75e6493da121ce653f9389eb_Transcription_20231029224625.json?Expires=1698677343&OSSAccessKeyId=LTAI****************&Signature=Kliw%2Fp2xCKHpKR06********Y8%3D",
"CustomPrompt":"http://speech-swap-hangzhou.oss-cn-hangzhou.aliyuncs.com/tingwu/output/1503864348104017/5a7343ad75e6493da121ce653f9389eb/5a7343ad75e6493da121ce653f9389eb_CustomPrompt_20231029224656.json?Expires=1698677343&OSSAccessKeyId=LTAI****************&Signature=rY5v4CbGjKnh0Nu*********Xhs%3D"
}
},
"Message":"success",
"RequestId":"4EEBD53F-BCC9-1A0E-B17E-C459********"
}The value of the CustomPrompt field is the HTTP download link for the summary.
Protocol analysis
The LLM summary result URL from the preceding output returns a JSON message. The following is an example.
{
"TaskId": "c8b8f8cac1134675a8722ae3********",
"CustomPrompt": [
{
"Name": "split-summary-demo",
"Result": "Speaker 1 (Zhijie) from DAMO Academy introduces its research and future plans. Speaker 2, a science video creator from Xigua Video, asks questions to learn about and share the technology.\n\nSpeaker 1 notes that predicting future technology is risky, but they do it to share their vision with the public. He then describes DAMO Academy's global presence and highlights its large headquarters and research institute in Hangzhou. Zhijie also discusses DAMO Academy's report on top ten future technologies, showing their interest in advanced AI, especially speech technology.\n\nZhijie discusses speech recognition challenges in noisy environments or meetings. He mentions current solutions, such as using machine learning to classify issues before routing them to human agents. He also mentions a long-term goal: using AI in meetings to improve efficiency and decision-making.\n\nSpeaker 2, as an observer, demonstrates applications of DAMO Academy's speech technology, such as voice-controlled vending machines and smart TVs. These examples show the potential and challenges of the technology in daily life.\n\nFinally, both speakers stress the goal of making interactive voice response \"ubiquitous\". They discuss how this technology can serve society and how companies must remain user-focused as they grow.\n\nIn summary, the conversation covers DAMO Academy's research and future plans for speech technology and AI. It also shows the current uses and potential impact of these technologies.",
"Truncated": false
},
{
"Name": "inspection-demo",
"Result": "No",
"Truncated": false
}
]
}The fields are described as follows.
Parameter name | Type | Description |
TaskId | string | The TaskId generated when the task was created. |
CustomPrompt | list[] | A list of custom prompt results. |
CustomPrompt.Name | string | Corresponds to CustomPrompt.Contents[i].Name in the request parameters. |
CustomPrompt.Result | string | The result returned by the LLM. |
CustomPrompt.Truncated | boolean | Indicates whether truncation occurred. |