API reference-Platform For AI(PAI)-阿里云帮助中心

The LLM-as-a-Judge model supports algorithm service calls through the Python software development kit (SDK) for OpenAI and HTTP. This topic describes how to call the API operations for the LLM-as-a-Judge model, explains the request parameters, and provides call examples.

Chat completions

Call examples

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    completion = client.chat.completions.create(
        model='pai-judge',
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "mode": "single",
                        "type": "json",
                        "json": {
                            "question": "Provide the second line of the couplet for 'Changde Deshan Mountain has virtue'",
                            "answer": "Changsha's sandy water has no sand"
                        }
                    }
                ]
            }
        ]
    )
    print(completion.model_dump())

if __name__ == '__main__':
    main()

curl

$ curl -X POST https://aiservice.cn-hangzhou.aliyuncs.com/v1/chat/completions \
  -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "pai-judge",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "mode": "single",
                    "type": "json",
                    "json": {
                        "question": "Provide the second line of the couplet for ''Changde Deshan Mountain has virtue''",
                        "answer": "Changsha''s sandy water has no sand"
                    }
                }
            ]
        }
    ]
}'

Response

{
  "id": "73d74ed7-1a94-4eac-b67b-14b9a5e20b35",
  "object": "chat.completion",
  "created": 1633389,
  "model": "pai-judge",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I think the overall score for this response is [[3]]. The reasons are as follows.\nStrengths of the current response:\n1. Accuracy: The response correctly provides the second line 'Changsha's sandy water has no sand' for the first line 'Changde Deshan Mountain has virtue'. This shows a certain degree of accuracy in handling couplets. [[4]]\n2. Relevance: The response directly addresses the user's question with a concise and clear answer, without deviating from the topic. [[4]]\n\nWeaknesses of the current response:\n1. Completeness: Although the response provides a seemingly correct second line, it does not offer any background information or explanation on how this line was derived. For users unfamiliar with couplets, the lack of explanation may hinder understanding. [[3]]\n2. Clarity and structure: The response structure is relatively simple. However, due to the lack of explanation, there is limited room for improvement in its educational value and user comprehension. [[3]]\n3. User level adaptation: The response does not consider that the user may not be familiar with the creation or reasoning process of couplets and fails to provide sufficient background information to aid understanding. [[3]]\n\nIn summary, while the response performs well in accuracy and relevance, it has shortcomings in completeness, clarity, structure, and adaptation to the user's level. Therefore, the overall score is 3.\n",
        "refusal": "",
        "function_call": null,
        "tool_calls": null},
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 790,
    "completion_tokens": 257,
    "total_tokens": 1047
  },
  "system_fingerprint": "",
  "service_tier": ""
}

Request parameters

The API operations for the LLM-as-a-Judge model are compatible with OpenAI's API operations and support the following parameters. For more information about request parameter settings, see the official OpenAI documentation.

Note

If you cannot access the link, set up a proxy and try again.

Parameter	Type	Required	Default	Description
model	string	Yes	None	The model name. Valid values: LLM-as-a-Judge Standard Edition (pai-judge) LLM-as-a-Judge Premium Edition (pai-judge-plus) For more information about the models, see Model list.
messages	array	Yes	None	The content to be evaluated.
temperature	float	No	0.2	Controls the randomness and diversity of the model's response. The value must be in the range of [0, 2).
top_p	float	No	None	The probability threshold for nucleus sampling during generation.
stream	boolean	No	False	Specifies whether to use streaming output.
stream_options	object	No	None	This parameter specifies whether to display the number of tokens used during streaming output. This parameter takes effect only when stream is set to True. To count the number of tokens in streaming output mode, set this parameter to `stream_options={"include_usage":True}`.
max_tokens	integer	No	2048	The maximum number of tokens that the model can generate.
frequency_penalty	integer or null	No	0	A number between -2.0 and 2.0. A positive value penalizes new tokens based on their frequency in the text so far, which reduces the likelihood of the model repeating the same line verbatim.
presence_penalty	float	No	0	Controls the degree of repetition in the entire sequence during model generation. A higher value for presence_penalty reduces the repetition. The value must be in the range of [-2.0, 2.0].
seed	integer	No	None	The random number seed used during generation to control the randomness of the generated content. The seed supports unsigned 64-bit integers.
stop	string or array	No	None	Provides precise control over the content generation process. The process stops automatically when the generated content is about to include a specified string or token ID.
tools	array	No	None	Specifies a library of tools that the model can call. The model selects one tool from this library for each function call.
tool_choice	string or object	No	None	Controls which tool the model calls.
parallel_tool_calls	object	No	True	Specifies whether to enable parallel function calls when using tools.
user	string	No	None	A user identifier.
logit_bias	map	No	None	Modifies the likelihood of specified tokens appearing in the completion.
logprobs	boolean	No	False	Specifies whether to return the log probabilities of output tokens. Valid values: False: No value is returned. True: Returns the log probabilities of each output token in the message content.
top_logprobs	integer	No	None	An integer from 0 to 20 that specifies the number of most likely tokens to return at each token position. Each token has an associated log probability. If you use this parameter, you must set logprobs to True.
n	integer	No	1	The number of chat completion choices to generate for each input message.
response_format	object	No	{"type": "text"}	An object that specifies the format that the model must output. Valid values: `{"type": "text"}` (default): The model returns an evaluation response in natural language. `{"type": "json_object"}`: The model returns an evaluation response in JSON format. For example: `{ "Overall Score": "4", "Accuracy": { "Score": "5", "Reason": "The response accurately provides the second line 'Changsha's sandy water has no sand' for the first line 'Changde Deshan Mountain has virtue', adhering to the rules of couplets and demonstrating high accuracy." }, "Relevance": { "Score": "3", "Reason": "The response directly answers the user's question without including any unnecessary information, ensuring high relevance." } }`
service_tier	string	No	None	Specifies the latency tier to use for processing the request.

Description of the messages field

The messages parameter contains the input that the LLM-as-a-Judge model receives. The following is an example:

messages=[
    {
        "role": "user",
        "content": [
            {
                "mode": "single",
                "type": "json",
                "json": {
                    "question": "Provide the second line of the couplet for 'Changde Deshan Mountain has virtue'",
                    "answer": "Changsha's sandy water has no sand"
                }
            }
        ]
    }
]

The messages parameter is an array where each element is an object with the format {"role": role, "content": content}. In this object, role is user and content is the input for the model evaluation, which includes the following:

mode: The model evaluation method. Valid values are single for single-model evaluation and pairwise for pairwise evaluation.
type: The type of content. You must set this parameter to json.
json: The detailed content to be evaluated.

The following table describes the parameters of json.

Parameter name	Type	Required	Description	Default
question	string	Yes	The question asked by the user.	None
answer	string	Required for single-model evaluation	The response from the user's model.	None
answer1	string	Required for pairwise evaluation	The response from user model 1.	None
answer2	string	Required for pairwise evaluation	The response from user model 2.	None
ref_answer	string	No	The reference answer.	None
scene	string	No	The scenario name.	Automatically generated by the LLM-as-a-Judge model. Example: Answering open-ended questions.
scene_desc	string	No	The scenario description.	Automatically generated by the LLM-as-a-Judge model. Example: An open communication instruction, which is usually an open-domain question that expects an open-ended response, such as for a casual chat, advice consultation, or recommendation request.
metric	string	No	The scenario dimension.	Automatically generated by the LLM-as-a-Judge model. For example: Accuracy: Ensures the accuracy of the provided information, follows common sense and facts, and avoids misleading users. Relevance: The response must address the user's question, avoid irrelevant content, and ensure information relevance. Cultural sensitivity and harmlessness: Understands and respects the user's cultural background and differences, adheres to ethics, avoids cultural bias and insensitive expressions, and avoids any potentially offensive content. Information richness: Provides detailed information while ensuring accuracy, especially background information that the user may not have explicitly requested but is helpful for understanding the question. Clarity: Uses clear and easy-to-understand language to answer questions, avoiding jargon or complex structures that may cause misunderstanding. User engagement: Encourages further communication, shows attention to and consideration of the user's question, and promotes interaction by asking questions or providing feedback. Empathy: Considers the user's emotional state when responding and appropriately expresses empathy and understanding, especially when answering questions with emotional undertones. Constructive feedback: Maintains a positive and constructive attitude even when faced with critical or negative questions, providing valuable responses and suggestions.
max_score	integer	No	The scoring range. We recommend a value from 2 to 10.	5
score_desc	string	No	A detailed description of each score level. We recommend that you define the response quality descriptions for scores from 1 to max_score.	1: The response contains major bugs, completely deviates from the standard, and is unusable in practice. 2: Part of the response meets the standard and can be accepted, but as a whole, the response quality is not satisfactory. 3: The response has both strengths and weaknesses. The overall strengths outweigh the weaknesses within the required evaluation criteria. 4: The response quality is good and generally meets the standard, with a few minor issues that can be improved. When a reference answer is given, this score represents the quality of the reference answer. 5: The response is perfect and strictly meets the standard in all aspects. When a reference answer is given, this score represents a response quality that is superior to the reference answer.
steps	string	No	The evaluation steps.	Recall the relevant standards for AI assistant responses, and carefully read and understand the response to be evaluated. Identify the key standards for the current user instruction and response from all standards, including both good and insufficient aspects. In addition to the given standards, add any other important standards you think are necessary for evaluating the current user instruction response. Based on the final standards you have selected, score the response sequentially (from 1 to 5). After weighting and summing up all sub-item scores, provide an overall score for the response. Think carefully, and then give your conclusion. Your response template is as follows. Note that the output must retain the '[[' and ']]' in the template:.

The parameters of the content field are used to populate prompt templates. When you call the LLM-as-a-Judge model using the single-model evaluation and pairwise evaluation examples, the system constructs a request using the following templates. The values in the content field are automatically inserted into the corresponding placeholders in the templates.

Request template for single-model evaluation

Your task is to rate the quality of an AI assistant's response.

You are very clear that when a user gives an instruction for a [${scene}] scenario (the definition of this scenario is: ${scene_desc}), an AI assistant's response should meet the following standards (listed in descending order of importance):
[Start of Standards]
${metric}
[End of Standards]

The scoring is on a ${max_score}-point scale (1-${max_score}), and the meaning of each score level is as follows:
[Start of Score Level Meanings]
${score_desc}
[End of Score Level Meanings]

For the user instruction, we have collected the following response from an AI assistant.
Please evaluate this response and provide your assessment based on the known standards for an assistant's response in the current scenario. Here is the user instruction and assistant response data:
[Start of Data] 
***
[User Instruction]: ${question}
***
[Response]: ${answer}
***
[Reference Answer]: ${ref_answer}
***
[End of Data]

You need to evaluate the above response according to the following process:
${steps}

Think for a moment, and then give your conclusion.

Request template for pairwise evaluation

Your task is to rate the quality of an AI assistant's response.

You are very clear that when a user gives an instruction for a [${scene}] scenario (the definition of this scenario is: ${scene_desc}), an AI assistant's response should meet the following standards (listed in descending order of importance):
[Start of Standards]
${metric}
[End of Standards]

The scoring is on a ${max_score}-point scale (1-${max_score}), and the meaning of each score level is as follows:
[Start of Score Level Meanings]
${score_desc}
[End of Score Level Meanings]

For a user instruction in a [${scene}] scenario, we have collected responses from two AI assistants.
Please evaluate and judge which response is better or if they are tied (including both being good or both being bad) based on the known standards for an assistant's response in the current scenario. Here is the user instruction and assistant response data:
[Start of Data]
***
[User Instruction]: ${question}
***
[Response 1]: ${answer1}
***
[Response 2]: ${answer2}
***
[Reference Answer]: ${ref_answer}
***
[End of Data]

You need to evaluate and compare the two responses according to the following process:
{steps}

Think for a moment, and then give your conclusion.

Note

If you leave the ${scene} parameter empty, the LLM-as-a-Judge model automatically classifies the scenario based on the value of ${question}. The model also generates the corresponding scenario description (${scene_desc}) and scenario dimensions (${metric}).

Response parameters

Parameter	Type	Description
id	string	The ID generated by the system to identify the call.
model	string	The name of the model used for the call.
system_fingerprint	string	The configuration version used by the model at runtime. This is not currently supported and an empty string "" is returned.
choices	array	The details of the content generated by the model.
choices[i].finish_reason	string	It can be one of the following three values: null: Generation is in progress. stop: The process ended because a stop condition in the request parameters was triggered. length: The process ended because the generated content was too long.
choices[i].message	object	The message output by the model.
choices[i].message.role	string	The role of the model. The value is fixed to assistant.
choices[i].message.content	string	The text generated by the model.
choices[i].index	integer	The sequence number of the generated result. Default value: 0.
created	integer	The timestamp of the current generated result, in seconds.
usage	string or array	The metering information, which indicates the token data consumed by this request.
usage.prompt_tokens	integer	The length of the user input text after it is converted into tokens.
usage.completion_tokens	integer	The length of the model-generated response after it is converted into tokens.
usage.total_tokens	integer	The sum of usage.prompt_tokens and usage.completion_tokens.

Status codes

Status code	Code	Error message	Description
200	OK	None	The request was successful.
400	MessagesError	"messages" not in body or type of "messages" is not list.	The messages field cannot be empty. The cause may be a format error. The messages parameter must be a list.
400	ContentError	Content should be like: {"content": [{"type": "json", "mode": "[single / pairwise]", "json": {"question": "<question>", "answer": "<answer>" ...}}]	The content is incorrect. Fill the content by referring to the example: `{ "content": [ { "type": "json", "mode": "[single / pairwise]", "json": { "question": "<question>", "answer": "<answer>", ... } } ] }`
400	ResponseFormatError	Response_format should be one of [{"type": "text"}, {"type": "json_object"}]	The response_format parameter must be one of the following two values: {"type": "text"} {"type": "json_object"}
400	ModeError	Mode must be in [single, pairwise], mode: {mode}.	The mode parameter must be one of the following two values: single pairwise
400	QuestionError	Question should not be empty	The question parameter cannot be empty.
400	AnswerError	Answer should not be empty when mode=single.	When mode is set to single, the answer parameter cannot be empty.
400	AnswerError	Answer1 or answer2 should not be empty when mode=pairwise, answer1: {answer1}, answer2: {answer2}.	When mode is set to pairwise, the answer1 and answer2 parameters cannot be empty.
400	SceneError	Scene need to be specified a judge-native scece when scene_desc and metric is empty.	When scene_desc and metric are empty, the scene parameter must be an internal scenario: Answering mathematical questions Answering deterministic questions Answering open-ended questions Text rewriting Creative writing Informational and professional writing Practical writing Professional writing Translation Reading comprehension and information extraction Role assumption Code generation, modification, and analysis
400	SceneError	Scene_desc and metric need not be specified when scene is not empty and not a inner scene, scene_desc: {scene_desc}, metric: {metric}.	When the scene parameter is not empty and is not an internal scenario, the scene_desc and metric parameters cannot be empty.
400	SceneError	Scene_desc and metric need not to be specified when scene is empty, scene_desc: {scene_desc}, metric: {metric}.	When the scene parameter is empty, the scene_desc and metric parameters must also be empty.
400	ScoreError	Score_desc need to be specified when max_score is not empty.	When the max_score parameter is not empty, the score_desc parameter cannot be empty.
400	ScoreError	Score_desc need not to be specified when max_score is empty.	When the max_score parameter is empty, the score_desc parameter must also be empty.
401	InvalidToken	Invalid Token provided.	The provided token is invalid.
402	InvalidBody	json load request body error	The request body is not in JSON format.
403	GreenNetFilter	The output content contains high risk. risk_info: xxx	The output content contains high risks.
404	ModelNotFound	Model not found, model must in ['pai-judge', 'pai-judge-plus']	The accessed model does not exist.
500	ModelServiceFailed	Scenario_division, load error, request_id: xxx, errmsg: xxx	The call to the scenario classification model failed.
500	ModelServiceFailed	Request_judge_model, load error, request_id: xxx, errmsg: xxx	The call to the LLM-as-a-Judge model failed.
500	ModelServiceFailed	Request_judge_model_with_stream, load error, request_id: xxx, errmsg: xxx	The streaming call to the LLM-as-a-Judge model failed.

Files

Upload a file: POST /v1/files

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    upload_files = client.files.create(
        file=open("/home/xxx/input.jsonl", "rb"),
        purpose="batch",
    )
    print(upload_files.model_dump_json(indent=4))


if __name__ == '__main__':
    main()

curl

$ curl -XPOST https://aiservice.cn-hangzhou.aliyuncs.com/v1/files \
  -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}" \
  -F purpose="batch"  \
  -F file="@/home/xxx/input.jsonl"

Response

{
    "id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
    "object": "file",
    "bytes": 698,
    "created_at": 1742454203,
    "filename": "input.jsonl",
    "purpose": "batch"
}

Request parameters

Parameter

Type

Required

Description

file

file

Yes

The file object to upload.

purpose

string

Yes

The intended purpose of the uploaded file.

assistants: Assistant and message files.
vision: Assistant image file input.
batch: The Batch API.
fine-tune: Fine-tuning.

Response parameters
For more information, see File object description.

List files: GET /v1/files

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    list_files = client.files.list(
        purpose="batch",
        order="desc",
        limit=10,
        after=""
    )
    print(list_files.model_dump_json(indent=4))

    
if __name__ == '__main__':
    main()

curl

$ curl -XGET https://aiservice.cn-hangzhou.aliyuncs.com/v1/files \
  -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}"

Response

{
    "object": "list",
    "data": [
        {
            "id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
            "object": "file",
            "bytes": 698,
            "created_at": 1742454203,
            "filename": "input.jsonl",
            "purpose": "batch"
        },
        {
            "id": "file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022",
            "object": "file",
            "bytes": 1420,
            "created_at": 1742455638,
            "filename": "file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022_success.jsonl",
            "purpose": "batch_output"
        }
    ]
}

Request parameters

Parameter	Type	Required	Description
purpose	string	No	Only returns files with the specified purpose.
limit	string	No	The intended purpose of the uploaded file. Currently, only batch (for the Batch API) is supported. Default value: 10000.
order	string	No	Sorts the order of files by the created_at timestamp of the object. asc: Ascending order. desc (default): Descending order.
after	string	No	A cursor for use in paging. The after parameter is an object ID that defines your place in the list. For example, if you make a list request and receive 100 objects, ending with obj_foo, your subsequent call can include after=obj_foo to fetch the next page of the list.

Response parameters
Parameter
Type
Description
object
string
Only returns files with the specified purpose.
data
array
File object description.

Retrieve a file: GET /v1/files/{file_id}

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    retrieve_files = client.files.retrieve(
        file_id="file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
    )
    print(retrieve_files.model_dump_json(indent=4))


if __name__ == '__main__':
    main()

curl

$ curl -XGET https://aiservice.cn-hangzhou.aliyuncs.com/v1/files/file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713 \
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}"

Response

{
    "id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
    "object": "file",
    "bytes": 698,
    "created_at": 1742454203,
    "filename": "input.jsonl",
    "purpose": "batch"
}

Request parameters
Parameter
Type
Required
Description
file_id
string
Yes
The ID of the file to retrieve.
Response parameters
For more information, see File object description.

Retrieve or download file content: GET /v1/files/{file_id}/content

Note

You can query only files whose purpose is batch_output.

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    content_files = client.files.content(
        file_id="file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022",
    )
    print(content_files)


if __name__ == '__main__':
    main()

curl

$ curl -XGET https://aiservice.cn-hangzhou.aliyuncs.com/v1/files/file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022/content \
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}" > output.jsonl

Response

{"id":"dcee3584-6f30-9541-a855-873a6d86b7d9","custom_id":"request-1","response":{"status_code":200,"request_id":"dcee3584-6f30-9541-a855-873a6d86b7d9","body":{"created":1737446797,"usage":{"completion_tokens":7,"prompt_tokens":26,"total_tokens":33},"model":"qwen-max","id":"chatcmpl-dcee3584-6f30-9541-a855-873a6d86b7d9","choices":[{"finish_reason":"stop","index":0,"message":{"content":"2+2 equals 4."}}],"object":"chat.completion"}},"error":null}
{"id":"dcee3584-6f30-9541-a855-873a6d86b7d9","custom_id":"request-2","response":{"status_code":200,"request_id":"dcee3584-6f30-9541-a855-873a6d86b7d9","body":{"created":1737446797,"usage":{"completion_tokens":7,"prompt_tokens":26,"total_tokens":33},"model":"qwen-max","id":"chatcmpl-dcee3584-6f30-9541-a855-873a6d86b7d9","choices":[{"finish_reason":"stop","index":0,"message":{"content":"2+2 equals 4."}}],"object":"chat.completion"}},"error":null}

Request parameters
Parameter
Type
Required
Description
file_id
string
Yes
The ID of the file to retrieve.
Response parameters
Description of file objects

Delete a file: DELETE /v1/files/{file_id}

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    delete_files = client.files.delete(
        file_id="file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022",
    )
    print(delete_files)


if __name__ == '__main__':
    main()

curl

$ curl -XDELETE https://aiservice.cn-hangzhou.aliyuncs.com/v1/files/file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022 \
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}"

Response

{
    "id": "batch_66f245a0-88d1-458c-8e1c-a819a5943022",
    "object": "file",
    "deleted": "true"
}

Request parameters
Parameter
Type
Required
Description
file_id
string
Yes
The ID of the file to retrieve.
Response parameters
Parameter
Type
Description
id
string
The ID of the deleted object.
object
string
The type of the deleted object.
deleted
string
Indicates whether the object has been deleted.

File object description

Parameter	Type	Description
id	string	The ID is deleted.
object	string	The object type to delete.
bytes	integer	The size of the file.
created_at	integer	The time when the file object was created.
filename	string	The name of the uploaded file.
purpose	string	The intended purpose of the uploaded file.

Batch

Create a batch job: POST /v1/batches

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    create_batches = client.batches.create(
        endpoint="/v1/chat/completions",
        input_file_id="file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
        completion_window="24h",
    )
    print(create_batches.model_dump_json(indent=4))


if __name__ == '__main__':
    main()

curl

$ curl -XPOST https://aiservice.cn-hangzhou.aliyuncs.com/v1/batches \ 
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}" \
    -d '{
        "input_file_id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
        "endpoint": "/v1/chat/completions",
        "completion_window": "24h"
 }'

Response

{
    "id": "batch_66f245a0-88d1-458c-8e1c-a819a5943022",
    "object": "batch",
    "endpoint": "/v1/chat/completions",
    "errors": null,
    "input_file_id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
    "completion_window": "24h",
    "status": "Creating",
    "output_file_id": null,
    "error_file_id": null,
    "created_at": 1742455213,
    "in_process_at": null,
    "expires_at": null,
    "FinalizingAt": null,
    "completed_at": null,
    "failed_at": null,
    "expired_at": null,
    "cancelling_at": null,
    "cancelled_at": null,
    "request_counts": {
        "total": 3,
        "completed": 0,
        "failed": 0
    },
    "metadata": null
}

Request parameters

Parameter	Type	Required	Description
input_file_id	string	Yes	The ID of the uploaded file. The file must be in the .jsonl format and must be uploaded with the target batch. The file can contain up to 50,000 requests and be up to 20 MB in size.
endpoint	string	Yes	The endpoint for all requests in the batch. Currently, `/v1/chat/completions` is supported.
completion_window	string	Yes	The time frame in which the batch should be processed. Currently, only 24h is supported.
completion_window	object	No	Custom metadata for the batch.

Response parameters
For more information, see Batch job object description.

List batch jobs: GET /v1/files

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "http://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    list_batches = client.batches.list(
        after="batch_66f245a0-88d1-458c-8e1c-a819a5943022",
        limit=10,
    )
    print(list_batches.model_dump_json(indent=4))


if __name__ == '__main__':
    main()

curl

$ curl -XGET https://aiservice.cn-hangzhou.aliyuncs.com/v1/batches \
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}"

Response

{
    "object": "list",
    "data": [
        {
            "id": "batch_66f245a0-88d1-458c-8e1c-a819a5943022",
            "object": "batch",
            "endpoint": "/v1/chat/completions",
            "errors": null,
            "input_file_id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
            "completion_window": "24h",
            "status": "Succeeded",
            "output_file_id": "file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022",
            "error_file_id": null,
            "created_at": 1742455213,
            "in_process_at": 1742455640,
            "expires_at": 1742455640,
            "FinalizingAt": 1742455889,
            "completed_at": 1742455889,
            "failed_at": null,
            "expired_at": null,
            "cancelling_at": null,
            "cancelled_at": null,
            "request_counts": {
                "total": 3,
                "completed": 3,
                "failed": 0
            },
            "metadata": null
        }
    ],
    "first_id": "",
    "last_id": "",
    "has_more": false
}

Request parameters

Parameter	Type	Required	Description
purpose	string	No	Only returns files with the specified purpose.
limit	string	No	The intended purpose of the uploaded file. assistants: Assistant and message files. vision: Assistant image file input. batch: The Batch API. fine-tune: Fine-tuning.
order	string	No	Sorts the order of files by the created_at timestamp of the object. asc: Ascending order. desc (default): Descending order.
after	string	No	A cursor for use in paging. The after parameter is an object ID that defines your place in the list. For example, if you make a list request and receive 100 objects, ending with obj_foo, your subsequent call can include after=obj_foo to fetch the next page of the list.

Response parameters
Parameter
Type
Description
object
string
The object type.
data
array
Batch task object description.

Retrieve a batch job: GET /v1/batches/{batch_id}

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "http://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    retrieve_batches = client.batches.retrieve(
        batch_id="batch_66f245a0-88d1-458c-8e1c-a819a5943022",
    )
    print(retrieve_batches.model_dump_json(indent=4))


if __name__ == '__main__':
    main()

curl

$ curl -XGET https://aiservice.cn-hangzhou.aliyuncs.com/v1/batches/batch_66f245a0-88d1-458c-8e1c-a819a5943022 \
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}"

Response

{
    "id": "batch_66f245a0-88d1-458c-8e1c-a819a5943022",
    "object": "batch",
    "endpoint": "/v1/chat/completions",
    "errors": null,
    "input_file_id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
    "completion_window": "24h",
    "status": "Succeeded",
    "output_file_id": "file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022",
    "error_file_id": null,
    "created_at": 1742455213,
    "in_process_at": 1742455640,
    "expires_at": 1742455640,
    "FinalizingAt": 1742455889,
    "completed_at": 1742455889,
    "failed_at": null,
    "expired_at": null,
    "cancelling_at": null,
    "cancelled_at": null,
    "request_counts": {
        "total": 3,
        "completed": 3,
        "failed": 0
    },
    "metadata": null
}

Request parameters
Parameter
Type
Required
Description
batch_id
string
Yes
The ID of the batch job to retrieve.
Response parameters
Batch task object description

Cancel a batch job: POST /v1/batches/{batch_id}/cancel

This operation cancels an in-progress batch job. The batch job enters the cancelling state for up to 10 minutes and then changes to the cancelled state. At this point, any partial results are available in the output file.

Request examples

Python

import os
from openai import OpenAI


def main():
    base_url = "https://aiservice.cn-hangzhou.aliyuncs.com/v1"
    judge_model_token = os.getenv("JUDGE_MODEL_TOKEN")

    client = OpenAI(
        api_key=f'Authorization: Bearer {judge_model_token}',
        base_url=base_url
    )
    cancel_batches = client.batches.cancel(
        batch_id="batch_66f245a0-88d1-458c-8e1c-a819a5943022",
    )
    print(cancel_batches.model_dump_json(indent=4))


if __name__ == '__main__':
    main()

curl

$ curl -XPOST https://aiservice.cn-hangzhou.aliyuncs.com/v1/batches/batch_66f245a0-88d1-458c-8e1c-a819a5943022/cancel \
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}"

Response

{
    "id": "batch_66f245a0-88d1-458c-8e1c-a819a5943022",
    "object": "batch",
    "endpoint": "/v1/chat/completions",
    "errors": null,
    "input_file_id": "file-batch-EC043540BE1C7BE3F9F2F0A8F47D1713",
    "completion_window": "24h",
    "status": "Stopping",
    "output_file_id": "file-batch_output-66f245a0-88d1-458c-8e1c-a819a5943022",
    "error_file_id": null,
    "created_at": 1742455213,
    "in_process_at": 1742455640,
    "expires_at": 1742455640,
    "FinalizingAt": 1742455889,
    "completed_at": 1742455889,
    "failed_at": null,
    "expired_at": null,
    "cancelling_at": null,
    "cancelled_at": null,
    "request_counts": {
        "total": 3,
        "completed": 3,
        "failed": 0
    },
    "metadata": null
}

Request parameters
Parameter
Type
Required
Description
batch_id
string
Yes
The ID of the batch job to cancel.
Response parameters
Description of batch task objects

Delete a batch job: DELETE /v1/batches/{batch_id}

Request example

$ curl -XDELETE https://aiservice.cn-hangzhou.aliyuncs.com/v1/batches/batch_66f245a0-88d1-458c-8e1c-a819a5943022 \
    -H "Authorization: Bearer ${JUDGE_MODEL_TOKEN}"

Response

{
    "id": "batch_66f245a0-88d1-458c-8e1c-a819a5943022",
    "object": "batch",
    "deleted": "true"
}

Request parameters
Parameter
Type
Required
Description
batch_id
string
Yes
The ID of the batch job to delete.
Response parameters
Parameter
Type
Description
id
string
The ID of the deleted object.
object
string
The type of the deleted object.
deleted
string
Indicates whether the object has been deleted.

Batch job object description

Parameter	Type	Description
id	string	The ID of the item that was deleted.
object	string	The type of the object.
endpoint	string	The data endpoint.
errors	string	The error message.
input_file_id	string	The input file ID.
completion_window	string	The time window.
status	string	The running status.
output_file_id	string	The output file ID.
error_file_id	string	The error file ID.
created_at	integer	The creation time.
in_process_at	integer	The processing start time.
expires_at	integer	The latest expiration time.
finalizing_at	integer	The finalization time.
completed_at	integer	The completion time.
failed_at	integer	The failure time.
expired_at	integer	The actual expiration time.
cancelling_at	integer	Cancel the scheduled start.
cancelled_at	integer	The cancellation completion time.
request_counts	object	Details about the number of requests.
request_counts.total	integer	The total number of requests.
request_counts.completed	integer	The number of successful requests.
request_counts.failed	integer	The number of failed requests.
metadata	object	The metadata.