Create a baseline evaluation task
Baseline evaluation provides 13 public standard benchmarks across five major categories: general knowledge, mathematics, code, context reasoning, and domain-specific. It requires no dataset preparation or evaluation dimension configuration on your part and quickly delivers benchmark scores for model basic capabilities.
Create a baseline evaluation task overview
Baseline evaluation provides 13 public standard benchmarks across five major categories: general knowledge, mathematics, code, context reasoning, and domain-specific. It automatically evaluates model basic capabilities—no need to prepare datasets or configure evaluation dimensions yourself—so you can quickly obtain benchmark scores.
Important
Baseline evaluation is available only in the Beijing region. In other regions, this option is not displayed, which is expected behavior.
Benchmark library
The benchmark library is organized into 5 major categories:
- General knowledge: Assesses model performance in general capabilities such as multidisciplinary expertise, multilingual understanding, and instruction following.
- Mathematics: Assesses the model's mathematical reasoning and competition mathematics capabilities.
- Code: Assesses the model's code generation capabilities.
- Context reasoning: Assesses the model's retrieval, understanding, and reasoning capabilities in long texts.
- Domain-specific: Assesses the model's capabilities in professional scenarios such as finance and medicine.
The tags, number of questions, evaluation dimensions, and description of each benchmark are as follows:
Category | Benchmark | Tags | Questions | Evaluation dimensions | Description |
|---|---|---|---|---|---|
General knowledge | MMLU-Pro | Knowledge QA, 10 options | 12,032 | 14 items | Enhanced multiple-choice questions covering 14 disciplines such as STEM, testing professional knowledge and deep reasoning capabilities |
General knowledge | MMMLU | Text reasoning, multilingual | 196,588 | 14 items | The multilingual version of MMLU, with test questions professionally translated into 14 languages to assess multilingual capabilities |
General knowledge | BrowseComp-ZH-Revised | Reasoning, text, Chinese | 289 | — | A high-difficulty benchmark for the Chinese web environment, with 289 multi-hop questions across 11 domains |
General knowledge | IFEval | Instruction following, 25 rule types | 541 | — | Covers 25 types of verifiable instruction rules such as format constraints and length limits, assessing the capability to precisely follow instructions |
Mathematics | GSM8K | Mathematics, text, English | 1,319 | — | Elementary school math word problems that require multi-step reasoning and basic arithmetic, assessing fundamental mathematical reasoning capabilities |
Mathematics | MATH-500 | Mathematical reasoning, 5 difficulty levels | 500 | 5 items | 500 graded competition math problems covering algebra, geometry, and number theory |
Mathematics | AIME-2025 | Competition mathematics, integer answers | 30 | — | 30 high-difficulty competition problems from the American Invitational Mathematics Examination |
Code | HumanEval | Python function generation | 164 | — | 164 Python function generation problems (including signatures, docstrings, and test cases), testing code generation capabilities |
Context reasoning | LongBench-v2 | Bilingual, 6 task types, long text | 503 | 3 items | Covers 6 types of bilingual long-text tasks, including single/multi-document QA, long-context learning, and code repository understanding |
Context reasoning | OpenAI-MRCR | Long context, coreference resolution, bilingual | 2,400 | 3 items | Multi-turn dialogue coreference resolution tasks, testing the capability to precisely retrieve and distinguish similar content in ultra-long contexts |
Domain-specific | FinanceQA | Financial reasoning, numerical calculation, multi-step | 148 | 3 items | Multi-step numerical calculation problems from real financial analysis scenarios such as hedge funds and investment banks, testing financial reasoning capabilities |
Domain-specific | PubMedQA | Biomedical, three-class, literature QA | 1,000 | — | Research QA based on PubMed literature abstracts, with yes/no/maybe three-class answers |
Domain-specific | MedMCQA | Medicine, 21 disciplines, MCQ | 4,183 | — | Multiple-choice questions covering 21 medical disciplines such as anatomy, physiology, and pharmacology |
Creation workflow
- In the Model Studio console, select the Evaluation Tasks tab and click Create Evaluation Task.
- Select Baseline Evaluation as the evaluation method.
- Select the target model to evaluate. You can select Official Models (Qwen-series commercial and open-source models) and My Models (deployed fine-tuned models).
- Select baseline evaluation data. The benchmark library is displayed in 5 category tabs, with cards containing the benchmark name, tags, description, and number of questions, and multiple selection is supported. Some benchmarks support Evaluation Dimension selection, with all selected by default; you can deselect to run only the specified sub-dimension data.
- (Optional) Enable the Data Sampling Percentage switch to sample evaluation data by percentage and shorten the evaluation duration; when disabled, all data is evaluated.
- Click Start Evaluation to submit the task. Costs are calculated based on actual inference and scoring consumption, and you can view the calculation details before submission.
Differences from custom evaluation
- Baseline evaluation does not use custom evaluation dimensions. The system automatically scores with preset standards.
- Baseline evaluation does not support inference result sets as the data source and only uses platform preset public datasets.
- Baseline evaluation tasks display the overall score in the task list, without the dimension details column (a dash is displayed).
- Baseline evaluation tasks cannot enter the manual annotation page.
For creating custom evaluations, see Create a custom evaluation task.
FAQ
Question: I cannot see the baseline evaluation option when creating an evaluation task.
Baseline evaluation is available only in the Beijing region. In other regions, the evaluation method shows only custom evaluation—this is expected behavior.