DocsICP FilingConsole
Documentation
Home Page AgentLoop Developer Reference API Reference API catalog Evaluation Evaluation tasks CreateEvaluationTask

CreateEvaluationTask

Updated at:
Copy as MD
Product
My Favorites

Creates an evaluation task.

Operation description

Calls the CreateEvaluationTask operation to create an evaluation task in a specified AgentSpace. The server verifies AgentSpace permissions, initializes evaluation result storage, checks the uniqueness of the task name, and asynchronously creates and executes an EvaluationRun based on taskMode and runStrategies.

This operation is applicable to running built-in or custom evaluators on Trace, Dataset, or SLS Log data. It supports two execution strategies: historical backfill and continuous evaluation.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

No authorization for this operation. If you encounter issues with this operation, contact technical support.

Request syntax

POST /api/v1/evaluation-task/{agentSpace} HTTP/1.1

Path Parameters

Parameter

Type

Required

Description

Example
agentSpace

string

No

The AgentSpace name.

prod-agentspace

Request parameters

Parameter

Type

Required

Description

Example
body

object

No

The request body.

{"taskName":"trace_task_completion_eval","taskMode":"batch","dataType":"trace","evaluators":[{"evaluatorRef":"Builtin.agent_task_completion","resultName":"agent_task_completion","resultType":"score","variableMapping":{"input":"trace.input","output":"trace.output","agent_trajectory":"trace.agent_trajectory"}}],"config":{"dataScope":"trace"}}
taskName

string

No

The task name. The name must be unique among non-deleted tasks within the same user and AgentSpace. The name can be up to 256 characters in length.

trace_task_completion_eval
taskMode

string

No

The evaluation task mode. Set this parameter to batch to create a persistent evaluation task.

Valid values:

  • batch :

    batch

batch
dataType

string

No

The data source type of the evaluation object. Set this parameter to trace for trace-based evaluation.

Valid values:

  • trace :

    trace

  • log :

    log

  • atif :

    atif

  • dataset :

    dataset

trace
dataFilter

string

No

The filter conditions for evaluation data. This parameter supports a JSON object or a JSON string. Common fields include query, provided, maxRecords, and samplingRate.

{"query":"serviceName='checkout-service'","maxRecords":10,"samplingRate":100}
evaluators

array

No

The list of evaluator configurations. This parameter cannot be empty. Within the same task, evaluatorRef takes precedence as the unique identifier. Otherwise, name is used.

[{"evaluatorRef":"Builtin.agent_task_completion","resultName":"agent_task_completion","resultType":"score","variableMapping":{"input":"trace.input","output":"trace.output","agent_trajectory":"trace.agent_trajectory"}}]
Evaluator

No

The configuration object of a single evaluator.

channel

string

No

The task source. If this parameter is not specified, the backend uses default.

Valid values:

  • default :

    default

default
config

object

No

The data source and execution configuration. When dataType is set to trace, the backend automatically populates the SLS Project and sets storeName to logstore-tracing. For trace-level evaluation, set dataScope to trace.

{"dataScope":"trace"}

string

No

The value of a single configuration item. The key name is determined by the specific field convention.

trace
tags

object

No

The key-value pairs of task tags. You do not need to specify this parameter by default. Specify this parameter only when you want to associate or filter tasks by business tags.

{"env":"prod","serviceId":"checkout-service","planId":"plan-20260703"}

string

No

The value of a single task tag. The key name is customized based on business dimensions.

prod
runStrategies

RunStrategies

No

The task execution strategy. This parameter supports a JSON object or a JSON string. Set this parameter to backfill for historical data backfill or continuous for continuous evaluation of new data.

description

string

No

The description of the evaluation task.

评估线上 Agent 链路任务完成度
clientToken

string

No

The idempotency token. This query parameter is declared in CloudSpec, but the backend does not currently perform idempotency comparison.

a1b2c3d4-1234-5678-90ab-cdef12345678

Response elements

Element

Type

Description

Example

object

Schema of Response

requestId

string

The request ID.

3FE4CD1E-FF41-56BE-B590-7A021D9C1524
taskId

string

The evaluation task ID.

eval-task-8b36f2e2b1f94f9c91ce7a4b0f6d9c25
status

string

The evaluation task status. After creation, the status is typically Pending. After asynchronous orchestration, the status may change to Running or Scheduling.

Valid values:

  • Failed :

    Failed

  • Running :

    Running

  • Completed :

    Completed

  • Scheduling :

    Scheduling

  • Deleted :

    Deleted

  • Terminated :

    Terminated

  • Pending :

    Pending

Pending

Examples

Success response

JSON format

{
  "requestId": "3FE4CD1E-FF41-56BE-B590-7A021D9C1524",
  "taskId": "eval-task-8b36f2e2b1f94f9c91ce7a4b0f6d9c25",
  "status": "Pending"
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.

Previous：GetEvaluationTaskNext：UpdateEvaluationTask