Attribution labels

Custom attribution labels

Configuration entry: On the Create evaluator or Edit evaluator page, click the Attribution configuration tab at the top.

Procedure:

1. Turn on the Attribution for low-score results switch.

2. In Trigger conditions, configure Attribute when the normalized score is below X. X ranges from 0 to 1, with a default value of 1. Attribution is triggered only when the evaluation succeeds and the normalized score is below this value.

3. In the Attribution label JSON area, configure the label tree. You can edit the JSON directly, or click Upload JSON to import a file. To restore the default labels, click Restore platform defaults.

The root structure of the attribution label JSON is as follows:

{
  "labels": [
    {
      "name": "Tool",
      "description": "Issues with tool definition, capability, return contract, or availability",
      "evidenceAttributes": [
        "gen_ai.system_instructions",
        "gen_ai.input.messages",
        "gen_ai.output.messages"
      ],
      "children": [
        {
          "name": "Tool unavailable",
          "description": "Tool call fails, times out, or returns an unusable result"
        }
      ]
    }
  ]
}

JSON field descriptions:

• labels: Required. The root field, whose value is an array of label nodes. It must contain at least one label node, and the final attribution resolves to a leaf node path.

• name: Required. The label name. Labels at the same level cannot have duplicate names and cannot contain /. Keep it short and clear.

• description: Optional. Describes how the label is determined. Clearly state when a case should be attributed to this label instead of providing only the label name.

• evidenceAttributes: Optional. The Trace Span Attributes fields that this label preferentially references, such as gen_ai.system_instructions and gen_ai.tool.*.

• children: Optional. An array of child labels. When child labels exist, the final attribution continues down to the lowest-level leaf label.

The final label path is formed by concatenating the name values from the root to the leaf, for example:

• Tool/Tool unavailable

• Model/Missing context information

Recommendations for configuration:

• Describe the root cause in the label, not just the final symptom.

• Keep labels mutually exclusive as much as possible, and avoid multiple labels expressing the same problem.

• The clearer the description is, the more consistently the model can make a judgment.

• Do not use too many labels, which can make attribution results unstable.

Effects

After you enable attribution analysis, an evaluation task produces additional attribution information beyond the original evaluation results.

When a low-score case meets the attribution conditions, the evaluation results display the attribution label, key evidence, and optimization suggestions.

If the attribution conditions are met, you see:

• The final attribution label path, for example Prompt/Instruction conflict.

• Key evidence generated based on the attribution label, and the corresponding follow-up optimization suggestions.

After attribution succeeds, you can use it to:

• View the main reason for a single low-score case.

• Analyze the proportion of different problem types in an evaluation task.

• Compare low-score reasons across different evaluators, AgentSpaces, or versions.

• Guide users to prioritize optimizing prompts, skills, RAG, tool configuration, or model selection.

Note the following:

• Attribution is triggered only for low-score cases.

• No attribution label is produced when the score is not below the threshold.

• Currently, only the Agent evaluator supports attribution analysis.

• If the trace evidence is insufficient, only the attribution status may be recorded, without generating labels or suggestions.

• Attribution results do not affect the original evaluation score and serve only as a reference for problem localization and optimization.