Binary classification evaluation-Platform For AI(PAI)-阿里云帮助中心

The Binary Classification Evaluation component measures the performance of binary classification models. It calculates AUC (area under the ROC curve), KS (Kolmogorov-Smirnov), and F1 score, and generates a set of charts for visual analysis. Use this component after training a binary classifier in PAI Designer to understand how well the model separates positive and negative classes.

Outputs

Metrics

Metric	What it measures
AUC (area under the ROC curve)	Overall separability between positive and negative classes across all decision thresholds. A higher AUC means the model ranks positive samples above negative ones more reliably. Values range from 0 to 1; 0.5 indicates no discrimination.
KS (Kolmogorov-Smirnov)	Maximum separation between the cumulative distributions of positive and negative samples. A higher KS value indicates stronger class separation. Commonly used in credit scoring and risk modeling.
F1 score	Harmonic mean of precision and recall. Use F1 when you need to balance the trade-off between false positives and false negatives, particularly on imbalanced datasets.

Charts

Chart	What it shows
KS curve	The cumulative distribution gap between positive and negative samples at each decision threshold.
Precision-recall (PR) curve	The trade-off between precision and recall across thresholds. Useful when positive samples are rare.
Receiver operating characteristic (ROC) curve	The true positive rate versus the false positive rate across thresholds. The area under this curve equals AUC.
Lift chart	How much better the model performs compared to random selection, given a specific fraction of the population. For example, a lift of 3 at the top 10% means the model finds three times as many positives as random sampling in that group.
Gain chart	The cumulative percentage of positive samples captured as you increase the fraction of the population targeted. Useful for ranking-based selection tasks such as marketing campaigns.

Configure the component

Method 1: Use the GUI

On the Designer workflow page, add the Binary Classification Evaluation component and configure its parameters in the right pane.

Parameter	Description
Label column name	The name of the target column.
Score column name	The prediction score column. This is usually the `prediction_score` column.
Positive sample label value	The label value that identifies positive samples.
Number of bins for equal frequency binning	The number of buckets for equal-frequency partitioning.
Group column name	The group ID column. Evaluation metrics are calculated separately for each group. Use this for evaluation-by-group scenarios.
Advanced options	Select this check box to show the Prediction detail column, Are prediction and evaluation targets consistent, and Save performance metrics parameters.
Prediction detail column	The name of the column that contains prediction details.
Are prediction and evaluation targets consistent	Whether the model predicts the same class that the evaluation metric targets. See Prediction and evaluation target consistency.
Save performance metrics	Whether to save the performance metrics to an output table.

Prediction and evaluation target consistency

This setting controls how rank-based metrics such as Lift are calculated when the class the model predicts differs from the class the metric evaluates.

Scenario	Model predicts	Metric evaluates	Consistent?
Finance (risk scoring)	Probability that a customer is "bad" — a higher score means more likely bad	Bad-customer detection rate	Yes
Credit scoring	Probability that a customer is "good" — a higher score means more likely good	Bad-customer detection rate	No

Method 2: Use PAI commands

Call the component via the SQL Script component using the following PAI command syntax. For more information about calling PAI commands, see SQL Script.

PAI -name=evaluate -project=algo_public
    -DoutputMetricTableName=output_metric_table
    -DoutputDetailTableName=output_detail_table
    -DinputTableName=input_data_table
    -DlabelColName=label
    -DscoreColName=score

Parameter	Required	Default	Description
`inputTableName`	Yes	—	The name of the input table.
`inputTablePartitions`	No	Full table	The partitions in the input table.
`labelColName`	Yes	—	The name of the target column.
`scoreColName`	Yes	—	The name of the score column.
`groupColName`	No	—	The name of the group column. Use this for evaluation-by-group scenarios.
`binCount`	No	1000	The number of equal-frequency bins for calculating metrics such as KS and PR.
`outputMetricTableName`	Yes	—	The output table for metrics, which includes AUC, KS, and F1 score.
`outputDetailTableName`	No	—	The detailed data table used to plot charts.
`positiveLabel`	No	1	The label value for positive samples.
`lifecycle`	No	—	The lifecycle of the output table.
`coreNum`	No	Automatically calculated	The number of cores.
`memSizePerCore`	No	Automatically calculated	The memory size per core.