Set a performance baseline-Performance Testing(PTS)-阿里云帮助中心

A performance baseline is a standard for your stress test reports. Comparing a report against the baseline helps you better analyze test results and optimize your product's performance. This topic describes how to set a performance baseline and compare a stress test report against it.

Prerequisites

Set an existing stress test report as a baseline using one of the following methods:

On the Edit Scene page, click the Stress Test Report tab. In the Actions column for the target report, click Set as Baseline, and then click Continue in the dialog box that appears.
In the left-side navigation pane of the PTS console, choose Performance Testing > Report List. In the Actions column for the target report, click View Report. On the Report Details page, click Baseline Settings in the upper-right corner, and then click Confirm in the dialog box that appears.

Note PTS retains stress test report data for 30 days. After this period, you can no longer set a report as a baseline.

Set a baseline

PTS consolePerformance testingScene list
On the Scene List page, click Edit in the Actions column for the target scene.
On the Edit Scene page, click the Stress Test Report tab, and then click Baseline Settings.

On the Baseline Settings page, click the 编辑图标.png

icon next to each parameter to modify its value, and then click OK.

Baseline metrics include metrics for the entire scene and for individual APIs.

Table 1. Scene-wide metrics
Metric	Description
Average RT (ms)	Response time (RT) is the time it takes for a transaction to complete. This metric shows the average RT of all APIs in the scene, measured in milliseconds (ms).
Average TPS	Transactions per second (TPS) is the number of transactions that the system processes per second. This metric shows the average TPS for the scene during the test period.
Transaction success rate	The transaction success rate for the entire scene. This metric is available only if checkpoints (assertions) are configured.
Number of failed transactions	The total number of failed transactions for the entire scene. This metric is available only if checkpoints (assertions) are configured.
Request success rate	The success rate of requests for the entire scene.
Number of failed requests	The total number of failed requests for the entire scene.
90th Percentile RT (ms)	The value at the 90th percentile of all collected RT data. For more information, see What do the percentile values in a stress test report mean?.
99th Percentile RT (ms)	The value at the 99th percentile of all collected RT data. For more information, see What do the percentile values in a stress test report mean?.

Table 2. API metrics
Metric	Description
Average TPS	Transactions per second (TPS) is the number of transactions that the system processes per second. This metric shows the average TPS for this API during the test period.
Average RT (ms)	Response time (RT) is the time it takes for a transaction to complete. This metric shows the average RT for this API, measured in ms.
Max RT (ms)	The maximum response time (RT) for this API, measured in ms.
Min RT (ms)	The minimum response time (RT) for this API, measured in ms.
Transaction success rate	The transaction success rate for this API. This metric is available only if checkpoints (assertions) are configured.
Number of failed transactions	The total number of failed transactions for this API. This metric is available only if checkpoints (assertions) are configured.
Request success rate	The success rate of requests for this API.
Number of failed requests	The total number of failed requests for this API.
50th Percentile RT (ms)	The value at the 50th percentile of all collected RT data. For more information, see What do the percentile values in a stress test report mean?.
70th Percentile RT (ms)	The value at the 70th percentile of all collected RT data. For more information, see What do the percentile values in a stress test report mean?.
90th Percentile RT (ms)	The value at the 90th percentile of all collected RT data. For more information, see What do the percentile values in a stress test report mean?.
99th Percentile RT (ms)	The value at the 99th percentile of all collected RT data. For more information, see What do the percentile values in a stress test report mean?.
Average connection time	The average time to establish a server connection for this API, measured in ms.
Request receiving time	The average time to receive a response for this API, measured in ms.
Request sending time	The average time to send a request for this API, measured in ms.
Request waiting time	The average time spent waiting for a response for this API, measured in ms.

Baseline comparison

After setting a baseline, you can compare a stress test report with it to better analyze the test results.

PTS consolePerformance testingScene list
On the Scene List page, click Edit in the Actions column for the target scene.
On the Edit Scene page, click the Stress Test Report tab, select the reports to compare, and then click Report Comparison.
The Report Comparison page automatically displays data from the selected stress test reports and the baseline, including a comparison for the entire scene.