Create a JMeter scenario

Limitations

• Your resource plan limits the maximum number of concurrent users. For more information, see Pricing.

• JMeter performance tests share the concurrent user quota with native PTS tests. For example, if you have a pay-as-you-go basic resource plan that supports a maximum of 50,000 concurrent users, and a native PTS test scenario is already using 10,000 concurrent users, any JMeter test running at the same time can use a maximum of 40,000 concurrent users.

Step 1: Create a JMeter test scenario

1. Upload JMeter test files.

   1. Log on to the PTS console. In the navigation pane on the left, choose performance testing > Create Scenario, and then click JMeter.

   2. On the Create JMeter Scenario page, enter a Scenario Name.

   3. In the Scenario Configuration section, click the icon to upload a JMeter test file in JMX format. The filename must not contain spaces and must have the .jmx extension.

      After you upload a JMX test file, PTS automatically adds any missing plugins. For more information, see JMeter plugin auto-completion.

   4. Click Upload File to add JMeter scripts, JAR files (files located in the .../lib/ext directory of your JMeter installation), and other data files, such as CSV parameter files.

      Important

      Uploading a file with the same name overwrites the existing file. To verify if a file has changed, you can get its MD5 value in the Actions column and compare it with the MD5 value of your local file.

      Limitations:

      • JMX script file: The file size cannot exceed 2 MB. You can upload multiple JMX script files, but you must select one to use for the test.

      • JAR file: The file size cannot exceed 10 MB. Before uploading, make sure you have successfully debugged the JAR file in your local JMeter environment.

      • CSV file: The file size cannot exceed 60 MB. If your file is larger than 60 MB, use an OSS data source. You can upload a maximum of 20 non-JMX files.

        Important

        • When preparing a CSV file, do not simply change the file extension of an .xlsx file. We recommend exporting the file from an application such as Excel or Numbers, or generating it programmatically by using a library like Apache Commons CSV.

        • If your JMX script references a data file, make sure the Filename in the CSV Data Set Config element is set to the filename only. Do not include a path. Otherwise, PTS cannot read the data file. Similarly, if you use the __CSVRead function or specify a file in a JAR package, you must use only the filename.

   5. If you uploaded multiple JMX files, select the one you want to use for this test scenario.

   6. Optional: For a CSV file, select Split File to ensure that data from the file is not duplicated across agents. If you do not select this option, each agent uses the same dataset. For more information, see Use a CSV parameter file in a JMeter test.

   7. Choose whether to Use Dependent Environment. For more information, see Use an environment.

      • If you select Yes, you must select an environment that you have already created in Environment Management.

      • If you select No, you must select a JMeter version. Apache JMeter 5.0 and Java 11 are currently supported.

2. Set the load configuration.

   For instructions, see Load configuration.

3. Optional: Configure advanced settings.

   These settings are disabled by default. To configure them, turn on the switch.

   • If advanced settings are disabled:

     The default settings are used. The log sampling rate is 1%, the DNS cache is not cleared after each iteration, the system DNS resolver is used, and both the synchronizing timer and constant throughput timer have a per-agent scope.

   • If advanced settings are enabled:

     • Log Sampling Rate

       You can customize the sampling rate for logs. The default rate is 1%. To decrease the sampling rate, enter a value in the range (0, 1]. To increase the rate, enter a multiple of 10 in the range (1, 50], such as 20.

       Important

       If you specify a sampling rate greater than 1%, you will be charged an additional fee of rate%. For example, a sampling rate of 20% incurs an additional charge of 20% of the VUM cost. For more information, see Pay-as-you-go.

     • DNS Cache Settings

       You can choose whether to clear the DNS cache for each request. If you choose to clear the cache, you can use either the system DNS resolver or a custom DNS resolver.

     • Distributed Component Settings

       If your script contains distributed components, such as timers or controllers, you can configure their distributed settings for multiple agent IP addresses. This configuration allows for more accurate and effective performance tests.

       1. Configure the synchronizing timer. If your JMeter script includes a timer, you must specify whether the value in the script has a global scope or per-agent scope.

          • Global scope: The value in your script is treated as the total threshold for the entire cluster. PTS divides this value by the number of agents to determine the threshold for each agent. For example, if you use two agents and set the threshold to 1,000, the threshold for each agent is 500.

          • Per-agent scope: The value in your script is applied to each agent individually. The script is not modified. For example, if you use two agents and set the threshold to 1,000, the threshold for each agent is 1,000, and the total threshold for the cluster is 2,000.

       2. Configure the constant throughput timer. If your JMeter script includes a timer, you must specify whether the value in the script has a global scope or per-agent scope.

          • Global scope: The value in your script is treated as the total throughput target for the entire cluster. PTS divides this value by the number of agents to determine the target for each agent. For example, assume you use two agents, 100 concurrent users, and the script has one thread group with a target throughput of 100 per minute, calculated based on the number of threads. The total target throughput for the scenario would be 2 × (100 concurrent users / 2) × (100 / 2) = 5,000 per minute.

          • Per-agent scope: The value in your script is applied as the target for each agent individually. The script is not modified. For example, assume you use two agents, 100 concurrent users, and the script has one thread group with a target throughput of 100 per minute, calculated based on the number of threads. The total target throughput for the scenario would be 2 × (100 concurrent users / 2) × 100 = 10,000 per minute.

4. Optional: Add cloud resource monitoring.

   Adding monitoring lets you quickly view metrics during and after the performance test. For more information, see Cloud resource monitoring. This section is disabled by default. To configure monitoring, turn on the switch.

5. Optional: Add other information.

   You can assign an owner and add remarks. This section is disabled by default. To do so, turn on the switch.

Step 2: Start the test

1. After you complete the basic configurations, click Debug Scenario to run a preliminary check on network connectivity, plug-in integrity, and script configuration correctness to prevent performance test startup failures. For more information, see Debug Scenario.

2. Click Save and Start Test. In the dialog box that appears, select an Execution Cycle.

   • Select Execute Immediately. The test starts after the preparation is complete.

   • Select Scheduled Task to start the test at a specified time. For more information, see Scheduled test.

3. Optional: During the test, you can monitor test data and adjust the scenario speed.

Data information

Configuration information

This page lists the basic configuration of the scenario, including the pressure source, configured duration, traffic model, and specified number of IP addresses.

Test information

• On the Test Details tab, click View Chart next to a sampler to view its real-time data, such as TPS, success rate, response time, and traffic.

• Click the Agent Performance tab to view time-series charts for all agents, including CPU utilization, Load5, memory utilization, and network traffic. You can also filter to view the performance of a specific agent.

• If you have enabled monitoring for services such as ECS, SLB, RDS, and ARMS, click the Cloud Resource Monitoring tab to view the relevant monitoring information. For more information, see View monitoring details.

Sampled log details

Select the Sampled Logs tab to filter logs by sampler, response status, or RT range, and view log details.

1. Filter logs by sampler, response status, or other criteria. In the Actions column of a log entry, click View Details.

2. In the Log Details dialog box, view the log fields and values on the General tab. In the upper-right corner of the dialog box, you can switch the display format between General and HTTP protocol style.

3. If you configure Embedded Resources from HTML Files (Embedded Resources from HTML Files) in your JMeter script, the Sub-request Details tab is displayed in the log details.

   By selecting a specific sub-request, you can filter for its corresponding request log.

   • The Timing Waterfall tab shows the time taken for the main request and each sub-request.

   • The Trace View tab displays the call trace details for the tested API and its upstream and downstream services.

Step 3: View the test report

After the test stops, the system automatically collects data and generates a performance test report. The report shows performance data over time for the JMeter test scenario and specific samplers.

1. PTS consolePerformance testingReport list

2. On the Reports page, select the JMeter scenario type. In the Actions column of the target report, click View Report. For more information, see View a JMeter performance test report.

   Note

   In the trend chart, the statistical sampling period for each data point is 15s. This may cause data delays at the beginning of the report. Performance test reports are retained for 30 days. If needed, you can export reports and save them locally.

(Optional) Step 4: Export the test report

You can export the performance test report to your local machine.

1. On the Report Details page, click Export Report.

2. Select Version with watermark or Version without watermark to download the performance test report as a PDF file.

Native JMeter use cases

Native JMeter performance testing in PTS is suitable for the following scenarios. For a specific example, see Simulate file uploads with JMeter.

• Running high-concurrency, distributed performance tests.

• Requiring real-time, per-second monitoring during a test and an automatically generated report after the test.

• Centrally managing JMeter scripts and their environmental dependencies.