DocsICP FilingConsole
官方文档
首页

Performance testing with the AHBench-jenkins graphical tool

更新时间：
复制 MD 格式
产品详情
我的收藏

This topic explains how to use the AHBench-jenkins test suite and provides important notes.

Overview

It features test set management, process control, and results aggregation. The suite's simple configuration allows you to run performance tests with just a few clicks.

Before you begin

  • Provision an Elastic Compute Service (ECS) instance as the stress testing client. For more information, see Preparation.
  • Download AHBench-jenkins, upload the package to the stress testing client, and extract it.

Important notes

  • Stress testing can place a heavy load on the target cluster, which may cause it to become unresponsive. Do not run this test suite in a production environment.
  • The test suite executes tasks within a set sequentially. Do not run tasks from different test sets in parallel.
  • The test suite uses the ahbenchtest-read and ahbenchtest-write tables for testing. These tables may be deleted and recreated during the test. Ensure that these tables can be safely deleted.
  • Ensure that the target cluster has sufficient storage space.
  • ECS instances are virtualized environments. Performance test results for the same instance type may fluctuate by 5% to 10%. This fluctuation is normal.

Runtime environment

Ensure the stress testing client's environment meets the following requirements:

  • Linux operating system
  • JDK 1.8 or later
  • Python 2.7
  • A dedicated CPU with 16 or more cores is recommended for the client.

Start the service

After you start the AHBench-jenkins service, access it in your browser at its URL (for example, HTTP://IP:PORT).

# Use the following command to set the port. The default port is 80.
cd AHBench-jenkins
./start.sh 8080

Select a test set template

The available test set templates include FastTest and FullTest.

  • FastTest: This test set contains 10 million rows of test data. It requires at least 20 GB of storage and takes about 2 hours to run. The actual duration may vary based on cluster specifications.
  • FullTest: This test set contains 2 billion rows of test data. It requires at least 2 TB of storage and takes about 25 hours to run. The actual duration may vary based on cluster specifications.

Configure the test task

After selecting a test template, click Build with Parameters in the left navigation pane to configure the test task.

Configure the cluster endpoint

You can configure the cluster address by using the cluster_addr parameter. The configuration methods and paths are shown in the following table.

Version cluster_addr description Path
ApsaraDB for HBase Standard Edition The ZooKeeper connection string of the cluster. Console > Database Connection > Connection InformationOn the , navigate to > . Choose a VPC or public endpoint based on your network environment.
ApsaraDB for HBase Performance-enhanced Edition The Java API endpoint of the cluster.
ApsaraDB for Lindorm The HBase Java API endpoint for the wide table engine of the cluster. Console page > Database connection > Wide table engineOn the , navigate to > . Choose a VPC or public endpoint based on your network environment.

Service-specific parameters (Optional)

The following table describes the service-specific parameters for each version.
Version Description
ApsaraDB for HBase Standard Edition cluster_version: The version of the cluster.

Set this to 1 for version 1.x or 2 for version 2.x. The default value is 2.
ApsaraDB for HBase Performance-enhanced Edition
  • username: The username for accessing the database. Ensure the account has the required permissions.
  • password: The password for the specified username.
ApsaraDB for Lindorm
  • username: The username for accessing the database. Ensure the account has the required permissions.
  • password: The password for the specified username.

General test parameters (Optional)

The following parameters control the test configuration, such as compression, encoding, thread count, data volume, field size, and test cases. You can use the default values or customize them as needed.

Parameter Description
test_name The name of the test, used for display purposes only.
table_name The name of the test table used for read and write operations.
row_len The length of the row key.
col_num The number of columns per row.
col_len The length of each column value.
re_create_table Specifies whether to recreate the test table.
compression The compression algorithm for the table. Valid values: NONE, LZO, SNAPPY, GZ, LZ4, and ZSTD.
Note ZSTD is supported only by ApsaraDB for HBase Performance-enhanced Edition and ApsaraDB for Lindorm.
encoding The encoding algorithm for the table. Valid values: NONE, DIFF, and INDEX.
Note INDEX is supported only by ApsaraDB for HBase Performance-enhanced Edition and ApsaraDB for Lindorm.
regions The number of predefined partitions.
pre_load Specifies whether to preload data. This is required for read tests.
load_params Parameters for the data loading phase, including the number of concurrent threads, target operations per second (throttling), and the total number of rows to load.
load_wait The wait time in seconds after the data loading phase is complete.
warm_up Specifies whether to perform a warm-up read phase before the test.
warm_up_params Parameters for the warm-up phase, including the number of concurrent threads, duration in seconds, and the data range to read.
threads The number of concurrent threads for the test.
exec_time The execution duration for each test case.
recordcount The range of keys to use for read tests.
batch_rows The number of rows per batch for batch read and write operations.
scan_rows The number of rows to retrieve in each scan operation.
interval_time The interval in seconds between test cases.
case1-case10 The sequence of test cases to run. Select the test cases based on your requirements.
  • none: Do not run this test case.
  • singleread: Random single-row reads (uniform distribution).
  • uniformsinglewrite: Single-row write.
  • uniformscan: Range scans (uniform distribution).
  • batchwrite: Batch writes (uniform distribution).
  • workload-a: 50/50 read/write mix (Zipfian distribution).
  • workload-b: 95/5 read/write mix (Zipfian distribution).
  • workload-c: Read-only (Zipfian distribution).
  • workload-d: 95/5 read/insert mix (latest distribution). Reads the most recently inserted records.
  • workload-e: 95/5 short-range-scan/insert mix (Zipfian distribution).
  • workload-f: 50/50 read/read-modify-write mix (Zipfian distribution).
Note
  • Some parameters, such as the ZSTD compression algorithm and the INDEX encoding algorithm, are supported only by ApsaraDB for HBase Performance-enhanced Edition and ApsaraDB for Lindorm. You can use ZSTD and INDEX to achieve better performance.
  • re_create_tablepre_loadTo rerun a test, you can disable and to skip table creation and data loading if these steps were completed previously. Skipping the data loading phase reduces the total test time to approximately 3.5 hours, though the actual time may vary based on cluster specifications.

Run the test

Click the Build button to submit the test task to the execution queue.

recordcount10000000batch_rows100scan_rows100interval_time600Case 1Case 10Build

Analyze the test results

After the test completes, one or more CSV files are generated. You can view and download these files from your browser.

The generated CSV files include batchwrite.csv, scan.csv, singleread.csv, singlewrite.csv, total_result.csv, and workload result files from workloada.csv to workloadf.csv. On the Jenkins build details page, go to the Build Artifacts section and click view to see the content of a file.

The following example shows the data format in a CSV file.

TestName,OperationType,Throughput(rows/s),AverageLatency(us),P95Latency(us),P99Latency(us),P999Latency(us)
batchwrite,BATCH,101437.0,125750.6753771419,210431.0,254463.0,314111.0
scan,SCAN,1655724.0,7704.259769699419,21295.0,58655.0,85887.0
singleread,READ,75501.0,1689.6162362271912,2044.0,2669.0,22415.0
singlewrite,INSERT,42512.0,3001.4856138550078,4483.0,9263.0,50271.0
workloada,READ,26010.0,1996.9924856520854,2789.0,4135.0,27535.0
workloada,UPDATE,26011.0,2907.7582520171654,3963.0,5891.0,31071.0
workloadb,READ,59728.0,1989.3114753677578,2771.0,3699.0,19631.0
workloadb,UPDATE,3150.0,2770.810318072335,4025.0,5311.0,22783.0
workloadc,READ,67388.0,1892.5555709970954,2555.0,3433.0,17103.0
workloadd,READ,61513.0,1926.9758923129186,2717.0,3705.0,15391.0
workloadd,INSERT,3237.0,2781.8499603085943,4055.0,5623.0,18815.0
workloade,INSERT,188.0,4851.740461394097,13071.0,48319.0,109183.0
workloade,SCAN,349277.0,26626.44060447983,68415.0,417535.0,463615.0
workloadf,READ,28806.0,2688.784531208804,4383.0,10071.0,55967.0
workloadf,UPDATE,14402.0,3481.2656232319496,5551.0,8895.0,55455.0
workloadf,READ-MODIFY-WRITE,14402.0,6398.633906954003,9551.0,24639.0,66623.0

Troubleshooting

If the test terminates with an error, check the following:

  • Verify that Java and Python are installed and that their versions are correct.
  • Verify that the cluster endpoint and any service-specific parameters are configured correctly.
  • Verify that the target cluster supports the specified compression algorithm.
  • Verify that the target cluster is running and responding to requests.
Previous：Use Shell to test performanceNext：Benchmark methods