Deploy high-performance MySQL on Block Storage
MySQL is a popular open-source relational database. In this hands-on lab, you will learn how to optimize its performance using 16K atomic writes and parameter tuning.
Alibaba Cloud's RDS for MySQL, which is based on an Alibaba-maintained source code branch of MySQL, supports 16 KB atomic write by default for higher performance. For more information, see 16 KB atomic write.
Background information
MySQL parameters
MySQL has hundreds of parameters that cover aspects such as memory management, cache size, connection handling, logging, replication configuration, and query optimization. The specific number of parameters depends on the MySQL version and the installed storage engine. Correctly configuring these parameters can significantly improve MySQL's performance, but incorrect configurations can degrade performance and, in severe cases, even compromise data accuracy.
The following sections recommend how to tune MySQL parameters for optimal performance. You can configure these parameters in the provided lab environment, or you can purchase your own instance and follow the instructions in this topic.
MySQL with 16K atomic write
Using 16K atomic write with MySQL improves database read and write performance and reduces write amplification. Implementing 16K atomic write requires support from the database, the underlying hardware, and the file system.
MySQL typically uses the DoubleWrite mechanism to ensure data consistency. However, this mechanism increases I/O load and latency, which can degrade performance, especially under high workloads. This lab improves performance by using the 16K atomic write feature of ESSD, which allows you to disable the DoubleWrite parameter. Disabling this parameter reduces the number of write operations and their latency. To maintain data consistency without DoubleWrite, this approach requires support from both the underlying hardware and the file system.
Billing
When you deploy the MySQL service, you are billed for the Alibaba Cloud resources used, such as ECS instances and ESSD cloud disks, based on their billing methods. If you use one-click deployment in the lab, there is no charge for the lab task itself.
Alibaba Cloud resources used for one-click lab deployments are available only on a pay-as-you-go basis.
Deploy the MySQL service
Manual deployment
The one-click deployment feature in Labs is designed for quick validation and testing of new features. It is not intended for direct use in a production environment. For production environments, use manual deployment to ensure system security and stability.
To manually deploy the MySQL service, first purchase an ECS instance and an ESSD cloud disk. Then, run commands on the ECS instance to deploy the MySQL service.
Procedure
-
Configure the ECS instance as follows:
-
Instance type: ecs.c6.2xlarge.
-
This topic uses CentOS 7.9 as an example. Adjust the commands based on your operating system.
-
Mount an ESSD cloud disk as the data disk. This topic uses a 500 GiB ESSD PL1 cloud disk as an example.
-
Ensure that MySQL is not already installed on the ECS instance.
For more information, see Create a custom instance.
-
-
Connect to the ECS instance.
For more information, see Use Workbench to log on to a Linux instance.
-
Install MySQL.
-
Run the following commands to download the MySQL installation package. This example uses MySQL 8.0.
cd /tmp/ wget http://mirrors.cloud.aliyuncs.com/mysql/MySQL-8.0/mysql-8.0.27-1.el7.x86_64.rpm-bundle.tar -
Run the following command to extract the installation package.
tar -xf mysql-8.0.27-1.el7.x86_64.rpm-bundle.tar -
Run the following command to install MySQL.
sudo yum install -y mysql-community-{server,client,common,libs,devel}-* -
Run the following command to stop the MySQL process.
sudo systemctl stop mysqld
-
-
Initialize the cloud disk.
-
Run the following command to obtain the device name of the data disk.
sudo fdisk -lThe following output shows that the device name of the uninitialized data disk is
vdb.[ecs-user@iZxxx Z tmp]$ sudo fdisk -l Disk /dev/vda: 42.9 GB, 42949672960 bytes, 83886080 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x000c2760 Device Boot Start End Blocks Id System /dev/vda1 * 2048 83886046 41941999+ 83 Linux Disk /dev/vdb: 42.9 GB, 42949672960 bytes, 83886080 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes -
Run the following command to set a global environment variable.
DEV_LABEL=vdbReplace
vdbwith the device name of your data disk. -
Run the following command to set the MySQL data path.
MYSQL_HOME=/home/ecs-user/${DEV_LABEL} -
Run the following command to prevent virtio from splitting I/O requests.
sudo sh -c 'echo 256 >/sys/block/vdb/queue/max_sectors_kb' -
Run the following command to create the MySQL directory.
if [ ! -d ${MYSQL_HOME} ]; then sudo mkdir -p ${MYSQL_HOME} fi -
If the data disk is already mounted, run the following command to unmount it.
If it is not mounted, skip this step.
sudo umount /dev/${DEV_LABEL} -
Run the following command to format the file system, enable the
bigallocoption, and set the file system block size to 16 KB.echo y | sudo mkfs.ext4 -O bigalloc -C 16k /dev/${DEV_LABEL} -
Run the following command to mount the data disk.
sudo mount /dev/${DEV_LABEL} ${MYSQL_HOME} -
Run the following command to create the required MySQL directories.
sudo mkdir -p ${MYSQL_HOME}/data/dbs sudo mkdir -p ${MYSQL_HOME}/data/mysql sudo mkdir -p ${MYSQL_HOME}/log/mysql sudo mkdir -p ${MYSQL_HOME}/log/redo
-
-
Configure MySQL.
-
Run the following command to create and populate the MySQL configuration file (/etc/my.cnf).
sudo tee /etc/my.cnf <<EOF [mysqld] port = 3306 back_log = 3000 datadir = ${MYSQL_HOME}/data/dbs log-bin = ${MYSQL_HOME}/log/mysql/mysql-bin.log log-error = ${MYSQL_HOME}/log/mysql/master-error.log innodb_data_home_dir = ${MYSQL_HOME}/data/mysql/ innodb_log_group_home_dir = ${MYSQL_HOME}/log/redo/ innodb_doublewrite_dir = ${MYSQL_HOME}/doublewrite character_set_server = utf8 sync_binlog = 1000 innodb_doublewrite = 0 max_connections=151 max_allowed_packet=64M innodb_buffer_pool_size=128M innodb_buffer_pool_instances=1 innodb_read_io_threads=4 innodb_write_io_threads=4 innodb_buffer_pool_instances=1 innodb_flush_log_at_trx_commit=1 innodb_flush_method=O_DIRECT default_authentication_plugin = mysql_native_password EOFFor more information about MySQL parameters, see MySQL parameters.
-
Run the following command to initialize MySQL.
sudo mysqld --defaults-file=/etc/my.cnf --initialize
-
-
Start MySQL.
-
Run the following command to start MySQL.
sudo mysqld --user=root --daemonize -
Run the following command to verify that MySQL has started.
ps -aux | grep mysqldThe following output indicates that MySQL has started.
[ecs-user@iZbp... ~]$ ps -aux | grep mysqld root 17144 0.1 4.6 1794466 365248 ? S1 14:03 0:01 mysqld --user=root --daemonize ecs-user 17223 0.0 0.8 112816 976 pts/1 S+ 14:18 0:00 grep --color=auto mysqld
-
-
(Optional) Configure MySQL to start on boot.
-
Run the following command to go to the target directory. If the directory does not exist, use
mkdirto create it.cd /etc/rc.d/init.d -
Run the following command to create a startup script.
sudo tee /etc/rc.d/init.d/ebs_mysql_16k_auto_start.sh <<EOF #!/bin/bash # chkconfig: 345 20 80 # description: EBS MySQL 16K Auto Start Script start() { DEV_LABEL=vdb MYSQL_HOME=/home/ecs-user/\${DEV_LABEL} echo 256 >/sys/block/vdb/queue/max_sectors_kb umount /dev/\${DEV_LABEL} mount /dev/\${DEV_LABEL} \${MYSQL_HOME} mysqld --user=root --daemonize } stop() { killall mysqld echo "Stopping mysqld..." } restart() { stop sleep 2 start } case "\$1" in start) start ;; stop) stop ;; restart) restart ;; *) echo "Usage: \$0 {start|stop|restart}" exit 1 esac exit 0 EOF -
Run the following command to make the script executable.
sudo chmod +x /etc/rc.d/init.d/ebs_mysql_16k_auto_start.sh -
Run the following command to add the script as a startup task.
sudo chkconfig --add ebs_mysql_16k_auto_start.sh -
Run the following command to enable the script to start on boot.
sudo chkconfig ebs_mysql_16k_auto_start.sh on
-
One-click deployment in Labs
You can use the one-click deployment feature in the Elastic Block Storage (EBS) Labs to deploy the MySQL service without manually purchasing resources or building the environment. This lab scenario uses the automatic orchestration and deployment capabilities of Resource Orchestration Service (ROS) and combines the 16K atomic write capability of ESSD cloud disks with MySQL use cases for a simpler, hands-on MySQL experience.
-
Go to the Lab scenario page.
-
Log on to the Elastic Block Storage (EBS) console.
If you are logging on to the EBS console for the first time, follow the on-screen instructions to create an EBS service-linked role. For more information, see Service-linked role for EBS.
-
In the left-side navigation pane, choose Laboratory > Lab scenario.
-
In the upper-left corner of the top menu bar, select a region.
-
On the Deploy high-performance MySQL and a comparison environment based on EBS template, click One-click creation of experiment tasks.
-
-
On the Create lab task wizard page, configure the basic information for the lab scenario and click Next.
Parameter
Description
Lab scenario
The default value is Deploy a High-performance MySQL Service Based on EBS.
Lab task name
A default name is provided, which you can customize.
Description
(Optional) Enter a description for the lab task.
-
On the dependency check page, review the dependency check results. When No error found. appears, click Next.
-
On the Resource parameters page, configure the resource parameters and click Next.
Parameter
Description
Zone
Select the availability zone for the instance. The example uses
cn-hangzhou-g.ECS instance type
Select an instance type. This example uses
ecs.c6.2xlarge, which provides 8 vCPUs and 16 GiB of memory.Instance password
A default value is provided. You can modify it as needed.
ImportantSecurely store the instance password. This password is required to connect to the ECS instance.
System disk type
Select a system disk type. This example uses
cloud_essd.The system disk size.
Specify the system disk size as needed. The example uses
40 GiB.Data disk type
Only cloud_essd (ESSD PL1 cloud disk) is supported.
Data disk performance level
Only PL1 is supported.
Data disk size
Specify the data disk size as needed. The example uses
500 GiB.Database password
A default value is provided. You can modify it as needed.
ImportantSecurely store the database password. It is required to connect to MySQL.
Database port
The MySQL server listens on port 3306.
To access the MySQL database, you must also open port 3306 in the security group of the ECS instance. For more information, see Add a security group rule.
-
Review the configuration parameters and billing information, and then click OK. In the Create lab task dialog box, click OK.
After the lab task is created, the system prepares the resources and builds the lab scenario. This process takes about 5 minutes.
Lab tasks rely on the orchestration capabilities of ROS. Each lab task corresponds to one stack. You can create a maximum of 200 lab tasks.
-
When the task status changes to Created, click the Lab task ID to view the task configuration details.
Verification and cleanup
Verify the solution
Verify MySQL availability
-
Log on to the ECS instance.
-
Go to ECS console - Instances. In the top-left corner, select the region and resource group for the target resource.
-
Go to the details page of the target instance, click Connect, and then select Workbench. Follow the on-screen instructions to log on and open the terminal.
-
-
(Conditionally required) Run the following command to get the temporary password for MySQL.
If you deployed the service using the one-click deployment feature in Labs, you can skip this step. Log on to MySQL using the Database Password that you set when you created the lab task.
sudo cat ${MYSQL_HOME}/log/mysql/master-error.log | grep root | grep "temporary password" | awk '{print $13}' -
Run the following command to log on to MySQL.
mysql -uroot -pWhen prompted, enter the password that you obtained in the previous step to log on to MySQL.
[ecs-user@iZbpxxx init.d]$ mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 10 Server version: 8.0.27 Copyright (c) 2000, 2021, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> -
(Conditionally required) Run the following command to change the password for the MySQL root user. If you deployed the service by using one-click deployment in Labs, you can skip this step.
alter user 'root'@'localhost' identified by '<new_password>'; -
Run the following command to exit MySQL.
exit -
(Conditionally required) If you configured MySQL to start on boot, run the following command to restart the ECS instance. The MySQL service starts automatically after the restart.
sudo reboot -
Run the following command to verify that the MySQL service has restarted properly.
ps aux | grep mysqld
Test MySQL performance
To test MySQL performance, follow these steps:
-
Log on to the ECS instance.
-
Go to ECS console - Instances. In the top-left corner, select the region and resource group for the target resource.
-
Go to the details page of the target instance, click Connect, and then select Workbench. Follow the on-screen instructions to log on with your password and open the terminal.
-
-
Run the following command to install SysBench.
sudo yum install -y sysbench -
Run the following command and enter the MySQL database password to log on to MySQL.
mysql -uroot -p -
Run the following command to create the sbtest database.
create database sbtest; -
Run quit to exit MySQL, and then run the following command to perform stress tests for OLTP read/write mixed, OLTP read-only, and OLTP write-only scenarios.
OLTP read and write
Run the following commands to perform the test. For parameter descriptions, see the Performance Testing Guide. The value of
threadsis affected by many factors, such as the instance type, cloud disks, and environment. This example uses 16, but you should adjust this value for your environment.## Prepare data sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=16 oltp_read_write prepare ## Run workload sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=16 --percentile=95 --report-interval=1 oltp_read_write run ## Clean up data sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=16 --percentile=95 oltp_read_write cleanupThe following data is for reference only:
-
QPS: 40131.20
-
TPS: 2006.56
-
Latency: 12.08 ms
SQL statistics: queries performed: read: 8428364 write: 2408104 other: 1204052 total: 12040520 transactions: 602026 (2006.56 per sec.) queries: 12040520 (40131.20 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 300.0273s total number of events: 602026 Latency (ms): min: 2.78 avg: 7.97 max: 93.89 95th percentile: 12.08 sum: 4798582.91 Threads fairness: events (avg/stddev): 37626.6250/109.14 execution time (avg/stddev): 299.9114/0.01OLTP read-only
Run the following commands to perform the test. For parameter descriptions, see the Performance Testing Guide. The value of
threadsis affected by many factors, such as the instance type, cloud disks, and environment. This example uses 16, but you should adjust this value for your environment.## Prepare data sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=16 oltp_read_only prepare ## Run workload sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=16 --percentile=95 --skip-trx=1 --report-interval=1 oltp_read_only run ## Clean up data sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=16 --percentile=95 oltp_read_only cleanupThe following data is for reference only:
-
QPS: 51967.26
-
TPS: 3711.95
-
Latency: 6.55 ms
SQL statistics: queries performed: read: 15591072 write: 0 other: 0 total: 15591072 transactions: 1113648 (3711.95 per sec.) queries: 15591072 (51967.26 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 300.0158s total number of events: 1113648 Latency (ms): min: 1.34 avg: 4.31 max: 78.50 95th percentile: 6.55 sum: 4797588.65 Threads fairness: events (avg/stddev): 69603.0000/222.22 execution time (avg/stddev): 299.8493/0.00OLTP write-only
Run the following commands to perform the test. For parameter descriptions, see the Performance Testing Guide. The value of
threadsis affected by many factors, such as the instance type, cloud disks, and environment. This example uses 32, but you should adjust this value for your environment.## Prepare data sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=32 oltp_write_only prepare ## Run workload sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=32 --percentile=95 --report-interval=1 oltp_write_only run ## Clean up data sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=root --mysql-password=<database_root_password> --mysql-db=sbtest --table_size=25000 --tables=100 --events=0 --time=300 --threads=32 --percentile=95 oltp_write_only cleanupThe following data is for reference only.
-
QPS: 42661.18
-
TPS: 7110.20
-
Latency: 7.84 ms
SQL statistics: queries performed: read: 0 write: 8533300 other: 4266650 total: 12799950 transactions: 2133325 (7110.20 per sec.) queries: 12799950 (42661.18 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 300.0360s total number of events: 2133325 Latency (ms): min: 0.91 avg: 4.50 max: 824.81 95th percentile: 7.84 sum: 9596350.52 Threads fairness: events (avg/stddev): 66666.4062/189.47 execution time (avg/stddev): 299.8860/0.01 -
Performance result comparison
Compare this topic's solution with a standard deployment to verify the MySQL performance improvement.
-
Comparison specifications
For a fair comparison, both the standard deployment and the solution in this topic use the same configuration: an ecs.g7.large instance type, a 500 GiB ESSD PL1 data disk, a CentOS 7 image, and MySQL 8.0.
-
MySQL deployment steps
-
Deployment in this topic: Refer to Deploy the MySQL service and Verify the solution to install and configure MySQL and install the benchmarking tool.
-
Standard deployment: You can use the comparison instance deployed using one-click deployment in Labs (the instance name starts with EBS_MySQL_Instance_Compare in the ECS console), or you can run the following commands to deploy MySQL manually.
-
-
Test result comparison
ImportantTest results vary based on factors like instance, cloud disk, and environment, so the data in this example is for reference only.
-
MySQL performance test results for this topic's deployment
The SysBench write benchmark shows a transaction throughput of 7110.20 TPS, a query throughput of 42661.18 QPS, and a 95th percentile latency of 7.84 ms.
SQL statistics: queries performed: read: 0 write: 8533300 other: 4266650 total: 12799950 transactions: 2133325 (7110.20 per sec.) queries: 12799950 (42661.18 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 300.0360s total number of events: 2133325 Latency (ms): min: 0.91 avg: 4.50 max: 824.81 95th percentile: 7.84 sum: 9596350.52 Threads fairness: events (avg/stddev): 66666.4062/189.47 execution time (avg/stddev): 299.8860/0.01 -
MySQL performance test results for the standard deployment
SQL statistics: queries performed: read: 0 write: 4780400 other: 2390200 total: 7170600 transactions: 1195100 (3983.21 per sec.) queries: 7170600 (23899.24 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 300.0333s total number of events: 1195100 Latency (ms): min: 0.78 avg: 8.03 max: 266.19 95th percentile: 16.12 sum: 9597436.64 Threads fairness: events (avg/stddev): 37346.8750/106.14 execution time (avg/stddev): 299.9199/0.01
The QPS for this topic's deployment is 42661, while the QPS for the standard MySQL deployment is 23899, representing a performance improvement of approximately 79%. TPS and latency also improved.
-
Clean up resources
If you used one-click deployment in Labs, clean up the resources after you finish testing to avoid incurring additional charges.
To avoid errors, use the one-click delete feature in Labs to remove all resources for a lab task. Do not delete instances or cloud disks directly from the ECS or EBS consoles.
-
Log on to the Elastic Block Storage (EBS) console.
-
In the left-side navigation pane, choose Laboratory > Experiment records.
-
In the upper-left corner of the top menu bar, select a region.
-
In the Actions column of the target lab task, click Delete.
You can also select multiple lab tasks and click Batch Delete at the bottom of the list.
By default, deleting a lab task does not release its associated resources. If you select Delete Resource, the resources in the lab task, such as ECS instances and database instances, are released. Proceed with caution.