Quick start with columnar table
This topic describes how to quickly enable and try the columnar table analysis feature of PolarDB for MySQL.
Applicable scope
Cluster version: Must be MySQL 8.0.2, and the minor kernel version must be 8.0.2.2.34 or later.
Primary node specification: Memory must be at least 8 GB.
Enable columnar table analysis
Enable columnar analysis for an existing cluster
The columnar table analysis feature of PolarDB for MySQL is supported only for MySQL 8.0.2. The minor kernel version must be 8.0.2.2.34 or later. If your version does not meet this requirement, upgrade the minor kernel version first, and then enable columnar table analysis.
Log in to the PolarDB console,In the navigation pane on the left, click Clusters. Select the Region where the cluster is deployed, and then click the cluster ID to go to the cluster details page。
On the Basic Information page, click Column-Oriented Table Analysis Acceleration and then click Enable. After the cluster restarts, the columnar table analysis feature is enabled.
Enable columnar analysis when purchasing a new cluster
Log on to the PolarDB console. In the left-side navigation pane, click Clusters. Select the region where the cluster resides and click Create Cluster.
On the purchase page, configure the following key settings:
Creation Method: Create Primary Cluster
Engine Version: MySQL 8.0.2
Column-Oriented Table Analysis Acceleration: Enable
Configure other settings based on your business requirements and complete the purchase. For more information, see Custom purchase.
Quick start
After you enable columnar table acceleration, tables created with the specified storage engine and storage format use the columnar format by default. The following example shows how to create a columnar table, bulk-load data, and run an analytical query.
Step 1: Create a columnar table
Run the following SQL statement to create a columnar table.
CREATE TABLE sales_records (
sale_id INT PRIMARY KEY AUTO_INCREMENT,
product_id INT NOT NULL,
region VARCHAR(50),
sale_date DATE,
amount DECIMAL(10, 2)
);Step 2: Create a stored procedure for bulk loading
Create a stored procedure that inserts one million rows and commits every 5,000 rows.
DELIMITER //
CREATE PROCEDURE InsertMillionRows()
BEGIN
DECLARE i INT DEFAULT 1;
SET autocommit = 0;
WHILE i <= 1000000 DO
INSERT INTO users (username, email, age)
VALUES (
CONCAT('user_', LPAD(i, 7, '0')),
CONCAT('user_', LPAD(i, 7, '0'), '@test.com'),
FLOOR(18 + RAND() * 50)
);
SET i = i + 1;
IF i % 5000 = 0 THEN
COMMIT;
END IF;
END WHILE;
COMMIT;
SET autocommit = 1;
END //
DELIMITER ;Step 3: Call the stored procedure
Run the following statement to call the stored procedure and bulk-load data.
CALL InsertMillionRows();Step 4: Run an aggregate query
Run the following aggregate query to verify that the data has been loaded.
SELECT COUNT(*) FROM users;Expected output:
+----------+
| COUNT(*) |
+----------+
| 1000000 |
+----------+
1 row in set (0.01 sec)Step 5: View the columnar execution plan
Run the EXPLAIN command to view the columnar execution plan and verify that the query uses the columnar engine.
EXPLAIN SELECT COUNT(*) FROM users;The expected output is as follows. The Extra Info column displays IMCI Execution Plan, which indicates that the query uses the columnar execution engine.
+----+--------------------+-------+--------+--------+---------------------------------------------------------------+
| ID | Operator | Name | E-Rows | E-Cost | Extra Info |
+----+--------------------+-------+--------+--------+---------------------------------------------------------------+
| 1 | Select Statement | | | | IMCI Execution Plan (max_dop = 12, max_query_mem = 858993459) |
| 2 | └─Aggregation | | | | |
| 3 | └─Table Scan | users | | | |
+----+--------------------+-------+--------+--------+---------------------------------------------------------------+Configure row-columnar execution routing
Columnar tables are designed for large-scale data analytics. They combine columnar storage, vectorized execution, and parallel computing to deliver high-performance SQL analytics at low cost. Read-only nodes use this columnar execution model by default. However, this model cannot fully leverage primary key indexes for high-concurrency point lookups.
If your workloads require both high-concurrency point lookups and complex analytical queries, you can create separate cluster endpoints to isolate resources so that the two workload types do not affect each other:
Complex analytical queries: Add standard read-only nodes and set the
loose_use_imci_engineparameter toON(the default isON). Queries automatically use the columnar execution engine.High-concurrency point lookups: Set the
loose_use_imci_engineparameter toOFFon the corresponding read-only node. Queries use the row-based execution engine to take advantage of primary key indexes.
Cluster parameter settings for columnar tables
After you enable the columnar table analysis acceleration option, X-Engine is enabled by default and the default table format is columnar. The following table describes the default settings of the related parameters:
Parameter | Description | Valid values | Default value | Restart required |
| Specifies whether to enable the high compression engine (X-Engine). | [0-1] | 1 | Yes |
| The default storage engine for the cluster. | [InnoDB|xengine] | xengine | Yes |
| The default table format for the X-Engine storage engine. | [default|row|column] | column | No |
| Specifies whether to enable the integration between columnar tables and the In-Memory Column Index (IMCI) execution engine. | [ON|OFF] | ON | Yes |
| Specifies whether to enable the OSS data LRU cache for In-Memory Column Index (IMCI). | [ON|OFF] | ON | No |
| The percentage of memory used by the X-Engine storage engine. | [5-95] | 90 | No |
| The percentage of X-Engine memory allocated to the columnar table analysis feature. | [0-100] | 80 | No |