Storage modes

Capability matrix

Use OLAP Mode (default)

No additional configuration is required. Use standard SQL directly:

CREATE INDEX orders_idx USING COLUMNAR ON orders(*)
PARTITION BY ENUMERABLE (dt, bucket(128, id))
WITH (
  `lindorm_columnar.user.index.database` = 'my_index_db',
  `lindorm_columnar.user.index.table`    = 'orders_index'
);

To explicitly specify the OLAP resource group to use (for example, when multiple OLAP resource groups exist in the instance), configure the sr.resourceGroup parameter:

CREATE INDEX orders_idx USING COLUMNAR ON orders(*)
PARTITION BY ENUMERABLE (dt, bucket(128, id))
WITH (
  `lindorm_columnar.user.index.database`   = 'my_index_db',
  `lindorm_columnar.user.index.table`      = 'orders_index',
  `lindorm_columnar.user.sr.resourceGroup` = 'cg0'
);

OLAP Mode description

• __timestamp metadata column: Lindorm automatically adds a __timestamp BIGINT column to the columnstore table for the OLAP engine primary key table to determine record versions (larger values indicate newer records). Users typically do not need to be aware of this column in business SQL.

• FULL_WAL automatically enabled: When creating an OLAP Mode columnstore index for a wide table, Lindorm automatically enables the following properties on the wide table to ensure that the WAL carries complete row data:

• WAL_EDIT_WITH_FULL_ROW = true

• FULL_ROW_EDIT_CARRY_LATEST_DATA = true

This operation is idempotent and is executed when the first OLAP Mode index is created.

• Write semantics: Data is written to the OLAP engine in APPEND mode. The primary key table internally performs UPSERT based on __timestamp, which is transparent to users.

• Deletion latency: After executing DELETE on a wide table, the deletion takes approximately 1 day to take effect in the OLAP Mode columnstore index. If your business has stricter requirements for deletion visibility, use Lake Mode.

Directly connect to the OLAP engine to query columnstore tables

Columnstore tables in OLAP Mode are warehouse column store tables within the OLAP resource group. You can directly connect to the target OLAP resource group through the MySQL protocol for queries without going through computing engine hint routing. The database and table names are those configured in the WITH clause during CREATE INDEX:

• Database name: lindorm_columnar.user.index.database

• Table name: lindorm_columnar.user.index.table

Using the preceding orders_idx as an example, the direct connection query is as follows:

-- Execute after connecting to the target OLAP resource group
SET CATALOG default_catalog;

SELECT dt, COUNT(*) AS order_cnt, SUM(amount) AS gmv
FROM my_index_db.orders_index
WHERE dt BETWEEN '2026-08-01' AND '2026-08-10'
GROUP BY dt;

For more information about connecting to OLAP resource groups, see Service access. For more information about warehouse column store tables, see Warehouse column store tables.

Use Lake Mode

Explicitly disable OLAP routing in the WITH clause (when the tenant administrator has enabled this switch), or contact Lindorm technical support to set the default engine of the instance to lake:

CREATE INDEX orders_idx USING COLUMNAR ON orders(*)
PARTITION BY ENUMERABLE (dt, bucket(128, id))
WITH (
  `lindorm_columnar.user.index.database`   = 'my_index_db',
  `lindorm_columnar.user.index.table`      = 'orders_index',
  `lindorm_columnar.user.engine`           = 'lake'
);

Directly connect to the computing engine to query columnstore tables

Columnstore tables in Lake Mode are lake mode column store tables built on Apache Iceberg. You can query them directly through Lindorm Data Processing Service (LDPS) Spark SQL, or read them within the same instance by ETL resource groups or external Spark/Flink engines. The database and table names also come from the WITH parameters during CREATE INDEX:

• Database name: lindorm_columnar.user.index.database

• Table name: lindorm_columnar.user.index.table

Using the preceding orders_idx as an example, the direct connection query is as follows:

-- Execute through LDPS Spark SQL
SET CATALOG lindorm_columnar;

SELECT dt, COUNT(*) AS order_cnt, SUM(amount) AS gmv
FROM my_index_db.orders_index
WHERE dt BETWEEN '2026-08-01' AND '2026-08-10'
GROUP BY dt;

For more information about lake mode column store tables, see Lake mode column store tables.

Selection recommendations

Both storage modes can cover typical columnstore index use cases. Choose based on your data scale, update characteristics, and ecosystem requirements.

Prefer OLAP Mode

Recommended for scenarios with data volumes less than 100 TB that require high update efficiency and data freshness for real-time analytics. OLAP Mode uses an advanced delete vector mechanism, where row-level updates and deletions are completed efficiently without rewriting data files. It has the following advantages:

• Highly efficient data updates: Primary key UPSERT/DELETE operations are written incrementally to delete vectors and new version files, avoiding large-scale data rewriting.

• High data freshness: Low update costs result in shorter incremental synchronization latency. Data can be hit by OLAP queries shortly after being written.

• Low synchronization resource usage: Spark synchronization resources required per unit of data are significantly lower than in Lake Mode, allowing the same write throughput with fewer computing resources.

Prefer Lake Mode

Recommended for scenarios that require deep lakehouse ecosystem integration or do not want to depend on OLAP resource groups. Columnstore tables in Lake Mode are standard Iceberg tables with the following advantages:

• Spark-based ELT query analysis: You can directly read and process columnstore indexes using Spark SQL in LDPS, making it easy to handle ETL/ELT data pipelines.

• High open-source ecosystem compatibility: Iceberg tables can be read by mainstream open-source engines such as Spark, Presto, and Trino, enabling cross-engine data sharing.

• No dependency on OLAP resource groups: Uses only LindormDFS and computing resources, without the need to additionally activate and maintain OLAP engine resource groups.

If you need help evaluating which mode to choose, contact Lindorm technical support (DingTalk ID: s0s3eg3).