FAQ
This topic provides answers to frequently asked questions about Lindorm columnstore indexes.
Billing and capacity
Q: Are additional fees incurred after creating a columnstore index?
A: Yes. Fees consist of two main parts:
-
Storage fees for columnstore index data.
-
CU fees for the actual data synchronization between the primary table and the columnstore index.
Q: After data in a wide table is cleared due to TTL expiration, is the columnstore index data automatically cleared?
A: No. Columnstore indexes are not aware of the TTL policy of wide tables. To clean up columnstore index data, handle it explicitly on the business side.
Partitioning strategies
Q: Can partition expressions contain non-primary key fields?
A: No. Fields in both regular partition expressions and bucket partition expressions must all be primary key fields of the wide table.
Q: Can bucket partition expressions contain complex computations?
A: No. Bucket partition expressions only accept the bucket(bucket_num, column_name) format, where column_name must be a primary key field.
Q: What are the impacts of having too many or too few partitions?
A:
-
Too many: Metadata bloat, which affects query efficiency. We recommend that the data volume of a single partition be ≥ 50 MB and the number of buckets be ≤ 1024.
-
Too few: Affects data read/write throughput or causes skew. We recommend that the data volume of a single partition be ≤ 512 MB.
Index management
Q: Can multiple columnstore indexes be created on a single wide table?
A: Yes. A single wide table supports multiple columnstore indexes, and they can be hosted by different engine modes (for example, one OLAP Mode index + one Lake Mode index).
Q: Why do I get an error when trying to create a columnstore index again after a failed creation?
A: You cannot create columnstore indexes with the same name on a wide table, regardless of whether the existing index is in a failed state. First run DROP INDEX to delete the failed index, and then create a new one. For more information, see DROP INDEX.
Q: Can I directly access columnstore index data through the Lindorm computing engine?
A: Yes. You need to first customize the index table name (lindorm_columnar.user.index.database / lindorm_columnar.user.index.table) and then access it through the computing engine. For more information, see Access columnstore data.
Exercise caution when performing manual modifications on columnstore index tables (such as ALTER TABLE and DROP TABLE), as these operations may cause data synchronization failures. If you need to make modifications, contact Lindorm technical support (DingTalk ID: s0s3eg3).
OLAP Mode
Q: Why is the WAL_EDIT_WITH_FULL_ROW property of the primary table automatically enabled after creating an OLAP Mode columnstore index?
A: This is a necessary prerequisite for OLAP Mode. To ensure that the OLAP engine primary key table can obtain complete row data for UPSERT, Lindorm automatically enables FULL_WAL so that the WAL always carries complete row content. This operation is idempotent and does not affect existing business write semantics on the primary table.
Q: What is the purpose of the __timestamp column in OLAP Mode columnstore tables?
A: __timestamp is an implicit column automatically written by Lindorm, used by the OLAP engine primary key table to determine record versions (larger values indicate newer records). Business SQL typically does not need to be aware of this column.
Q: Can I switch an existing Lake Mode columnstore index to OLAP Mode online?
A: Online switching is not supported. We recommend using the hot rebuilding approach: create a new OLAP Mode index, and after it becomes ACTIVE, delete the old index and switch the columnstore table name.
Q: In OLAP Mode, sr.resourceGroup reports the error does not exist for current instance. How do I troubleshoot this?
A: Check the following:
-
Whether the parameter spelling is correct (case-insensitive, but we recommend using lowercase to match the console).
-
Whether the resource group exists in the current instance and its type is OLAP (Lindorm only allows OLAP-type resource groups as columnstore index backends).
-
Whether the current token has access permissions for this resource group.
If you cannot resolve the issue, contact Lindorm technical support.
JSON and schema
Q: Can a JSON field be configured with both static and dynamic flattening simultaneously?
A: No. The same JSON field can only be configured in either jsonMapping. or dynamicJsonColumns, not both.
Q: Why are the JSON flattened fields empty in existing data after I configured dynamicJsonColumns?
A: Dynamic JSON flattening takes effect only during the incremental synchronization phase, not during the full build phase. This is a known limitation of the current implementation. If you need to flatten existing JSON data, use static flattening (jsonMapping.).
Q: The primary table has wildcard columns enabled, and I created an index with dynamicSchema=true in OLAP Mode. Why are the wildcard columns not synchronized?
A: Wildcard column synchronization is currently supported only in Lake Mode. OLAP Mode skips wildcard columns and logs a warning. If you rely heavily on wildcard column synchronization, use Lake Mode.
Others
Q: How do I contact technical support?
A: Lindorm technical support DingTalk ID: s0s3eg3.