Delta external tables (XIHE SQL)

更新时间:
复制 MD 格式

The XIHE engine of AnalyticDB for MySQL supports querying existing Apache Delta Lake tables on OSS through external tables. After you register an external table, you can query the latest snapshot data of a Delta table with standard SQL and leverage partition pruning and column pruning to optimize performance.

Important

The XIHE engine currently supports only querying existing Delta Lake tables. Writing to Delta tables by using XIHE SQL is not supported. To write to Delta tables, use the Spark SQL engine. For more information, see Read and write data to Delta external tables.

Capability overview

Capability

Support status

Query existing Delta tables

Supported

Delta reader v1 / v3 tables

Supported

Parquet data files

Supported

Column pruning

Supported

Complex type reads (ARRAY / MAP)

Supported

Partitioned table reads

Supported

Partition pruning

Supported

Delta log statistics

Supported

Deletion Vector reads

Supported

JOIN / aggregation / sorting

Supported

INSERT / UPDATE / DELETE / MERGE

Not supported

Prerequisites

  • The cluster edition is Enterprise Edition, Basic Edition, or Data Lakehouse Edition.

  • The kernel version of the cluster is 3.2.8 or later.

  • An external database is created. For more information, see CREATE EXTERNAL DATABASE.

  • A Delta table directory (containing _delta_log) that was created by an engine such as Spark, Flink, or Databricks already exists on OSS.

Create a table

Syntax

CREATE EXTERNAL TABLE [IF NOT EXISTS] <db>.<table> (
    <col1>  <type1>,
    <col2>  <type2>,
    ...
)
STORED AS DELTA
LOCATION '<delta_table_location>';

Clause

Required

Description

STORED AS DELTA

Yes

Declares the external table format as Delta Lake.

LOCATION

Yes

The OSS path of the Delta table root directory (the directory that contains _delta_log). The path cannot point to the _delta_log directory itself or a partition subdirectory.

Note

After registration, the external table reads the actual schema and file list from _delta_log. We recommend that DDL columns match the underlying Delta schema for easier troubleshooting. The final query schema is determined by _delta_log, and type mismatches may cause query failures. Run DESCRIBE after registration to view the actual Delta schema.

Table creation example

-- Create an external database
CREATE EXTERNAL DATABASE IF NOT EXISTS ext_delta_db;

-- Register a Delta external table
CREATE EXTERNAL TABLE ext_delta_db.delta_orders (
    order_id     BIGINT,
    user_id      BIGINT,
    order_status STRING,
    total_price  DECIMAL(18, 2),
    order_date   DATE
)
STORED AS DELTA
LOCATION 'oss://<your-bucket>/warehouse/delta_orders/';

Other DDL operations

-- View column structure
DESCRIBE ext_delta_db.delta_orders;

-- Drop an external table
DROP TABLE IF EXISTS ext_delta_db.delta_orders;
Note

DROP TABLE only drops the external table definition in AnalyticDB. It does not delete the Delta table directory or data files on OSS.

Query data

Basic queries

By default, queries read the latest snapshot, which ensures data consistency within a single query.

-- Query all data from the table
SELECT * FROM ext_delta_db.delta_orders LIMIT 10;

-- Aggregate query
SELECT order_status, COUNT(*) AS order_count, SUM(total_price) AS total
FROM ext_delta_db.delta_orders
GROUP BY order_status;

-- Conditional filtering
SELECT order_id, order_status, total_price
FROM ext_delta_db.delta_orders
WHERE order_status IN ('PAID', 'SHIPPED') AND total_price >= 100
ORDER BY total_price DESC;

Partition pruning

Partition information is obtained from the metadata and AddFile entries in _delta_log. When you filter on partition columns, the engine prunes irrelevant partitions based on partition values in the Delta log, reducing the number of files scanned:

-- Single-value filtering
SELECT * FROM ext_delta_db.delta_orders
WHERE order_date = DATE '2026-06-11';

-- Range filtering
SELECT order_date, COUNT(*) AS cnt, SUM(total_price) AS total
FROM ext_delta_db.delta_orders
WHERE order_date >= DATE '2026-06-01' AND order_date < DATE '2026-07-01'
GROUP BY order_date;
Note

You do not need to declare PARTITIONED BY in the DDL when you register an external table. The partition information is automatically read from _delta_log.

Partition pruning currently covers common partition column types such as Boolean, integer, floating-point, string, binary, date, and timestamp. Decimal partition columns can be read, but partition pruning is not recommended when they are used as filter conditions.

Column pruning and predicate pushdown

When reading Parquet files, the engine requests only the columns required by the query. For structured fields, eligible subfields are also pruned at the Parquet read level. Predicates that cannot be fully pushed down are applied by the execution engine at runtime.

-- Read only the order_id and total_price columns
SELECT order_id, total_price
FROM ext_delta_db.delta_orders
WHERE total_price BETWEEN 100 AND 500;

Statistics

When the Delta log contains file-level statistics, the external table reads numRecords, min/max, and nullCount values to estimate row counts and column statistics for the optimizer.

If the Delta log lacks statistics or the file count exceeds the scan limit, statistics degrade to unknown or are estimated from file sizes. This may affect execution plan selection but does not change query results.

JOIN and aggregation

Delta external tables support JOIN, aggregation, sorting, and window functions in XIHE SQL. The query optimizer determines the JOIN strategy.

-- JOIN a Delta external table with another table
SELECT o.order_id, o.total_price, u.user_name
FROM ext_delta_db.delta_orders o
JOIN dim_db.users u ON o.user_id = u.user_id
WHERE o.order_date = DATE '2026-06-11';

Supported data types

Delta type

Query type

Description

BOOLEAN

BOOLEAN

Boolean

BYTE / SHORT / INTEGER / LONG

TINYINT / SMALLINT / INT / BIGINT

Integer types

FLOAT / DOUBLE

FLOAT / DOUBLE

Floating-point types

DECIMAL

DECIMAL(p, s)

Fixed-point

STRING

STRING / VARCHAR

String

BINARY

VARBINARY

Binary

DATE

DATE

Date

TIMESTAMP / TIMESTAMP_NTZ

TIMESTAMP

Timestamp. TIMESTAMP_NTZ (no time zone) is read as TIMESTAMP

ARRAY

ARRAY<type>

Array

MAP

MAP<key_type, value_type>

Map

Limits

  • Delta external tables are primarily used to query existing Delta tables. The table structure, partitions, and snapshots are maintained by _delta_log.

  • Only querying existing Delta tables is supported. INSERT, UPDATE, DELETE, and MERGE operations through XIHE SQL are not supported.

  • Only Delta tables whose underlying data format is Parquet are supported. Non-Parquet formats are not supported.

  • Reading Delta tables with Deletion Vectors enabled is supported. Delete markers are automatically applied during queries.

  • LOCATION must point to the Delta table root directory (the directory that contains _delta_log). It cannot point to the _delta_log directory itself or a partition subdirectory.

  • We recommend that the columns in the external table DDL be consistent with the underlying Delta schema. The final query schema is determined by _delta_log. Type mismatches may cause query failures.

  • DROP TABLE only drops the external table definition in AnalyticDB. It does not delete the Delta table directory or data files on OSS.

  • ALTER TABLE only modifies the external table definition in AnalyticDB without affecting the underlying Delta schema. The query schema is always determined by _delta_log. To evolve the Delta schema, make changes in the upstream production pipeline.

  • Decimal partition columns can be read, but partition pruning may have limited effect when these columns are used as filter conditions.

  • Creating new Delta partitioned tables through external table DDL is not supported. Existing partitioned tables can be read and benefit from partition pruning.

  • Complex types (ARRAY, MAP) can be read. Complex filter conditions on complex types are typically applied by the execution engine.

  • If the Delta log schema and the underlying Parquet file schema are inconsistent, a schema mismatch error may occur during queries.

  • Partition values are derived from the string representation in the Delta log. An error may occur during partition value conversion if the partition column type does not match expectations.