Beam: A proprietary high-performance storage engine

Beam is a next-generation storage engine for AnalyticDB for PostgreSQL, built on the Table Access Method of PostgreSQL 12.

Beam consists of two parts:

• Delta: A row-oriented format for real-time writes.

• Base: A PAX-structured, column-oriented format for batch writes and large-scale scan scenarios.

Compared to heap tables, Beam significantly reduces disk I/O and improves analytical query performance. It also supports primary keys, write deduplication, and concurrent Update and Delete operations, enabling direct data synchronization through Data Transmission Service (DTS). With a single storage copy, Beam serves both transactional processing (TP) and analytical processing (AP) workloads, eliminating the need for separate row-oriented and column-oriented engines.

Notes

• The Beam storage engine is supported only in Serverless mode for instances of v7.0.x.

• Beam ended its public preview and was officially released in Serverless mode version v7.0.6.2, which includes fixes from the public preview. Upgrade to v7.0.6.2 or later as soon as possible.

Beam features

High-performance real-time writes

Beam storage consists of a row-oriented Delta store and a PAX-structured, column-oriented Base store. Beam automatically routes data to the appropriate store based on the write method. Data written through real-time streaming methods such as INSERT INTO VALUES goes to the Delta store, achieving write performance comparable to heap tables.

High-throughput batch imports

Data written through batch methods such as COPY or INSERT INTO SELECT goes directly to the column-oriented Base store, delivering higher throughput and better write performance.

High-performance AP queries

Beam optimizes analytical query performance through the following methods:

• Column pruning

• Support for multiple compression algorithms

• Support for Zonemap filtering

• Support for I/O prefetch

Together, these methods reduce disk I/O during queries and improve I/O utilization, significantly enhancing analytical query performance.

Support for primary keys and write deduplication

Beam fully implements PostgreSQL native primary keys. You can create a primary key index on a Beam table just as you would on a heap table to enable data deduplication. Beam also supports UPSERT syntax.

Support for DTS synchronization

With primary key support, Beam tables work in DTS data synchronization pipelines just like heap tables. You can set a Beam table as the target in a DTS task for better query performance, eliminating extra synchronization between row-oriented and column-oriented tables.

References

• To learn how to use the Beam storage engine, see Use Beam.

• Beam supports multiple compression algorithms. Dictionary encoding compresses string data into integer data, improving storage efficiency and accelerating filtering and aggregation queries. For more information, see Dictionary encoding.

• If you frequently perform range queries or equality filters on specific columns, you can specify a Beam sort key to improve query performance. For more information, see Beam sort key optimization (V7.0).