This topic describes how to use the PolarDB for MySQL columnar engine to read OSS external table data for joint analysis between columnar tables and external tables.
Background
As business data grows, you face increasing storage costs, more complex analytical workloads, and the need to share data across multiple systems. Using the columnar engine to read OSS external table data addresses these challenges:
Cost-effective storage: OSS provides cloud-native object storage at a competitive price.
High-performance analytics: The columnar engine delivers fast query execution and flexible analytical capabilities.
Open data formats: ORC and Parquet formats provide broad compatibility, high compression ratios, and seamless cross-system data exchange.
File-based external tables
An external table is a special table type whose data resides in an external storage system such as Object Storage Service (OSS), HDFS, or MaxCompute. The database stores only metadata and access paths. You can query external tables with standard SQL to access heterogeneous data sources.
PolarDB supports external tables that connect to external data sources. Unlike regular tables, PolarDB stores only the metadata for an external table. When you run a query, PolarDB reads data directly from the external source. No data is stored in PolarDB itself. With file-based external tables, you can query ORC and Parquet data files on external storage systems without importing the data into the database or relying on any Metastore service.
For more information about file-based external tables, see Query data in OSS external tables.
Joint analysis with columnar tables
Columnar tables use the PolarDB columnar storage engine. You can perform joint analysis between columnar tables and external tables to run high-performance computations on data from multiple sources within a single PolarDB cluster. This allows you to run correlated queries and aggregate analysis across PolarDB columnar data and OSS external data in a single SQL statement, without importing external data into the database.
Performance benchmarks
For performance test results of ORC and Parquet file-based external tables, see Query data in OSS external tables.