Supported data sources
AnalyticDB for MySQL can ingest data from databases, object storage, big data platforms, message queues, and local files into either a data warehouse or a data lake. The right import method depends on your data source and performance requirements.
How it works
AnalyticDB for MySQL supports two ingestion paths:
-
Data warehouse ingestion: Data is pre-processed before loading into AnalyticDB for MySQL's proprietary Xuanwu analytic storage engine. This path delivers high-throughput real-time writes and high-performance real-time queries, making it the right choice when query performance is your top priority.
-
Data lakehouse ingestion: Raw data is stored in open-source table formats (Iceberg and Paimon), either in ADB's built-in lake storage or in your own Object Storage Service (OSS) bucket. Because the data stays in open formats, it can be queried by both the Spark and XIHE engines of AnalyticDB for MySQL and by external engines such as MaxCompute. Use this path when your architecture requires open-source compatibility or multi-engine access. For higher read performance, enable LakeCache to reduce latency compared to reading directly from OSS.
Data lakehouse ingestion is available only for Enterprise Edition, Basic Edition, or Data Lakehouse Edition clusters.
Data warehouse ingestion
| Category | Data source | Import method | Edition | Documentation |
|---|---|---|---|---|
| Database | RDS MySQL | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from RDS MySQL using an external table |
|
One-click data warehouse creation |
Data Warehouse Edition |
Synchronize data to Data Warehouse Edition using one-click data warehousing |
||
|
MySQL analytic instance |
Data Warehouse Edition |
Synchronize data to Data Warehouse Edition using a MySQL analytic instance |
||
| DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS | ||
| DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks | ||
| Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using zero-ETL | ||
| RDS SQL Server | DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS | |
| DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks | ||
| PolarDB Distributed Edition (formerly DRDS) | DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS | |
| DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks | ||
| One-stop synchronization | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Automatically synchronize PolarDB-X metadata | ||
| PolarDB for MySQL | Federated analytics | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using the federated analytics feature | |
| DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS | ||
| Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using zero-ETL | ||
| MongoDB | External table | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from MongoDB using an external table | |
| Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using zero-ETL | ||
| Lindorm | Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from Lindorm | |
| Oracle | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from Oracle | |
| Self-managed MySQL | External table | Data Warehouse Edition | Import data from a self-managed MySQL database | |
|
AnalyticDB for MySQL V2.0 |
Migration tool |
Data Warehouse Edition |
||
| Self-managed HBase | DTS | Data Warehouse Edition | Import data from a self-managed HBase cluster | |
| Storage | OSS | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from OSS using an external table |
| DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks | ||
| Tablestore | External table | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Query and import data from Tablestore | |
| HDFS | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from HDFS using an external table | |
| DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks | ||
| Big data | MaxCompute | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from MaxCompute using an external table |
| DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks | ||
| Flink | Flink | Data Warehouse Edition | Import data from Flink | |
| Message queue | Kafka | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Logstash plugin | Data Warehouse Edition | Import data using Logstash | ||
| Log data | Log data | Data synchronization | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize log data using data synchronization |
| Logstash plugin | Data Warehouse Edition | Import data using Logstash | ||
| Local data | SQLAlchemy | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import DataFrame data using SQLAlchemy | |
| LOAD DATA | Data Warehouse Edition | Import data using LOAD DATA | ||
| Import tool | Data Warehouse Edition | Import data using the import tool | ||
| Kettle | Data Warehouse Edition | Import data using Kettle | ||
Data lakehouse ingestion
Available only for Enterprise Edition, Basic Edition, or Data Lakehouse Edition clusters.
| Category | Data source | Import method | Documentation |
|---|---|---|---|
| Message queue | Kafka | Data synchronization (Recommended) | Synchronize Kafka data using data synchronization |
| Log data | Simple Log Service (SLS) | Data synchronization (Recommended) | Synchronize SLS data using data synchronization |
| Big data | Hive | Data migration | Import data from Hive |
| Storage | OSS | Metadata discovery | Import data using metadata discovery |
References
AnalyticDB for MySQL also supports asynchronous submission of import tasks. For more information, see Submit an asynchronous import task.