Migrate Hive data to SelectDB-ApsaraDB for SelectDB(SelectDB)-阿里云帮助中心

ApsaraDB for SelectDB supports migrating Hive data through Catalog, X2Doris, DataWorks, and OSS Load. Select a method based on your data volume and business scenario.

Choose a method

The following table compares available migration methods to help you choose the right one:

Method	Scenarios	Recommendations	Procedure
Catalog	Data is stored on the Alibaba Cloud platform. Note This includes scenarios such as Alibaba Cloud EMR clusters.	No migration traffic costs. Note If Hive and SelectDB are in the same VPC, the migration uses internal network traffic. No dependency on external components.	Hive data source
OSS Load	Data is not stored on the Alibaba Cloud platform.	No migration traffic costs. Note OSS Load transfers data from Object Storage Service (OSS) to SelectDB over the internal network, avoiding traffic costs.	OSS Load
DataWorks	Your data is managed by DataWorks, or you use DataWorks as your data development platform.	Provides a visual interface that is simple and easy to use.	Import data using DataWorks
X2Doris	You want to use as few internal SelectDB resources as possible during migration. You want a one-stop solution for database and table schema migration.	Provides a visual interface that is simple and easy to use. Uses built-in Spark to reduce the consumption of internal SelectDB resources during migration. Lets you define database and table schemas during migration.	Import Hive source data

Incremental data migration guide

In production, Hive data typically includes both offline and incremental data. A common reason for migrating Hive data to SelectDB is to accelerate queries by replicating data into a data warehouse. For incremental data, consider the following methods:

Write data to SelectDB in parallel when producing the Hive data.
Schedule a periodic job to read partition data from Hive and write it to SelectDB.

References

For more information about Hive, see Hive data source.