DocsICP FilingConsole
Documentation

One-click data warehousing

Updated at:
Copy as MD
Product
My Favorites

The one-click data warehousing feature in Data Management (DMS) enables you to create a real-time data warehouse. Data is synchronized to an AnalyticDB for MySQL database with a seconds-level latency. This helps you obtain real-time and accurate business insights to improve analysis, decision-making, and business performance. This topic describes how to use the one-click data warehousing feature in DMS.

Background information

Traditional methods for creating a data warehouse involve using offline scripts to periodically collect data from a source database, followed by extract, transform, and load (ETL) development. This method results in high latency, often requires you to wait until the next day (T+1) to view data analysis reports, and incurs high technical costs.

The one-click data warehousing feature in DMS combines the data synchronization capabilities of Data Transmission Service (DTS) and the analytics engine of AnalyticDB for MySQL to create a real-time data warehouse.

Note

  • Data Transmission Service (DTS) provides high stability and data reliability. For more information, see What is Data Transmission Service?.

  • AnalyticDB for MySQL is a cloud-hosted, petabyte-scale data warehouse for massively parallel processing (MPP). It is fast, flexible, easy to use, highly scalable, and supports high-concurrency writes. For more information, see What is AnalyticDB for MySQL?.

Billing

  • AnalyticDB for MySQL instance fee: You are charged when you purchase a new AnalyticDB for MySQL instance. For more information, see AnalyticDB for MySQL Pricing.

    Note

    By default, an AnalyticDB for MySQL 3.0 instance is created with a 3-month subscription.

  • DTS fee: You are charged for data synchronization. For more information, see DTS Pricing.

Prerequisites

  • The source database is an ApsaraDB RDS for MySQL database that belongs to your Alibaba Cloud account.

  • If you want to synchronize data to an existing AnalyticDB for MySQL 3.0 instance, you must first add the instance to DMS and then log on to it. For more information, see Register a cloud database.

  • You must have an administrator, DBA, or regular user role.

    Note

    Regular users must have export permissions for the source ApsaraDB RDS for MySQL database and change permissions for the destination AnalyticDB for MySQL 3.0 data warehouse. Change permissions are not required for new instances. For more information about permissions, see Permission management.

Procedure

  1. Log on to the DMS console V5.0.

  2. In the top menu bar, choose Solutions > One-click data warehousing.

  3. In the upper-right corner of the page, click One-click data warehousing.

  4. On the One-click data warehousing tickets page, configure the Data Source and Analytics Engine parameters.

    Configuration item

    Description

    Data Source

    Select the source database to analyze.

    Analytics Engine

    Select an AnalyticDB for MySQL analytics engine:

    • Purchase an ADB instance (Default: 8 cores, 32 GB): By default, an AnalyticDB for MySQL 3.0 instance is created. The billing method is subscription and the subscription duration is 3 months.

      • Region: The region where the cluster resides. The region cannot be changed after purchase. To reduce latency, select a region that is close to your users.

      • Zone: A zone is an independent physical location within a region. Zones in the same region are similar.

      • Virtual Private Cloud (VPC): A VPC is an isolated network environment that provides high security. For more information, see Create a default VPC and vSwitch.

      • vSwitch: Select a vSwitch in the VPC.

      Note

      The system creates a default database account and password and automatically adds the instance to DMS. You can view the new instance and reset its password in the AnalyticDB for MySQL console.

    • Use an existing ADB instance: Search for and select an existing AnalyticDB for MySQL 3.0 instance.

  5. Optional: Click Advanced Configuration and configure the parameters.

    Configuration item

    Description

    Destination Database Name

    The name of the destination database. The following options are available:

    • Automatically create a database with the same name (Default): Creates a database with the same name as the source database in the destination instance.

    • Rename database: Specify a custom name for the destination database.

    Synchronization Types

    The supported options. You can select multiple options.

    • Initial schema synchronization (Default): Synchronizes the table schema to the destination database. An error occurs if a table with the same name exists.

    • Full data (Default): Synchronizes the full data to the destination database.

    • Incremental data: Synchronizes incremental data to the destination database. This option cannot be deselected.

    Synchronization Scope

    The supported options:

    • Full database (Default): Synchronizes all tables to the destination database.

    • Specific tables: Specify the tables to synchronize. You can select multiple tables.

  6. Click Submit Request.

    The system checks the database permissions.

  7. Click Submit for Approval.

    After the request is approved, the system runs the sync task based on the configuration. You can view the synchronization progress in the Data Synchronization section.

What to do next

On the details page of the data analytics ticket, you can perform the following operations: