Plan and configure resources

更新时间:
复制 MD 格式

DataWorks real-time sync nodes use exclusive resource groups for data integration to synchronize data. This topic describes how to plan and configure these resources.

Background information

  • Resource planning and preparation

    Before you use a data synchronization node to synchronize data, you must purchase an exclusive resource group for data integration and add the resource group to DataWorks for subsequent use.

    For more information about exclusive resource groups for Data Integration, see Exclusive resource groups for Data Integration.

  • Network connectivity

    An exclusive resource group for Data Integration is essentially a group of Elastic Compute Service (ECS) instances. After you purchase an exclusive resource group for Data Integration, it is isolated from other services. You must associate the resource group with a virtual private cloud (VPC) to ensure network connectivity between the resource group and data sources during subsequent data synchronization.

Purchase an exclusive data integration resource group

  1. Log on to the DataWorks console.
  2. Select a region. In the left-side navigation pane, click Resource Group List.
  3. On the Exclusive Resource Groups tab, click Create Exclusive Resource Group.
  4. In the Create Exclusive Resource Group dialog box, click Purchase next to Order No. to go to the purchase page.
  5. On the purchase page, select a Region, Exclusive Resource Type, Number of Resources, and Billing Cycle, and then click Buy Now.
    Note For the Exclusive Resource Type parameter, select Exclusive resource for data integration.
  6. Confirm your order details, select DataWorks Exclusive Resources (Subscription) Service Agreement, and then click Proceed to Pay.

Add an exclusive data integration resource group

  1. On the Exclusive Resource Groups tab of the Resource Group List page, click Create Exclusive Resource Group.
  2. In the Create Exclusive Resource Group dialog box, configure the following parameters.
    ParameterDescription
    Resource Group TypeThe resource group type. Select Exclusive resource for data integration for data synchronization tasks. Exclusive resource for scheduling is for general-purpose task scheduling.
    Resource Group NameA unique name for the resource group within your tenant.
    Note A tenant is a primary account. A primary account can have multiple users (sub-accounts).
    RemarksA brief description of the resource group.
    Order No.Select the order for the purchased exclusive resource group. If you have not purchased one, click Purchase to go to the purchase page.
  3. After configuring the parameters, click OK.
    Note The exclusive resource group may take up to 20 minutes to initialize. Wait until its status changes to Running.

    For more information about migrating data with an exclusive resource group for data integration, see Migrate data from a self-managed MySQL database on an ECS instance to MaxCompute.

Network configuration

Exclusive resource groups are deployed in a VPC managed by DataWorks and are isolated from other network environments. To enable data access, you must bind the resource group to a VPC that is connected to your data source.

  1. Click Network Settings for the required resource group.
    Note Before you bind a VPC, you must grant RAM permissions to allow DataWorks to access your cloud resources.
  2. Bind a VPC.
    1. On the VPC Binding tab, click Add Binding. In the Add VPC Binding dialog box, configure the parameters. The configuration varies based on your network environment.
      The following table describes the parameters.
      ParameterConfiguration (data source and resource group in the same VPC)Configuration (data source and resource group not in the same VPC)
      VPCIf your data source is in an Alibaba Cloud VPC, select that VPC.If your data source is not in the same VPC as the exclusive resource group, for example, if the data source is not in an Alibaba Cloud VPC, or you need to deploy them in separate VPCs, click Create VPC to create a new VPC for the resource group. After creation, select the new VPC.
      vSwitchWhen you select the VPC where the data source resides, select the vSwitch to which your data source is connected.If you select a different VPC or if no vSwitch is available, click Create vSwitch to create a new vSwitch for the resource group. After creation, select the new vSwitch.
      Note In this case, you must also configure vSwitch routes to ensure network connectivity between the resource group and the data source.
      Security GroupA security group controls the inbound and outbound traffic for the instances in the exclusive resource group. You can select an existing security group or click Create Security Group to create one. For more information about how to create a security group, see Add a security group rule.
    2. Click OK to bind the VPC.
  3. Optional: Configure hosts.
    If you access your data source by a domain name instead of a direct IP address, you must configure the host settings. Otherwise, the connectivity test fails when you add the data source by using its domain name.
    1. Click the Host Configuration tab. In the upper-left corner, click Add. In the Add Domain Name and Host dialog box, configure the parameters. The following table describes the parameters.
      ParameterDescription
      IP AddressThe IP address of the data source.
      Host Domain NameThe domain name that is used to access the data source. To add multiple domain names, enter each one on a new line.
      Note The domain name can contain digits, letters, hyphens (-), and periods (.). It must start with a letter and end with a letter or a digit.
    2. To add mappings for multiple IP addresses, click Add again.
      Note
      • The domain name in a new host configuration cannot be the same as one in an existing configuration.
      • An IP address can be mapped to multiple domain names, but a domain name can be mapped to only one IP address.
  4. Optional: Configure DNS.
    If your data source is accessed through a domain name resolved by a private DNS server (such as a load balancing domain), you must configure DNS settings. Otherwise, the connectivity test fails when you add the data source by using its domain name.
    Note If you configure both host and DNS settings for the same domain name, the host settings take precedence.
    1. Click the DNS Configuration tab. In the lower-left corner of the page, click Add. Configure the DNS parameters and then click Save. The following table describes the parameters.
      ParameterDescription
      Domain(Optional) The top-level domain shared by your data sources.

      For example, if your data sources are domain1.example.com and domain2.example.com, you can set this parameter to example.com.

      Note The domain name can contain digits, letters, hyphens (-), and periods (.). It must start with a letter and end with a letter or a digit.
      NameServerThe IP address of the DNS server for your data source. To add multiple DNS servers, enter each one on a new line.
    2. To change a DNS configuration, click Modify in the lower-right corner.
After configuring the network settings, you must add the EIPs of the resource group and the IP addresses of its elastic network interfaces to your database's allowlist.

Next steps

After planning and configuring your resources, the next step is to configure your data sources. This requires preparing network connections and account permissions for both the source and destination before you create a real-time sync node. Currently, you can synchronize data to AnalyticDB for MySQL only from PolarDB or MySQL data sources. For configuration details, see Configure a PolarDB data source or Configure data sources for data synchronization from MySQL.