Create a Kafka data source

Security risks

You can add the public IP address CIDR blocks of OceanBase Data Transmission Service to your whitelist, either automatically or manually. However, this action may create security risks. Using this product, you acknowledge these risks and must take basic security measures. These measures include using complex passwords, restricting open ports for CIDR blocks, using authentication for internal API calls, and regularly reviewing and restricting access from unnecessary IP address CIDR blocks.

OceanBase Data Transmission Service dynamically adjusts whitelists and security groups by adding or deleting entries based on business needs and security risks. Do not use these IP address segments for any business purpose other than for OceanBase Data Transmission Service. Any issues that arise from using these IP address segments for other business purposes are not covered by the OceanBase Data Transmission Service SLA. For more information, see the Add Whitelist documentation.

Limits

Data Transmission Service supports a Kafka data source only as the destination for data synchronization.

Background information

Security is important throughout process, including connectivity tests, link creation, and data transmission. Data Transmission Service leverages the security features of Kafka. It supports data encryption and user authentication to meet most security requirements.

Data Transmission Service supports the following Kafka authentication methods:

• GSSAPI

  Generic Security Services Application Program Interface (GSSAPI) is a framework that provides security services to applications in a generic way. This framework also supports the Kerberos protocol.

• PLAIN

  The PLAIN authentication method is simple. However, it does not support dynamic user changes. It also requires you to configure the username and password in plaintext, which is not secure.

• SCRAM-SHA-256

  The Salted Challenge Response Authentication Mechanism (SCRAM) method addresses the security issues of traditional username and password authentication mechanisms. Kafka supports SCRAM-SHA-256, which can be used with TLS for secure authentication.

  This method supports dynamic user changes. User data is stored in Zookeeper. Before you start a broker, you must communicate with Zookeeper to create a communication user for that broker. However, this method requires you to configure the username and password in plaintext.

• SCRAM-SHA-512

  Kafka supports SCRAM-SHA-512, which can be used with TLS for secure authentication.

Procedure

1. Log on to the OceanBase Management Console .

2. In the navigation pane on the left, you can click Data Transmission > Data Source Management.

3. On the Data Source List page, click Create Data Source in the upper-right corner.

   

4. In the Create Data Source dialog box, set Data Source Type to Kafka.

   

5. Select an Instance Type and configure the required parameters.

   • If you select Alibaba Cloud Kafka Instance, configure the following parameters. For more information, see Java SDK Overview.

     Parameter	Description
     Data source identifier	We recommend using a combination of Chinese characters, numbers, and letters. The name cannot contain spaces and must not exceed 32 characters.
     Cross-account	Data Transmission Service supports data migration and synchronization for instances under different Alibaba Cloud accounts. Select this option as needed. If you select this option, enter the ID of the other Alibaba Cloud account. If you do not have permissions on that account, request authorization first. For more information, see Request cross-account authorization.
     Kafka instance ID	The unique ID of the Kafka instance.
     Endpoint	The list of IP addresses and ports of the Kafka server. The system automatically imports this information. Note When you enter the Kafka endpoint, domain names are not supported. You can only enter IP addresses. If you must use a domain name, configure the domain name in the advertised.listener parameter of Kafka.
     Username	The logon username for Kafka.
     Password	The logon password for Kafka.
     Remarks (Optional)	The remarks for the data source.

   • If you select Self-hosted Kafka instance in a VPC or Public Kafka instance, configure the following parameters.

     Parameter	Description
     Data source identifier	The name can contain a combination of Chinese characters, letters, and numbers. The name must be 32 characters or less and cannot contain spaces.
     Cross-account	Data Transmission Service supports data migration and synchronization for instances under different Alibaba Cloud accounts. Select this option as needed. If you select this option, enter the ID of the other Alibaba Cloud account. If you do not have permissions on that account, request authorization first. For more information, see Request cross-account authorization. Important This parameter is not displayed if you set Instance Type to Public Kafka instance.
     VPC	Select the ID of the VPC from the drop-down list. You can also enter a VPC name to perform a fuzzy search. Important This parameter is displayed only if you set Instance Type to Self-hosted Kafka instance in a VPC.
     VPC deployment/Cross-network deployment	Cross-network deployment means the source and destination data sources are in different networks. These networks can include different VPCs or cloud service providers. As needed, select VPC deployment or Cross-network deployment. From the vSwitch drop-down list, select the vSwitch where all bootstrap server and broker server instances of the Kafka service are located. Also, add the vSwitch CIDR block to the security group whitelist of the current VPC. A vSwitch is a basic network module of a VPC. It connects different cloud resource instances. For more information, see vSwitch overview. Important You can select a deployment mode and a vSwitch only if you set Instance Type to Self-hosted Kafka instance in a VPC. For a cross-network deployment, the static route address is automatically associated based on the first vSwitch you select. The static route address can be an address or CIDR block in another cloud or an on-premises data center VPC.
     Endpoint	Enter the list of IP addresses and ports of the Kafka server.
     Enable SSL	Select whether to enable SSL as needed. If you enable SSL, click Upload File to upload a trusted certificate file with the .jks extension.
     Enable authentication	Select whether to enable authentication as needed. Kafka provides configurations for data encryption and multiple identity authentication mechanisms to ensure the security of user data and services.
     Authentication method	If you enable authentication, select an authentication method. Data Transmission Service supports GSSAPI, PLAIN, SCRAM-SHA-256, and SCRAM-SHA-512.
     KDC server address	Enter the IP address or domain name of the Kerberos Key Distribution Center (KDC) server. Note: This parameter is displayed only if you set Authentication method to GSSAPI.
     User principal	Enter the username. Note: This parameter is displayed only if you set Authentication method to GSSAPI.
     keytab file	Click Upload File to upload a key file with the .keytab extension. Note: This parameter is displayed only if you set Authentication method to GSSAPI.
     Username	The username for data migration or data synchronization. Note: This parameter is not displayed if you set Authentication method to GSSAPI.
     Password	The password for the user that is used for data migration or data synchronization. Note: This parameter is not displayed if you set Authentication method to GSSAPI.
     Remarks (Optional)	The remarks for the data source.

6. Click Test Connection to verify the network connectivity between Data Transmission Service and the data source, and validate the username and password.

7. After a successful connection test, click OK.