Create a custom offline source type
Create custom offline data sources in Dataphin for data synchronization across different business scenarios.
Prerequisites
-
RDBMS databases: You have obtained the database driver files.
-
Non-RDBMS databases: You have obtained the datasource connection JSON code and the read/write plug-ins.
Background information
RDBMS databases (MySQL, Oracle, SQL Server, etc.) share a common connection method: a database driver plus configuration items such as the connection address, username, and password. Dataphin provides default configuration items for RDBMS databases, so you only need to prepare the driver.
Non-RDBMS databases have varied connection methods. Define the connection configuration items in JSON and upload the read/write plug-in driver packages to create a non-RDBMS custom source type.
Auto-generated integration components
After you create an offline custom source type, integration components are automatically generated in the integration pipeline component library for integration development.
-
RDBMS source types: Input and output components are generated in the open directory of the component library.
-
Non-RDBMS source types: Read/write plug-in components are generated in the open directory based on the plug-ins you uploaded.
Create an RDBMS source type
-
On the Dataphin home page, select Management Center > Datasource Management .
-
In the left-side navigation pane, select Datasource > Custom Source Type .
-
On the Custom Source Type page, click Create Custom Source Type and select Create Offline Custom Source Type from the drop-down list.
-
On the Create Offline Custom Source Type page, configure the following parameters.
Parameter
Description
Basic Configuration
Type
Select RDBMS Database.
Name
Name of the custom source type.
Supports Chinese characters, letters, digits, and underscores (_). Maximum 64 characters.
Type Code
Backend identifier. Cannot be changed after creation.
Letters, digits, and underscores (_). Cannot start with a digit.
Resource Configuration
Driver Name
Database driver class name.
Letters, digits, underscores (_), and half-width periods (.). Maximum 64 characters. Example: com.mysql.jdbc.Driver.
Driver Files
Upload the database driver files.
NoteOnly .jar files are supported. Maximum file size: 200MB.
Description
Description
A brief description of the custom source type. Maximum 128 characters.
-
Click Create .
Create a non-RDBMS source type
-
On the Create Offline Custom Source Type page, configure the following parameters.
Parameter
Description
Basic Configuration
Type
Select Other Database.
Name
Name of the custom source type.
Supports Chinese characters, letters, digits, and underscores (_). Maximum 64 characters.
Type Code
Backend identifier. Cannot be changed after creation.
Letters, digits, and underscores (_). Cannot start with a digit.
Datasource JSON
JSON code that defines the datasource connection configuration. Example:
[ { "columnName": "url", "columnType": "NORMAL", "text": { "zh_CN": "链接地址", "en_US": "address", "zh_TW": "繁体" }, "placeholder": { "zh_CN": "请输入链接地址", "en_US": "input address", "zh_TW": "繁体" } }, { "columnName": "username", "columnType": "NORMAL", "text": { "zh_CN": "用户名", "en_US": "username", "zh_TW": "繁体" }, "placeholder": { "zh_CN": "请输入用户名", "en_US": "input username", "zh_TW": "繁体" } }, { "columnName": "password", "columnType": "ENCRYPT", "text": { "zh_CN": "密码", "en_US": "password", "zh_TW": "繁体" }, "placeholder": { "zh_CN": "请输入密码", "en_US": "input password", "zh_TW": "繁体" } } ]Resource Configuration
Read/write Plug-in
Select the plug-in type, specify the ClassName, and upload the plug-in file.
-
Only .jar files are supported. Maximum file size: 200MB.
-
At least one read or write plug-in is required.
-
A read plug-in generates a read component; a write plug-in generates a write component.
Description
Description
A brief description of the custom source type. Maximum 128 characters.
-
-
Click Create .
What to do next
After the custom source type is created, the system generates datasource configuration items for this type. You can then create a custom datasource to integrate it into Dataphin.