DocsICP FilingConsole
Documentation
Home Page Dataphin Intelligent Data construction and governance Dataphin-sharing mode (fully managed version) User Guide Data ingestion Data integration Offline Integration Widget library development Input component Configure the Amazon RDS for SQL Server Input Component

Configure the Amazon RDS for SQL Server Input Component

Updated at:
Copy as MD
Product
My Favorites

The Amazon RDS for SQL Server input component reads data from an Amazon RDS for SQL Server data source. To sync data from an Amazon RDS for SQL Server data source to another data source, first configure the Amazon RDS for SQL Server input component to read from the source. Then configure the destination data source for the sync.

Prerequisites

Procedure

  1. In the top menu bar on the Dataphin homepage, choose Develop > Data Integration.

  2. In the top menu bar of the Integration page, select a Project. In Dev-Prod mode, also select an environment.

  3. In the navigation pane on the left, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline you want to develop. The configuration page for that pipeline opens.

  4. In the upper-right corner of the page, click Component Library to open the Component Library panel.

  5. In the navigation pane on the left of the Component Library panel, click Input. In the list of input components on the right, find the Amazon RDS for SQL Server component and drag it onto the canvas.

  6. Click the image icon in the Amazon RDS for SQL Server input widget to open the Amazon RDS for SQL Server Input Configuration dialog box.

  7. In the Amazon RDS for SQL Server Input Configuration dialog box, configure the following parameters.

    Parameter

    Description

    Step Name

    The name of the Amazon RDS for SQL Server input component. Dataphin generates a step name automatically. You can change it as needed. Naming rules:

    • Use only Chinese characters, letters, underscores (_), and digits.

    • Maximum length is 64 characters.

    Datasource

    The drop-down list shows all Amazon RDS for SQL Server data sources. This includes sources you have sync-read permission for and those you do not. Click the image icon to copy the current data source name.

    Schema (optional)

    Select the schema where your table resides. This supports cross-schema table selection. If you do not specify a schema, the default is the one configured in the data source.

    Source Table Count

    Select how many tables to read from. Options are Single Table and Multiple Tables:

    • Single Table: Use this when syncing business data from one source table to one destination table.

    • Multiple Tables: Use this when syncing business data from multiple source tables to one destination table. When writing data from multiple tables into one destination table, the system uses the union algorithm.

      For more details about union, see INTERSECT, UNION, and EXCEPT.

    Table Matching Method

    Choose between Generic Rules and Database Regex.

    Note

    This option is available only when you select Multiple Tables for Source Table Count.

    Table

    Select the source table:

    • If you selected Single Table for Source Table Count, search by entering a keyword in the table name field. Or enter the exact table name and click Exact Match. After selecting a table, the system automatically checks its status. Click the image icon to copy the selected table name.

    • If you selected Multiple Tables for Source Table Count, use the matching method you chose to add tables with expressions.

      • If you chose Generic Rules: Enter an expression in the field to match tables with identical structure. Supported formats include enumeration, regex-like patterns, and combinations. Example: table_[001-100];table_102;.

      • If you chose Database Regex: Enter a regex pattern supported by your database. The system matches tables in the destination database using this pattern. At runtime, the task matches new tables dynamically based on this regex.

      After entering the expression, click Exact Match to view matched tables in the Confirm Match Details dialog box.

    Split Key (optional)

    The system partitions data based on the split key column you specify. Use this with concurrency settings to enable concurrent reads. You can use any column from the source table as the split key. For best performance, use a primary key or indexed column.

    Important

    If you select a date-time type, the system calculates min and max values and splits the full time range across concurrency threads. This split is not guaranteed to be even.

    Batch Read Size (optional)

    Number of records to read at once. Configure a batch size—such as 1024 records—to reduce round trips to the source database. This improves I/O efficiency and reduces network latency.

    Input Filter (optional)

    Enter filter conditions for input fields. Example: ds=${bizdate}. Use Input Filter for:

    • A fixed subset of data.

    • Parameter-based filtering.

    Hint (optional)

    • Table hints influence how the query optimizer accesses tables or views. Add hints to your query statement to force the database management system to use a specific execution plan instead of relying on default optimization.

    • Add hints after the table name in the FROM clause, inside parentheses. Separate multiple hints with commas. Example:

      SELECT * FROM MyTable WITH (INDEX(IX_MyIndex), NOLOCK)

    Output Fields

    This section lists all fields from the selected table and filters. You can perform these actions:

    • Field Management: Remove fields you do not need downstream:

      • Remove One Field: Click the sgaga icon in the Actions column to delete a single field.

      • Batch field deletion scenario: To delete many fields, you can click Field Management. In the Field Management dialog box, select multiple fields, click the image left-moving icon to move the selected input fields to the unselected input fields, and click OK to complete batch field deletion.

        image..png

    • Bulk Add: Click Bulk Add to configure fields in JSON, TEXT, or DDL format.

      Note

      After clicking OK, the bulk-add operation overwrites existing field configurations.

      • JSON format example:

        // Example:
[
    {
        "index": 1,
        "name": "Id",
        "type": "int(10)",
        "mapType": "Long",
        "comment": ""
    },
    {
        "index": 2,
        "name": "Data",
        "type": "nvarchar(100)",
        "mapType": "String",
        "comment": ""
    }
]
        Note

        index is the column number. name is the field name after import. type is the field type after import. Example: "index":3,"name":"user_id","type":"String" imports column 4 as user_id with type String.

      • TEXT format example:

        // Example:
1,Id,int(10),Long,
2,Data,nvarchar(100),String,
3,RowVersion,timestamp,Bytes,

        • Row delimiter separates field entries. Default is line feed (\n). Also supports semicolon (;) and period (.).

        • Column delimiter separates field names and types. Default is comma (,). Also supports','. Field type is optional and defaults to','.

      • DDL format example:

        CREATE TABLE tablename (
	user_id serial,
	username VARCHAR(50),
	password VARCHAR(50),
	email VARCHAR (255),
	created_on TIMESTAMP,
);

    • Add Output Field: Click + Add Output Field. Fill in Column, Type, Comment, and select a Mapping Type. Click the image icon to save the row.

  8. Click OK to finish configuring the Amazon RDS for SQL Server input component.

Previous：Configure the Amazon RDS for MySQL input componentNext：Configure the Amazon RDS for Oracle Input Component