DocsICP FilingConsole
Documentation

GetDIJob

Updated at:
Copy as MD
Product
My Favorites

Queries the configuration of a data integration (DI) synchronization job.

Operation description

This operation requires DataWorks Basic Edition or later.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

No authorization for this operation. If you encounter issues with this operation, contact technical support.

Request parameters

Parameter

Type

Required

Description

Example

Id

integer

No

The ID of the DI job.

11588

WithDetails

boolean

No

Whether to return detailed configuration (TransformationRules, TableMappings, and JobSettings). Default: true.

true

ProjectId

integer

No

The DataWorks workspace ID. Call ListProjects to obtain this ID.

10000

DIJobIddeprecated

integer

No

This field is deprecated. Use the Id field instead.

11588

Response elements

Element

Type

Description

Example

object

The data returned.

RequestId

string

The request ID. Used for log tracing and troubleshooting.

C99E2BE6-9DEA-5C2E-8F51-1DDCFEADE490
PagingInfo

object

The pagination information of the job.

Id

integer

The job ID.

32601

Description

string

The description of the job.

description
DestinationDataSourceSettings

array<object>

The settings for the destination data source.

object

The properties for a destination data source.

DataSourceName

string

The name of the destination data source.

dw_mysql

DestinationDataSourceType

string

The type of the destination data source. Valid values: Hologres, OSS-HDFS, OSS, MaxCompute, LogHub, StarRocks, DataHub, AnalyticDB for MySQL, Kafka, and Hive.

Hologres

JobName

string

The name of the job.

imp_ods_dms_det_dealer_info_df
JobSettings

object

The job settings.

ChannelSettings

string

Channel-specific settings. Supported channels: Holo2Holo (Hologres to Hologres) and Holo2Kafka (Hologres to Kafka).

  1. Holo2Kafka

  • Example: {"destinationChannelSettings":{"kafkaClientProperties":[{"key":"linger.ms","value":"100"}],"keyColumns":["col3"],"writeMode":"canal"}}

  • kafkaClientProperties: The parameters for the Kafka producer, used when writing data to Kafka.

  • keyColumns: The columns whose values are used as the key for Kafka records.

  • writeMode: The format for writing data to Kafka. Valid values: json and canal.

  1. Holo2Holo

  • Example: {"destinationChannelSettings":{"conflictMode":"replace","dynamicColumnAction":"replay","writeMode":"replay"}}

  • conflictMode: The conflict handling policy for writing data to Hologres. Valid values: replace (overwrite) and ignore (ignore).

  • writeMode: The method for writing data to Hologres. Valid values: replay and insert.

  • dynamicColumnAction: The method for handling dynamic columns when writing data to Hologres. Valid values: replay, insert, and ignore.

{"structInfo":"MANAGED","storageType":"TEXTFILE","writeMode":"APPEND","partitionColumns":[{"columnName":"pt","columnType":"STRING","comment":""}],"fieldDelimiter":""}
ColumnDataTypeSettings

array<object>

The column data type mappings.

object

A column data type mapping.

DestinationDataType

string

The data type in the destination, such as bigint, boolean, string, text, datetime, timestamp, decimal, and binary. Data types vary depending on the data source.

text

SourceDataType

string

The data type in the source, such as bigint, boolean, string, text, datetime, timestamp, decimal, and binary. Data types vary depending on the data source.

bigint
CycleScheduleSettings

object

The settings for periodic scheduling.

CycleMigrationType

string

The synchronization type for periodic scheduling. Valid values:

  • Full: full

  • OfflineIncremental: offline incremental

Full

ScheduleParameters

string

The scheduling parameters.

bizdate=$bizdate

DdlHandlingSettings

array<object>

DDL message handling settings. Each element maps a DDL type to a handling rule.

object

The rule for handling a DDL message type.

Action

string

The handling action. Valid values:

  • Ignore: Ignores the DDL message.

  • Critical: Reports an error.

  • Normal: Processes the DDL message.

Ignore

Type

string

The DDL message type. Valid values:

  • RenameColumn

  • ModifyColumn

  • CreateTable

  • TruncateTable

  • DropTable

CreateTable
RuntimeSettings

array<object>

The runtime settings.

object

A runtime setting.

Name

string

The name of the setting. Valid values:

  • src.offline.datasource.max.connection: The maximum number of connections to the source for an offline batch job.

  • dst.offline.truncate: Whether to truncate the destination table before the offline batch job starts.

  • runtime.offline.speed.limit.enable: Whether to enable throttling for an offline batch job.

  • runtime.offline.concurrent: The concurrency level for an offline batch synchronization job.

  • runtime.enable.auto.create.schema: Whether to automatically create a schema at the destination.

  • runtime.realtime.concurrent: The concurrency level for a real-time job.

  • runtime.realtime.failover.minute.dataxcdc: The wait duration (in minutes) before restarting a failed instance.

  • runtime.realtime.failover.times.dataxcdc: The maximum number of retries for a failed instance.

runtime.offline.concurrent

Value

string

The value of the setting.

1

MigrationType

string

The synchronization type. Valid values:

  • FullAndRealtimeIncremental: one-time full synchronization and real-time incremental synchronization (for an entire database).

  • RealtimeIncremental: real-time incremental synchronization (for a single table).

  • Full: one-time full synchronization (for an entire database).

  • OfflineIncremental: offline incremental synchronization (for an entire database).

  • FullAndOfflineIncremental: one-time full synchronization and offline incremental synchronization (for an entire database).

FullAndRealtimeIncremental

JobType

string

The job type.

  • DatabaseRealtimeMigration: real-time synchronization of multiple tables from multiple source databases. This type supports full, incremental, or both full and incremental synchronization.

  • DatabaseOfflineMigration: batch synchronization of multiple tables from multiple source databases. This type supports full, incremental, or both full and incremental synchronization.

  • SingleTableRealtimeMigration: real-time synchronization of a single source table.

DatabaseRealtimeMigration

ProjectId

integer

The DataWorks workspace ID. Available in the DataWorks console workspace settings.

98330
ResourceSettings

object

The resource settings.

OfflineResourceSettings

object

Resource settings for offline synchronization.

RequestedCu

number

CUs allocated from the DI resource group for offline synchronization.

2.0

ResourceGroupIdentifier

string

DI resource group name for offline synchronization.

di_resourcegroup_v1
RealtimeResourceSettings

object

Resource settings for real-time synchronization.

RequestedCu

number

CUs allocated from the DI resource group for real-time synchronization.

2.0

ResourceGroupIdentifier

string

DI resource group name for real-time synchronization.

di_resourcegroup_v1
ScheduleResourceSettings

object

The scheduling resource settings.

RequestedCu

number

CUs allocated from the scheduling resource group for offline scheduling.

2.0

ResourceGroupIdentifier

string

Scheduling resource group name for offline scheduling.

schedual_resourcegroup_v1
SourceDataSourceSettings

array<object>

The settings for the source data source.

array<object>

The settings for a source data source.

DataSourceName

string

The name of the data source.

dw_mysql
DataSourceProperties

object

The properties of the data source.

Encoding

string

The encoding of the database.

UTF-8

Timezone

string

The time zone.

GMT+8

SourceDataSourceType

string

The type of the source data source. Valid values: PolarDB, MySQL, Kafka, LogHub, Hologres, Oracle, OceanBase, MongoDB, RedShift, Hive, SQLServer, Doris, and ClickHouse.

Mysql
TableMappings

array<object>

Object transformation mappings. Each element defines source object selection rules and the transformation rules applied to them.

Note

[ { "SourceObjectSelectionRules":[ { "ObjectType":"Database", "Action":"Include", "ExpressionType":"Exact", "Expression":"biz_db" }, { "ObjectType":"Schema", "Action":"Include", "ExpressionType":"Exact", "Expression":"s1" }, { "ObjectType":"Table", "Action":"Include", "ExpressionType":"Exact", "Expression":"table1" } ], "TransformationRuleNames":[ { "RuleName":"my_database_rename_rule", "RuleActionType":"Rename", "RuleTargetType":"Schema" } ] } ]

array<object>

A mapping that defines the source objects and the transformation rules to apply.

SourceObjectSelectionRules

array<object>

Rules that select source objects for synchronization. Combined rules select a single table.

object

Selects source objects (database, schema, or table) for synchronization.

Action

string

The selection action. Valid values: Include and Exclude.

Include

Expression

string

The expression.

mysql_table_1

ExpressionType

string

The expression type. Valid values: Exact and Regex.

Exact

ObjectType

string

The object type. Valid values:

  • Table

  • Schema

  • Database

Table
TransformationRules

array<object>

An array of object transformation rule definitions.

object

A transformation rule that is applied to the source object.

RuleName

string

The name of the rule. The rule name must be unique for a specific action type (RuleActionType) and target type (RuleTargetType).

rename_rule_1

RuleActionType

string

The action type. Valid values:

  • DefinePrimaryKey

  • Rename

  • AddColumn

  • HandleDml

AddColumn

RuleTargetType

string

The target object type of the action. Valid values:

  • Table

  • Schema

  • Database

Table
TransformationRules

array<object>

A list of definitions for object transformation rules.

Note

[ { "RuleName":"my_database_rename_rule", "RuleActionType":"Rename", "RuleTargetType":"Schema", "RuleExpression":"{\"expression\":\"${srcDatasoureName}_${srcDatabaseName}\"}" } ]

object

A definition for an object transformation rule.

RuleActionType

string

The action type. Valid values:

  • DefinePrimaryKey

  • Rename

  • AddColumn

  • HandleDml

  • DefineIncrementalCondition

  • DefineCycleScheduleSettings

  • DefinePartitionKey

Rename

RuleExpression

string

The rule expression, in JSON string format.

  1. Rename rule (Rename)

  • Example: {"expression":"${srcDatasourceName}_${srcDatabaseName}_0922"}

  • expression: The expression for the rename transformation rule. The expression supports the following variables: ${srcDatasourceName} (source data source name), ${srcDatabaseName} (source database name), and ${srcTableName} (source table name).

  1. Add column rule (AddColumn)

  • Example: {"columns":[{"columnName":"my_add_column","columnValueType":"Constant","columnValue":"123"}]}

  • If you do not specify this parameter, no columns are added or copied by default.

  • columnName: The name of the column to add.

  • columnValueType: The value type of the added column. Valid values: Constant and Variable.

  • columnValue: The value of the added column. If columnValueType is Constant, the value is a custom string constant. If columnValueType is Variable, the value is a built-in variable. Valid built-in variables: EXECUTE_TIME (execution time, Long), DB_NAME_SRC (source database name, String), DATASOURCE_NAME_SRC (source data source name, String), TABLE_NAME_SRC (source table name, String), DB_NAME_DEST (destination database name, String), DATASOURCE_NAME_DEST (destination data source name, String), TABLE_NAME_DEST (destination table name, String), and DB_NAME_SRC_TRANSED (converted database name, String).

  1. Define primary key rule (DefinePrimaryKey)

  • Example: {"columns":["ukcolumn1","ukcolumn2"]}

  • By default, the primary key columns from the source table are used.

  • If the destination table already exists, the data integration system does not modify the table schema. If the specified primary key columns are not in the destination table, the job fails to start.

  • If the destination table is automatically created, the data integration system automatically creates the table schema that includes the defined primary key columns. If the specified primary key columns are not in the destination table, the job fails to start.

  1. DML handling rule (HandleDml)

  • Example: {"dmlPolicies":[{"dmlType":"Delete","dmlAction":"Filter","filterCondition":"id > 1"}]}

  • If you do not specify this parameter, the default value Normal is used for Insert, Update, and Delete operations.

  • dmlType: The DML operation type. Valid values: Insert, Update, and Delete.

  • dmlAction: The DML handling policy. Valid values: Normal (process normally), Ignore (ignore), Filter (process conditionally, used when dmlType is Update or Delete), and LogicalDelete (logically delete).

  • filterCondition: The DML filter condition. This parameter is used when dmlAction is Filter.

  1. Define incremental condition rule (DefineIncrementalCondition)

  • Example: {"where":"id > 0"}

  • Specifies the filter condition for incremental synchronization.

  1. Define cycle schedule settings rule (DefineCycleScheduleSettings)

  • Example: {"cronExpress":" * * * * * *", "cycleType":"1"}

  • Specifies the scheduling parameters for a periodic job.

  1. Define partition key rule (DefinePartitionKey)

  • Example: {"columns":["id"]}

  • Specifies the partition key.

{"expression":"${srcDatasoureName}_${srcDatabaseName}"}

RuleName

string

The name of the rule. The rule name must be unique for a specific action type (RuleActionType) and target type (RuleTargetType).

rename_rule_1

RuleTargetType

string

The target object type of the action. Valid values:

  • Table

  • Schema

  • Database

Table

JobStatus

string

The status of the job. Valid values:

  • Finished: The job is complete.

  • Failed: The job failed.

  • Running: The job is running.

  • Initialized: The job is initialized but has not started.

  • Stopping: The job is being stopped.

  • Stop: The job is stopped.

Running

DIJobIddeprecated

string

This field is deprecated. Use the Id field instead.

32601

Owner

string

Examples

Success response

JSON format

{
  "RequestId": "C99E2BE6-9DEA-5C2E-8F51-1DDCFEADE490",
  "PagingInfo": {
    "Id": 32601,
    "Description": "description",
    "DestinationDataSourceSettings": [
      {
        "DataSourceName": "dw_mysql"
      }
    ],
    "DestinationDataSourceType": "Hologres",
    "JobName": "imp_ods_dms_det_dealer_info_df",
    "JobSettings": {
      "ChannelSettings": "{\"structInfo\":\"MANAGED\",\"storageType\":\"TEXTFILE\",\"writeMode\":\"APPEND\",\"partitionColumns\":[{\"columnName\":\"pt\",\"columnType\":\"STRING\",\"comment\":\"\"}],\"fieldDelimiter\":\"\"}",
      "ColumnDataTypeSettings": [
        {
          "DestinationDataType": "text",
          "SourceDataType": "bigint"
        }
      ],
      "CycleScheduleSettings": {
        "CycleMigrationType": "Full",
        "ScheduleParameters": "bizdate=$bizdate\n"
      },
      "DdlHandlingSettings": [
        {
          "Action": "Ignore",
          "Type": "CreateTable"
        }
      ],
      "RuntimeSettings": [
        {
          "Name": "runtime.offline.concurrent\n",
          "Value": "1"
        }
      ]
    },
    "MigrationType": "FullAndRealtimeIncremental",
    "JobType": "DatabaseRealtimeMigration",
    "ProjectId": 98330,
    "ResourceSettings": {
      "OfflineResourceSettings": {
        "RequestedCu": 2,
        "ResourceGroupIdentifier": "di_resourcegroup_v1"
      },
      "RealtimeResourceSettings": {
        "RequestedCu": 2,
        "ResourceGroupIdentifier": "di_resourcegroup_v1"
      },
      "ScheduleResourceSettings": {
        "RequestedCu": 2,
        "ResourceGroupIdentifier": "schedual_resourcegroup_v1"
      }
    },
    "SourceDataSourceSettings": [
      {
        "DataSourceName": "dw_mysql",
        "DataSourceProperties": {
          "Encoding": "UTF-8\n",
          "Timezone": "GMT+8\n"
        }
      }
    ],
    "SourceDataSourceType": "Mysql",
    "TableMappings": [
      {
        "SourceObjectSelectionRules": [
          {
            "Action": "Include",
            "Expression": "mysql_table_1",
            "ExpressionType": "Exact",
            "ObjectType": "Table"
          }
        ],
        "TransformationRules": [
          {
            "RuleName": "rename_rule_1",
            "RuleActionType": "AddColumn",
            "RuleTargetType": "Table"
          }
        ]
      }
    ],
    "TransformationRules": [
      {
        "RuleActionType": "Rename",
        "RuleExpression": "{\"expression\":\"${srcDatasoureName}_${srcDatabaseName}\"}",
        "RuleName": "rename_rule_1",
        "RuleTargetType": "Table"
      }
    ],
    "JobStatus": "Running",
    "DIJobId": "32601",
    "Owner": ""
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.