DescribeStreamingJob - 查询实时数据同步任务

查询实时数据同步服务。

调试

您可以在OpenAPI Explorer中直接运行该接口,免去您计算签名的困扰。运行成功后,OpenAPI Explorer可以自动生成SDK代码示例。

授权信息

下表是API对应的授权信息,可以在RAM权限策略语句的Action元素中使用,用来给RAM用户或RAM角色授予调用此API的权限。具体说明如下:

  • 操作:是指具体的权限点。
  • 访问级别:是指每个操作的访问级别,取值为写入(Write)、读取(Read)或列出(List)。
  • 资源类型:是指操作中支持授权的资源类型。具体说明如下:
    • 对于必选的资源类型,用背景高亮的方式表示。
    • 对于不支持资源级授权的操作,用全部资源表示。
  • 条件关键字:是指云产品自身定义的条件关键字。
  • 关联操作:是指成功执行操作所需要的其他权限。操作者必须同时具备关联操作的权限,操作才能成功。
操作访问级别资源类型条件关键字关联操作
gpdb:DescribeStreamingJobdelete
  • 全部资源
    *

请求参数

名称类型必填描述示例值
RegionIdstring

地域 ID。

说明 您可以调用 DescribeRegions 接口查看可用的地域 ID。
cn-beijing
JobIdinteger

任务 ID。

1
DBInstanceIdstring

实例 ID。

gp-bp10g78o9807yv9h3

返回参数

名称类型描述示例值
object
RequestIdstring

请求 ID。

B4CAF581-2AC7-41AD-8940-D56DF7AADF5B
JobIdstring

任务 ID。

1
JobNamestring

任务名称。

test-job
JobDescriptionstring

任务描述。

test_job
DataSourceIdstring

数据源 ID。

2
SrcColumnsarray

源端字段列表。

srcColumnsstring

源端数据字段键值。

src_column_1
DestColumnsarray

目标数据表映射字段列表。

destColumnsstring

目标表里面的映射字段键值。

dest_column_1
MatchColumnsarray

匹配列,通常为目标表的所有主键列,此配置内所有列的值相同即视为两行数据为重复数据。

matchColumnsstring

匹配字段键值。

column_1
UpdateColumnsarray

更新列,通常为目标表的所有非主键列,当通过 MatchColumns 确定数据重复时,通过更新 UpdateColumns 列的值,产生使新数据覆盖旧数据的效果。

updateColumnsstring

更新字段键值。

column_1
Accountstring

目标数据库账号。

test-account
Passwordstring

目标数据库密码。

pwd123
DestDatabasestring

目标数据库名称。

dest-db
DestSchemastring

目标命名空间。

dest-schema
DestTablestring

目标表名称。

dest-table
WriteModestring

写入模式。

INSERT/UPDATE/MERGE
Modestring

配置模式

  1. basic 模式需要指定一些配置字段
  2. professional 支持提交 yaml 文件
basic/professional
JobConfigstring

任务配置文件。

DATABASE: adbpgss_test USER: adbpgss_test PASSWORD: adbpgssTest HOST: gp-xxx-master.gpdb.rds-aliyun-pre.rds.aliyuncs.com PORT: 5432 KAFKA: INPUT: SOURCE: BROKERS: broker1:9092,broker2:9092,broker3:9092 TOPIC: testtopic FALLBACK_OFFSET: earliest KEY: COLUMNS: - NAME: customer_id TYPE: int FORMAT: delimited DELIMITED_OPTION: DELIMITER: '|' VALUE: COLUMNS: - TYPE: integer NAME: l_orderkey - TYPE: integer NAME: l_partkey - TYPE: integer NAME: l_suppkey - TYPE: integer NAME: l_linenumber - TYPE: decimal NAME: l_quantity - TYPE: decimal NAME: l_extendedprice - TYPE: decimal NAME: l_discount - TYPE: decimal NAME: l_tax - TYPE: char NAME: l_returnflag - TYPE: char NAME: l_linestatus - TYPE: date NAME: l_shipdate - TYPE: date NAME: l_commitdate - TYPE: date NAME: l_receiptdate - TYPE: text NAME: l_shipinstruct - TYPE: text NAME: l_shipmode - TYPE: text NAME: l_comment FORMAT: delimited DELIMITED_OPTION: DELIMITER: '|' ERROR_LIMIT: 10 OUTPUT: SCHEMA: adbpgss_test TABLE: write_with_insert_plaintext MODE: MERGE MATCH_COLUMNS: - l_orderkey - l_partkey - l_suppkey UPDATE_COLUMNS: - l_linenumber - l_quantity - l_extendedprice - l_discount - l_tax - l_returnflag - l_linestatus - l_shipdate - l_commitdate - l_receiptdate - l_shipinstruct - l_shipmode - l_comment MAPPING: - EXPRESSION: l_orderkey NAME: l_orderkey - EXPRESSION: l_partkey NAME: l_partkey - EXPRESSION: l_suppkey NAME: l_suppkey - EXPRESSION: l_linenumber NAME: l_linenumber - EXPRESSION: l_quantity NAME: l_quantity - EXPRESSION: l_extendedprice NAME: l_extendedprice - EXPRESSION: l_discount NAME: l_discount - EXPRESSION: l_tax NAME: l_tax - EXPRESSION: l_returnflag NAME: l_returnflag - EXPRESSION: l_linestatus NAME: l_linestatus - EXPRESSION: l_shipdate NAME: l_shipdate - EXPRESSION: l_commitdate NAME: l_commitdate - EXPRESSION: l_receiptdate NAME: l_receiptdate - EXPRESSION: l_shipinstruct NAME: l_shipinstruct - EXPRESSION: l_shipmode NAME: l_shipmode - EXPRESSION: l_comment NAME: l_comment COMMIT: MAX_ROW: 1000 MINIMAL_INTERVAL: 1000 CONSISTENCY: ATLEAST POLL: BATCHSIZE: 1000 TIMEOUT: 1000 PROPERTIES: group.id: testgroup
Statusstring

服务状态,取值:

  • Init:初始化中

  • Running:运行中

  • Exception:异常

  • Paused:暂停

Running
ErrorMessagestring

服务状态信息,例如异常情况,则显示异常原因。正常 Running 状态下空值。

""
CreateTimestring

创建时间。

2019-09-08T16:00:00Z
ModifyTimestring

最后修改时间。

2019-09-08T17:00:00Z
GroupNamestring

kafka 分组名称

test_group
Consistencystring

投递保证。

ATLEAST / EXACTLY
ErrorLimitCountinteger

kafka 中的数据与 adbpg 目标表不匹配造时会造成写入失败,此值为允许错误行的行数,超出后会造成任务失败。

5
FallbackOffsetstring

FallbackOffset 回退点位,回退偏移量

  • FallbackOffset 参数定义了当 consumer 没有请求需要消费的位点或者请求的消费的位点超出当前 kafka 集群的记录的位点信息的时候的行为。可以选择从 earliest(最新)或者 latest(最晚)位点开始消费。
EARLIEST / LATEST
DataSourceNamestring

数据源名称。

test_kafka

示例

正常返回示例

JSON格式

{
  "RequestId": "B4CAF581-2AC7-41AD-8940-D56DF7AADF5B",
  "JobId": "1",
  "JobName": "test-job",
  "JobDescription": "test_job",
  "DataSourceId": "2",
  "SrcColumns": [
    "src_column_1"
  ],
  "DestColumns": [
    "dest_column_1"
  ],
  "MatchColumns": [
    "column_1"
  ],
  "UpdateColumns": [
    "column_1"
  ],
  "Account": "test-account",
  "Password": "pwd123",
  "DestDatabase": "dest-db",
  "DestSchema": "dest-schema",
  "DestTable": "dest-table",
  "WriteMode": "INSERT/UPDATE/MERGE",
  "Mode": "basic/professional",
  "JobConfig": "DATABASE: adbpgss_test\nUSER: adbpgss_test\nPASSWORD: adbpgssTest\nHOST: gp-xxx-master.gpdb.rds-aliyun-pre.rds.aliyuncs.com\nPORT: 5432\nKAFKA:\n  INPUT:\n    SOURCE:\n      BROKERS: broker1:9092,broker2:9092,broker3:9092\n      TOPIC: testtopic\n      FALLBACK_OFFSET: earliest\n    KEY:\n      COLUMNS:\n      - NAME: customer_id\n        TYPE: int\n      FORMAT: delimited\n      DELIMITED_OPTION:\n        DELIMITER: '|'\n    VALUE:\n      COLUMNS:\n      - TYPE: integer\n        NAME: l_orderkey\n      - TYPE: integer\n        NAME: l_partkey\n      - TYPE: integer\n        NAME: l_suppkey\n      - TYPE: integer\n        NAME: l_linenumber\n      - TYPE: decimal\n        NAME: l_quantity\n      - TYPE: decimal\n        NAME: l_extendedprice\n      - TYPE: decimal\n        NAME: l_discount\n      - TYPE: decimal\n        NAME: l_tax\n      - TYPE: char\n        NAME: l_returnflag\n      - TYPE: char\n        NAME: l_linestatus\n      - TYPE: date\n        NAME: l_shipdate\n      - TYPE: date\n        NAME: l_commitdate\n      - TYPE: date\n        NAME: l_receiptdate\n      - TYPE: text\n        NAME: l_shipinstruct\n      - TYPE: text\n        NAME: l_shipmode\n      - TYPE: text\n        NAME: l_comment\n      FORMAT: delimited\n      DELIMITED_OPTION:\n        DELIMITER: '|'\n    ERROR_LIMIT: 10\n  OUTPUT:\n    SCHEMA: adbpgss_test\n    TABLE: write_with_insert_plaintext\n    MODE: MERGE\n    MATCH_COLUMNS:\n    - l_orderkey\n    - l_partkey\n    - l_suppkey\n    UPDATE_COLUMNS:\n    - l_linenumber\n    - l_quantity\n    - l_extendedprice\n    - l_discount\n    - l_tax\n    - l_returnflag\n    - l_linestatus\n    - l_shipdate\n    - l_commitdate\n    - l_receiptdate\n    - l_shipinstruct\n    - l_shipmode\n    - l_comment\n    MAPPING:\n    - EXPRESSION: l_orderkey\n      NAME: l_orderkey\n    - EXPRESSION: l_partkey\n      NAME: l_partkey\n    - EXPRESSION: l_suppkey\n      NAME: l_suppkey\n    - EXPRESSION: l_linenumber\n      NAME: l_linenumber\n    - EXPRESSION: l_quantity\n      NAME: l_quantity\n    - EXPRESSION: l_extendedprice\n      NAME: l_extendedprice\n    - EXPRESSION: l_discount\n      NAME: l_discount\n    - EXPRESSION: l_tax\n      NAME: l_tax\n    - EXPRESSION: l_returnflag\n      NAME: l_returnflag\n    - EXPRESSION: l_linestatus\n      NAME: l_linestatus\n    - EXPRESSION: l_shipdate\n      NAME: l_shipdate\n    - EXPRESSION: l_commitdate\n      NAME: l_commitdate\n    - EXPRESSION: l_receiptdate\n      NAME: l_receiptdate\n    - EXPRESSION: l_shipinstruct\n      NAME: l_shipinstruct\n    - EXPRESSION: l_shipmode\n      NAME: l_shipmode\n    - EXPRESSION: l_comment\n      NAME: l_comment\n  COMMIT:\n    MAX_ROW: 1000\n    MINIMAL_INTERVAL: 1000\n    CONSISTENCY: ATLEAST\n  POLL:\n    BATCHSIZE: 1000\n    TIMEOUT: 1000\n  PROPERTIES:\n    group.id: testgroup",
  "Status": "Running",
  "ErrorMessage": "\"\"",
  "CreateTime": "2019-09-08T16:00:00Z",
  "ModifyTime": "2019-09-08T17:00:00Z",
  "GroupName": "test_group",
  "Consistency": "ATLEAST / EXACTLY",
  "ErrorLimitCount": 5,
  "FallbackOffset": "EARLIEST /  LATEST",
  "DataSourceName": "test_kafka"
}

错误码

访问错误中心查看更多错误码。