全部产品
云市场

BDS OPEN API 使用说明

更新时间:2019-11-14 16:13:48

前置条件

  • 购买BDS数据迁移同步服务
  • 确保客户端和BDS网络互通
    • 添加BDS白名单
    • telnet {{ BDSMaster }} 12311

HBase全量数据迁移接口说明

创建任务

接口(POST):http://{BDSMaster}:12311/hbase/fulldata/create
参数:
@FormParam(“src”) String srcName -> 源集群名
@FormParam(“dst”) String sinkName -> 目标集群名
@FormParam(“tableNames”) String tableNames -> 迁移的表(支持多张表”,”号分割)
@FormParam(“bulkload”) boolean bulkload -> 是否需要进行bulkload
@FormParam(“createTable”) boolean createTable -> 是否要创建目标表
@FormParam(“extraAttrs”) String extraAttrs -> 高级配置,可不填
例子:

  1. curl -d "src=hbase1&dst=hbase2&tableNames=t1,t2,t3&bulkload=true&createTable=true" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{BDSMaster}:12311/hbase/fulldata/create

返回值:

  1. {
  2. "success":"true", // 请求是否成功
  3. "message":"workflow-36c61d1240f84ecfbb472751a29b6deb" //任务id
  4. }

查看所有任务

接口(GET):http://{BDSMaster}:12311/hbase/fulldata
例子:curl http://{BDSMaster}:12311/hbase/fulldata
返回值:

  1. {
  2. "tunnels": [ // 罗列出所有的任务信息
  3. {
  4. "jobName": "workflow-779298e682a44d5f9c226cea0e6f4dfd",
  5. "tableNames": [
  6. "t1"
  7. ],
  8. "dst": "100.81.140.116",
  9. "fullJobs": [
  10. {
  11. "jobName": "fulldata-104f42d9bbf341099ae02c0b61a70f4c",
  12. "state": "SUCCEEDED",
  13. "table": "t1"
  14. }
  15. ],
  16. "bulkloadJobs": [
  17. {
  18. "TaskName": "bulkload-73bf97f03f6140da91c3183e3d4fc1d8",
  19. "state": "SUCCEEDED"
  20. }
  21. ],
  22. "src": "11.164.87.151",
  23. "mergeTasks": [],
  24. "state": "SUCCEEDED",
  25. "inrcJobs": [],
  26. "schemeTasks": [
  27. {
  28. "TaskName": "360338ff-abfb-4f78-9b73-580cc9fd48be",
  29. "state": "SKIPPED",
  30. "table": "t1"
  31. }
  32. ]
  33. },
  34. {
  35. "jobName": "workflow-36c61d1240f84ecfbb472751a29b6deb",
  36. "tableNames": [
  37. "t1",
  38. "t2"
  39. ],
  40. "dst": "100.81.140.116",
  41. "fullJobs": [
  42. {
  43. "jobName": "fulldata-9014e168fccd433dbd56bcc61ba4ccbb",
  44. "state": "SUCCEEDED",
  45. "table": "t2"
  46. },
  47. {
  48. "jobName": "fulldata-114cc535693344238572583268ae4c67",
  49. "state": "SUCCEEDED",
  50. "table": "t1"
  51. }
  52. ],
  53. "bulkloadJobs": [
  54. {
  55. "TaskName": "bulkload-92402af7ef544f6182af892fe4b35c65",
  56. "state": "SUCCEEDED"
  57. },
  58. {
  59. "TaskName": "bulkload-f7580e9ebf734c999656c1aaad41f67b",
  60. "state": "SUCCEEDED"
  61. }
  62. ],
  63. "src": "11.164.87.151",
  64. "mergeTasks": [],
  65. "state": "SUCCEEDED",
  66. "inrcJobs": [],
  67. "schemeTasks": [
  68. {
  69. "TaskName": "b31b8f7c-9a74-4840-814a-6cd62d2b08c9",
  70. "state": "SKIPPED",
  71. "table": "t1"
  72. },
  73. {
  74. "TaskName": "b1968a67-6684-4dd5-ba07-dbd7a52af24f",
  75. "state": "SKIPPED",
  76. "table": "t2"
  77. }
  78. ]
  79. }
  80. ],
  81. "success": "true",
  82. "message": "ok"
  83. }

查看指定任务id的任务

接口(GET):http://{BDSMaster}:12311/hbase/fulldata/{jobId}/detail

暂停任务

接口(GET):http://{BDSMaster}:12311/hbase/fulldata/{jobId}/abort
返回值:

  1. {
  2. "success": "true",
  3. "message": "ok"
  4. }

删除任务

接口(GET):http://{BDSMaster}:12311/hbase/fulldata/{jobId}/del
返回值:

  1. {
  2. "success": "true",
  3. "message": "ok"
  4. }

重试任务

接口(GET):http://{BDSMaster}:12311/hbase/fulldata/{jobId}/retry
返回值:

  1. {
  2. "success": "true",
  3. "message": "ok"
  4. }

HBase增量任务同步接口

创建任务

接口(POST):http://{BDSMaster}:12311/hbase/incr/create
参数:
@FormParam(“name”) String name -> 同步通道的名称
@FormParam(“src”) String srcName -> 源集群名
@FormParam(“dst”) String sinkName -> 目标集群名
@FormParam(“tableNames”) String tableNames -> 迁移的表(支持多张表”,”号分割)
@FormParam(“createTable”) boolean createTable -> 是否要创建目标表
@FormParam(“extraAttrs”) String extraAttrs -> 高级配置,可不填
例子:

  1. curl -d "name=test1&src=hbase1&dst=hbase2&tableNames=t1,t2,t3&createTable=true" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{BDSMaster}:12311/hbase/incr/create

返回值:

  1. {
  2. "success":"true",
  3. "message":"wal-08a087da488349969ff4a4b78efbceba" // 同步通道的id
  4. }

更新任务

接口(POST):http://{BDSMaster}:12311/hbase/incr/update
参数:
@FormParam(“jobId”) final String jobId
@FormParam(“tableNames”) final String tableNames
例子:

  1. curl -d "jobId=xxx&tableNames=t1,t2,t3" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{BDSMaster}:12311/hbase/incr/update

查看所有任务

接口(GET):http://{BDSMaster}:12311/hbase/incr
返回值:

  1. {
  2. "tunnels": [
  3. {
  4. "jobId": "wal-2be4c2c89b7140a7b42c050916b3f5bb",
  5. "tableNames": [
  6. "default:t1"
  7. ],
  8. "dst": "100.81.140.116",
  9. "offset": 1573560272951,
  10. "src": "11.164.87.151",
  11. "name": "wal-2be4c2c89b7140a7b42c050916b3f5bb",
  12. "syncDate": "2019-11-12 20:04:32",
  13. "state": "KILLED",
  14. "synctime": 1573560272951
  15. },
  16. {
  17. "jobId": "wal-08a087da488349969ff4a4b78efbceba",
  18. "tableNames": [
  19. "default:t1",
  20. "default:t2",
  21. "default:t3"
  22. ],
  23. "dst": "100.81.140.116",
  24. "offset": 1573651478405,
  25. "src": "11.164.87.151",
  26. "name": "test1",
  27. "syncDate": "2019-11-13 21:31:46",
  28. "state": "RUNNING",
  29. "synctime": 1573651906518
  30. }
  31. ],
  32. "success": "true",
  33. "message": "ok"
  34. }

查看指定id的任务

接口(GET):http://{BDSMaster}:12311/hbase/incr/{jobId}/detail

暂停任务

接口(GET):http://{BDSMaster}:12311/hbase/incr/{jobId}/abort
返回值:

  1. {
  2. "success": "true",
  3. "message": "ok"
  4. }

删除任务

接口(GET):http://{BDSMaster}:12311/hbase/incr/{jobId}/del
返回值:

  1. {
  2. "success": "true",
  3. "message": "ok"
  4. }

重试任务

接口(GET):http://{BDSMaster}:12311/hbase/incr/{jobId}/retry
返回值:

  1. {
  2. "success": "true",
  3. "message": "ok"
  4. }