前提条件
HBase全量数据迁移接口说明
创建任务
- 接口(POST):
http://{BDSMaster}:12311/hbase/fulldata/create
,BDSMaster需要修改为实例的Master hostname。 - 参数说明如下:
- @FormParam(“src”) String srcName:源集群名。
- @FormParam(“dst”) String sinkName:目标集群名。
- @FormParam(“tableNames”) String tableNames:迁移的表(支持多张表”,”号分割)
- @FormParam(“bulkload”) boolean bulkload:是否需要进行bulkload。
- @FormParam(“createTable”) boolean createTable:是否要创建目标表。
- @FormParam(“extraAttrs”) String extraAttrs:高级配置,可不填。
示例:curl -d "src=hbase1&dst=hbase2&tableNames=t1,t2,t3&bulkload=true&createTable=true" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{LTSMaster}:12311/hbase/fulldata/create
返回值:{ "success":"true", // 请求是否成功 "message":"workflow-36c61d1240f84ecfbb472751a29b6deb" //任务id }
- 查看所有任务
-
接口(GET):
http://{BDSMaster}:12311/hbase/fulldata
,BDSMaster需要修改为实例的Master hostname。示例:
curl http://{BDSMaster}:12311/hbase/fulldata
返回值:{ "tunnels": [ // 列出所有的任务信息 { "jobName": "workflow-779298e682a44d5f9c226cea0e6f4dfd", "tableNames": [ "t1" ], "dst": "*.*.*.*", "fullJobs": [ { "jobName": "fulldata-104f42d9bbf341099ae02c0b61a70f4c", "state": "SUCCEEDED", "table": "t1" } ], "bulkloadJobs": [ { "TaskName": "bulkload-73bf97f03f6140da91c3183e3d4fc1d8", "state": "SUCCEEDED" } ], "src": "*.*.*.*", "mergeTasks": [], "state": "SUCCEEDED", "inrcJobs": [], "schemeTasks": [ { "TaskName": "360338ff-abfb-4f78-9b73-580cc9fd48be", "state": "SKIPPED", "table": "t1" } ] }, { "jobName": "workflow-36c61d1240f84ecfbb472751a29b6deb", "tableNames": [ "t1", "t2" ], "dst": "*.*.*.*", "fullJobs": [ { "jobName": "fulldata-9014e168fccd433dbd56bcc61ba4ccbb", "state": "SUCCEEDED", "table": "t2" }, { "jobName": "fulldata-114cc535693344238572583268ae4c67", "state": "SUCCEEDED", "table": "t1" } ], "bulkloadJobs": [ { "TaskName": "bulkload-92402af7ef544f6182af892fe4b35c65", "state": "SUCCEEDED" }, { "TaskName": "bulkload-f7580e9ebf734c999656c1aaad41f67b", "state": "SUCCEEDED" } ], "src": "*.*.*.*", "mergeTasks": [], "state": "SUCCEEDED", "inrcJobs": [], "schemeTasks": [ { "TaskName": "b31b8f7c-9a74-4840-814a-6cd62d2b08c9", "state": "SKIPPED", "table": "t1" }, { "TaskName": "b1968a67-6684-4dd5-ba07-dbd7a52af24f", "state": "SKIPPED", "table": "t2" } ] } ], "success": "true", "message": "ok" }
- 查看指定任务ID的任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/fulldata/{jobId}/detail
,BDSMaster需要修改为实例的Master hostname。
- 暂停任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/fulldata/{jobId}/abort
,BDSMaster需要修改为实例的Master hostname。
- 删除任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/fulldata/{jobId}/del
,BDSMaster需要修改为实例的Master hostname。返回值:{ "success": "true", "message": "ok" }
- 重试任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/fulldata/{jobId}/retry
,BDSMaster需要修改为实例的Master hostname。返回值:{ "success": "true", "message": "ok" }
HBase增量任务同步接口
- 创建任务
- 接口(POST):
http://{BDSMaster}:12311/hbase/incr/create
,BDSMaster需要修改为实例的Master hostname。参数:- @FormParam(“name”) String name :同步通道的名称。
- @FormParam(“src”) String srcName :源集群名。
- @FormParam(“dst”) String sinkName :目标集群名。
- @FormParam(“tableNames”) String tableNames :迁移的表(支持多张表”,”号分割)。
- @FormParam(“createTable”) boolean createTable : 是否要创建目标表。
- @FormParam(“extraAttrs”) String extraAttrs :高级配置。
curl -d "name=test1&src=hbase1&dst=hbase2&tableNames=t1,t2,t3&createTable=true" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{LTSMaster}:12311/hbase/incr/create
返回值:{ "success":"true", "message":"wal-08a087da488349969ff4a4b78efbceba" // 同步通道的ID }
- 更新任务
-
接口(POST):
http://{BDSMaster}:12311/hbase/incr/update
,BDSMaster需要修改为实例的Master hostname。参数:- @FormParam(“jobId”) final String jobId:更新的任务ID。
- @FormParam(“tableNames”) final String tableNames:需要更新的表名,支持多张表,使用逗号(,)分开。
例子:curl -d "jobId=xxx&tableNames=t1,t2,t3" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{LTSMaster}:12311/hbase/incr/update
- 查看所有任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/incr
,BDSMaster需要修改为实例的Master hostname。返回值:{ "tunnels": [ { "jobId": "wal-2be4c2c89b7140a7b42c050916b3f5bb", "tableNames": [ "default:t1" ], "dst": "*.*.*.*", "offset": 1573560272951, "src": "*.*.*.*", "name": "wal-2be4c2c89b7140a7b42c050916b3f5bb", "syncDate": "2019-11-12 20:04:32", "state": "KILLED", "synctime": 1573560272951 }, { "jobId": "wal-08a087da488349969ff4a4b78efbceba", "tableNames": [ "default:t1", "default:t2", "default:t3" ], "dst": "*.*.*.*", "offset": 1573651478405, "src": "*.*.*.*", "name": "test1", "syncDate": "2019-11-13 21:31:46", "state": "RUNNING", "synctime": 1573651906518 } ], "success": "true", "message": "ok" }
- 查看指定ID的任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/incr/{jobId}/detail
,BDSMaster需要修改为实例的Master hostname。
- 暂停任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/incr/{jobId}/abort
,BDSMaster需要修改为实例的Master hostname。返回值:{ "success": "true", "message": "ok" }
- 删除任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/incr/{jobId}/del
,BDSMaster需要修改为实例的Master hostname。返回值:{ "success": "true", "message": "ok" }
- 重试任务
- 接口(GET):
http://{BDSMaster}:12311/hbase/incr/{jobId}/retry
,BDSMaster需要修改为实例的Master hostname。返回值:{ "success": "true", "message": "ok" }