HBase 迁移方案说明

更新时间:
复制为 MD 格式

本文详细介绍了将AWS HBase集群迁移至阿里云HBase的完整方案,涵盖建立复制关系、迁移存量快照、同步增量数据、版本兼容性配置、网络与权限要求,以及多表批量迁移工具的使用。

迁移流程概述

  1. 建立复制关系(Peer):

    • 使用HBase Replication功能,在源集群和目标集群之间建立表级别的复制关系,并暂时关闭自动同步功能。

    • 源集群会记录针对表的实时写入和变更操作,但不会立即将其同步到目标集群。

  2. 迁移存量数据:

    • 利用HBase Snapshot导出源集群的存量数据。

    • 如果源集群和目标集群的存储系统(如HDFSOSS-HDFS)互通,则直接将Snapshot导出到目标集群;否则,需先导出到源集群的中间路径,再同步到目标集群。

  3. 同步增量数据:存量数据恢复完成后,开启Replication Peer的自动同步功能,回放迁移期间产生的增量数据,确保源端和目标端的数据一致性。

通过上述步骤,本方案在保证源集群服务持续运行的同时,完整迁移存量和增量数据,避免数据丢失。

注意事项

  • HBase版本兼容性:

    • HBase 1.x版本:需设置hbase.replication=true,并重启主备集群以启用Replication功能。

    • HBase 2.x版本:默认已移除hbase.replication参数,无需额外设置。

  • Replication配置:HBase是以列族为单位设置是否启用Replication。如果需要迁移特定列族的数据,请确保相关列族已正确配置Replication属性。

  • 网络连通性:

    • 通过CEN(云企业网)、专线或VPN等方式,确保源HBase集群与目标HBase集群之间的网络互通。

    • 提前打通源集群和目标集群的网络连接,确保以下端口对源集群开放:

      • Zookeeper服务端口:目标集群Zookeeper所在ECS实例的2181端口。

      • HBase服务端口:目标集群EMR HBase所在ECS实例的160101602016030端口。

  • 用户与权限:

    • 实施前,在源端hbase 上,建议执行 su - hbase hbase 用户进行快照的创建/传输操作。

操作流程

步骤一:创建Peer(复制关系)

在源集群和目标集群之间建立表级别的复制关系,并暂时关闭自动同步功能,待完成存量Snapshot数据的同步之后再开启自动同步。

  1. 登录源集群的Master节点。

  2. 执行以下命令,进入HBase Shell。

hbase shell
  1. 添加Peer(复制关系)。

HBase Shell中执行以下命令,为目标集群添加一个Peer(复制关系),并指定待迁移的表。

add_peer '${peer_name}', CLUSTER_KEY => "${slave_zk}:${port}:/hbase", TABLE_CFS => { "${table_name}" => [] }

涉及参数如下:

  • ${peer_name}:复制关系名称,您可以自定义。本文示例为peer1

  • ${slave_zk}:目标集群ZooKeeper的地址。通常是多个ZooKeeper节点的内网IP地址或主机名,格式为{slave_zk1}:2181,{slave_zk2}:2181,{slave_zk3}:2181

  • ${port}:目标集群ZooKeeper的端口,默认端口为2181。

  • ${table_name}:待迁移的表名。本文示例为t1

  • 启用表级复制。

启用表级复制功能,确保指定的表能够将写入的数据同步到目标集群。

enable_table_replication 't1'
  1. 暂时关闭自动同步。

该命令用于暂停指定对等体的数据复制过程。禁用后,源集群将不再向目标集群发送新的数据变更,但已有的数据不会被删除或影响。

请注意:在开启 peer 复制后但不增量同步时,源端会持续向 wal 里面写入文件,而这些文件没有被消费后清理,过长的时间后可能会造成积压,所以此状态持续时长不宜过长,完成存量数据的同步后,建议尽快按照【步骤五】的说明开启增量同步,并在增量同步工作完成后,按照【步骤八】的说明清理 peer。

disable_peer 'peer1'

步骤二:创建Snapshot

在源集群的HBase Shell中,执行以下命令为待迁移的表创建Snapshot,用于导出存量数据。

snapshot '${table_name}', '${snapshot_name}'
  • 涉及参数如下:

    • ${table_name}:待迁移的表名。本文示例为t1

    • ${snapshot_name}:自定义的Snapshot名称。本文示例为t1-snapshot

  • 示例如下所示。

snapshot 't1', 't1-snapshot'

创建后查看快照是否已经创建成功:

hbase>list_snapshots

步骤三:导出Snapshot到目标集群

此步骤的实现有两种方式:

方式一:实现源集群和目标集群的存储系统互通,然后在源集群执行以下命令,直接导出Snapshot到目标集群。【此命令需要退出 hbase 命令行,在源 habse linux 服务器中执行】

hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot \
-snapshot ${snapshot_name} \
--copy-to ${target_rootdir_path}/habse/\

如果需要限速,则可以执行如下脚本,其中-mappers <num>用于控制并行的 mapper 数量,减少 mapper 数量可间接降低总带宽和系统负载。-bandwidth <MB/s>用于限制每个 mapper 的最大带宽(单位:MB/s)。例如 100 表示每个 map 任务最多使用 100 MB/s。注意:这是 每 mapper 的限制,不是总带宽。

hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot \
  -snapshot ${snapshot_name} \
  -copy-to ${target_rootdir_path}/hbase/ \
  -bandwidth 100 \
  -mappers 10
  • 涉及参数如下:

    • ${snapshot_name}:自定义的Snapshot名称。本文示例为t1-snapshot

    • ${target_rootdir_path}:目标集群HBase的根目录路径,请根据实际情况替换。

      • OSS-HDFS:您可以通过控制台,在目标集群的HBase服务中,查看hbase-site.xml文件的hbase.rootdir配置项,获取详细的路径信息。

image

  • HDFS:您可以通过控制台,在目标集群的Hadoop-Common服务中,查看core-site.xml中的fs.defaultFS配置项,获取详细的路径信息。

  • 示例如下所示。

hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot t1-snapshot -copy-to oss://xxx.cn-hangzhou.oss-dls.aliyuncs.com/hbase/
  • 如果多次传输,需要覆盖先前快照,则添加 -overwrite参数,示例如下:

hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot t1-snapshot -copy-to oss://xxx.cn-hangzhou.oss-dls.aliyuncs.com/hbase/ -overwrite 

方式二:实现源集群和目标集群的存储系统互通,然后在目标集群执行以下命令,直接导出Snapshot到目标集群。目标集群执行此操作有利于减少源端性能损耗【此命令需要退出 hbase 命令行,在源 habse linux 服务器中执行】

# 在目标集群的 HBase 客户端节点执行
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot \
  -snapshot ${snapshot_name} \
  -copy-from hdfs://<source-nn>:8020/hbase \
  -copy-to hdfs://<target-nn>:8020/hbase \
  -bandwidth 50 \
  -mappers 8

步骤四:使用Snapshot恢复存量数据

  1. 登录目标集群的Master节点。

  2. 执行以下命令,进入HBase Shell。

hbase shell
  1. 执行以下命令,在目标集群恢复Snapshot并启用表。 

#如果表已存在需要disable,如果不确定是否存在,可以执行disable,执行后报error说明不存在,可以忽略。
disable '${table_name}'
#恢复快照
restore_snapshot '${snapshot_name}'
enable '${table_name}'
  • 涉及参数如下:

    • ${snapshot_name}:自定义的Snapshot名称。

    • ${table_name}:待迁移的表名。

  • 示例如下所示。

disable 't1'
restore_snapshot 't1-snapshot'
enable 't1'

步骤五:开启增量数据同步

在源集群的HBase Shell中执行以下命令,开启Peer同步。

enable_peer '${peer_name}'

示例如下所示。

enable_peer 'peer1'

步骤六:验证数据

验证迁移后的数据是否完整。

  • 小数据量场景:使用Scan验证。

scan '${table_name}'

示例如下所示。

scan 't1'
  • 中等数据量场景:使用count验证。

count '${table_name}'
  • 大数据量场景:使用get抽样验证。

get '${table_name}', '${rowkey}'

代码中的${rowkey}HBase表中每一行的唯一标识符。

步骤七:删除Snapshot

验证无误后,执行以下命令删除与目标表相关的所有快照,以释放存储空间。

请务必执行此操作,否则会导致快照所在集群的磁盘使用率异常增长。

delete_table_snapshots '${table_name}'

示例如下所示。

delete_table_snapshots 't1'

步骤八:清理Peer

在增量数据同步完成后,需进行应用的双写或割接操作,确保所有读写请求切换到目标HBase集群。为避免数据重复同步,在完成切换后需要删除源集群与目标集群之间的复制关系(Peer)。

  1. 客户端迁移。

将涉及HBase集群的上下游应用切换到目标HBase集群,包括通过API或命令行读写HBase的应用。具体操作如下:

  • 更新连接配置:修改应用的配置文件或代码,将HBase连接信息(如Zookeeper地址、端口等)从源集群切换为目标集群。

  • 验证功能:确保应用能够正常读写目标集群,并进行必要的功能测试和数据校验。

  • 双跑或割接:根据业务需求,选择双跑(同时读写源和目标集群)或直接割接到目标集群。

  • 禁用Peer的自动同步功能。

在源集群的HBase Shell中,禁用指定Peer的自动同步功能,确保数据同步立即停止。

disable_peer '${peer_name}'

示例如下所示。

disable_peer 'peer1'
  1. 删除Peer。

禁用Peer后,删除Peer以彻底断开源集群与目标集群之间的复制关系。

remove_peer '${peer_name}'

示例如下所示。

remove_peer 'peer1'
  1. 验证Peer是否删除成功。

执行以下命令,列出当前的所有Peer,确保本文示例中的 peer1已被成功删除。

list_peers

多表批量存量数据迁移操作说明

使用场景

在需要迁移的表数量较大,按照上文中的步骤重复操作需要很大工作量,无法短时间内完成的情况下,可通过如下压缩包中的工具,批量生成执行文件,批量执行。

批量操作工具.zip

使用步骤

使用步骤

  1. 整理需要迁移的表清单,并按照示例格式整理在压缩包中的「tables.txt」文件中

tables.txt

TASK20240416000001_20250726
TASK20240416000002_20250726
TASK20240416000003_20250726
TASK20240416000004_20250726
TASK20240724000001_20250726
TASK20240725000001_20250726
TASK20240725000002_20250726
TASK20240725000003_20250726
TASK20240730000001_20250726
TASK20250522000001_20250726
TASK20250523000001_20250726
bao_shui_whid_detail
channel_to_sale_store_code
color_inv_bonded_target
color_sku_dim
dim_week_date
dwd_channel_erp_product_relation
dwd_oms_order
dwd_product_sku_exist
dwd_search_click_event_data
dwd_search_event_data
dwd_utm_visitor_product_detail
dwd_utm_visitor_session
dwd_utm_visitor_user_log
erp_sku_color_id
erp_sku_month_start_qty
erp_sku_to_channel_sku
flink_delivery_confirm_error_log
flink_delivery_confirm_relation
flink_delivery_lock_relation
flink_delivery_order
flink_delivery_order_detail
flink_delivery_order_lock
flink_oms_order
flink_oms_order_delivery_relation
flink_poas_translate_channel_order
hb_add_friends
hb_add_inchat
hb_cart_favorite_need_reflush
hb_channel_member_info
hb_channel_member_rk_acc
hb_channel_member_rk_user
hb_dim_channel_store
hb_dim_product_info
hb_dim_sigoweb_product_info
hb_dim_source_channel
hb_item_sku_info
hb_key_word_need_reflush
hb_logic_brand_info
hb_logic_brand_rel
hb_member_bind_relation_union
hb_member_bind_relation_user
hb_mes_equipment_info
hb_oms_order
hb_oms_order_detail
hb_page_view_need_reflush
hb_pay_order_first
hb_pay_order_first_product
hb_pay_order_first_union
hb_pay_order_first_user
hb_pay_order_upload_union
hb_product_sale_plan
hb_qy_user
hb_register_unionid_rel
hb_sdk_channel
hb_sku_to_product_code
hb_udf_user_chats
hb_udf_user_friends
hb_user_account_order_record
hb_user_brand_info
hb_utm_view_need_reflush
hb_utm_view_need_reflush_test
hb_view_first
hb_view_first_union
hb_view_first_user
hb_visitor_accum
hb_visitor_user_rel
hb_wecom_add_friends
hb_wecom_add_inchat
hb_wecom_first_order
member_action_detail
member_continuous_login
member_continuous_sign
pay_account_order_table
pay_order_upload_table
procure_transit_data
product_color_dim
product_color_sku_dim
product_inv_bonded_target
product_sale_plan_target
rebate_cost
sigo_web_product_map
sigoweb_brand_id_code_map
sku_bonded_stock_cental_detail
sku_id_to_code
sku_inv_bonded_target
sku_sale_msg
sku_stock_cental_detail
stock_in_transit_detail
subscribe_restocked_hb
user_view_pay
visitor_view_pay
whid_color_stock_central_target
whid_detail
whid_product_stock_central_target
whid_sku_stock_cental_target
xhs_ascribe
yjq_company_info
yjq_page_tag
yjq_page_type
yjq_product_dim
  1. 运行压缩包中的「genscript.py」文件,生成「create_file = 'snapshot_create.sh'」「export_file = 'snapshot_export.sh'」「restore_file = 'snapshot_restore.sh'」三个文件。

genScript.py

#!/usr/bin/env python3

# 输入文件路径(每行一个表名)
input_file = 'tables.txt'

# 输出的三个脚本文件
create_file = 'snapshot_create.sh'
export_file = 'snapshot_export.sh'
restore_file = 'snapshot_restore.sh'
count_file = 'count.sh'

# 目标 HDFS 地址
hdfs_target = 'hdfs://10.86.5.94:8020/hbase/'

# 打开三个输出文件进行写入
with open(input_file, 'r') as fin:
    with open(create_file, 'w') as f_create, \
         open(export_file, 'w') as f_export, \
         open(restore_file, 'w') as f_restore, \
         open(count_file, 'w') as f_count  :

        # 添加 shebang 行
        f_create.write('#!/bin/bash\n')
        f_export.write('#!/bin/bash\n')
        f_restore.write('#!/bin/bash\n')

        for line in fin:
            table_name = line.strip()
            if not table_name or table_name.startswith('#'):
                continue  # 忽略空行和注释行

            snapshot_name = f"{table_name}_snapshot"

            # 命令1:创建快照
            del_cmd = f"delete_snapshot '{snapshot_name}'"
            f_create.write(del_cmd + '\n')
            create_cmd = f"snapshot '{table_name}','{snapshot_name}'"
            f_create.write(create_cmd + '\n')

            # 命令2:导出快照
            export_cmd = (
                f"hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot "
                f"-snapshot {snapshot_name} -copy-to {hdfs_target}  -overwrite"
            )
            f_export.write(export_cmd + '\n')

            # 命令3:恢复快照(disable -> restore -> enable)
            restore_shell = (
                f"echo \"disable '{table_name}'; restore_snapshot '{snapshot_name}'; enable '{table_name}'\" | hbase shell -n "
            )
            f_restore.write(restore_shell + '\n')
            # 命令四:count(disable -> restore -> enable)
            count_cmd = (
                f"echo \"count '{table_name}'\" | hbase shell -n >>count.log"
            )
            f_count.write(count_cmd + '\n')


print("✅ 四个脚本已生成:")
print(f"   - {create_file}")
print(f"   - {export_file}")
print(f"   - {restore_file}")
print(f"   - {count_file}")

  1. 在源端环境hbaseshell中运行「snapshot_create.sh」文件,批量创建快照。

snapshot_create.sh

#!/bin/bash
delete_snapshot 'TASK20240416000001_20250726_snapshot'
snapshot 'TASK20240416000001_20250726','TASK20240416000001_20250726_snapshot'
delete_snapshot 'TASK20240416000002_20250726_snapshot'
snapshot 'TASK20240416000002_20250726','TASK20240416000002_20250726_snapshot'
delete_snapshot 'TASK20240416000003_20250726_snapshot'
snapshot 'TASK20240416000003_20250726','TASK20240416000003_20250726_snapshot'
delete_snapshot 'TASK20240416000004_20250726_snapshot'
snapshot 'TASK20240416000004_20250726','TASK20240416000004_20250726_snapshot'
delete_snapshot 'TASK20240724000001_20250726_snapshot'
snapshot 'TASK20240724000001_20250726','TASK20240724000001_20250726_snapshot'
delete_snapshot 'TASK20240725000001_20250726_snapshot'
snapshot 'TASK20240725000001_20250726','TASK20240725000001_20250726_snapshot'
delete_snapshot 'TASK20240725000002_20250726_snapshot'
snapshot 'TASK20240725000002_20250726','TASK20240725000002_20250726_snapshot'
delete_snapshot 'TASK20240725000003_20250726_snapshot'
snapshot 'TASK20240725000003_20250726','TASK20240725000003_20250726_snapshot'
delete_snapshot 'TASK20240730000001_20250726_snapshot'
snapshot 'TASK20240730000001_20250726','TASK20240730000001_20250726_snapshot'
delete_snapshot 'TASK20250522000001_20250726_snapshot'
snapshot 'TASK20250522000001_20250726','TASK20250522000001_20250726_snapshot'
delete_snapshot 'TASK20250523000001_20250726_snapshot'
snapshot 'TASK20250523000001_20250726','TASK20250523000001_20250726_snapshot'
delete_snapshot 'bao_shui_whid_detail_snapshot'
snapshot 'bao_shui_whid_detail','bao_shui_whid_detail_snapshot'
delete_snapshot 'channel_to_sale_store_code_snapshot'
snapshot 'channel_to_sale_store_code','channel_to_sale_store_code_snapshot'
delete_snapshot 'color_inv_bonded_target_snapshot'
snapshot 'color_inv_bonded_target','color_inv_bonded_target_snapshot'
delete_snapshot 'color_sku_dim_snapshot'
snapshot 'color_sku_dim','color_sku_dim_snapshot'
delete_snapshot 'dim_week_date_snapshot'
snapshot 'dim_week_date','dim_week_date_snapshot'
delete_snapshot 'dwd_channel_erp_product_relation_snapshot'
snapshot 'dwd_channel_erp_product_relation','dwd_channel_erp_product_relation_snapshot'
delete_snapshot 'dwd_oms_order_snapshot'
snapshot 'dwd_oms_order','dwd_oms_order_snapshot'
delete_snapshot 'dwd_product_sku_exist_snapshot'
snapshot 'dwd_product_sku_exist','dwd_product_sku_exist_snapshot'
delete_snapshot 'dwd_search_click_event_data_snapshot'
snapshot 'dwd_search_click_event_data','dwd_search_click_event_data_snapshot'
delete_snapshot 'dwd_search_event_data_snapshot'
snapshot 'dwd_search_event_data','dwd_search_event_data_snapshot'
delete_snapshot 'dwd_utm_visitor_product_detail_snapshot'
snapshot 'dwd_utm_visitor_product_detail','dwd_utm_visitor_product_detail_snapshot'
delete_snapshot 'dwd_utm_visitor_session_snapshot'
snapshot 'dwd_utm_visitor_session','dwd_utm_visitor_session_snapshot'
delete_snapshot 'dwd_utm_visitor_user_log_snapshot'
snapshot 'dwd_utm_visitor_user_log','dwd_utm_visitor_user_log_snapshot'
delete_snapshot 'erp_sku_color_id_snapshot'
snapshot 'erp_sku_color_id','erp_sku_color_id_snapshot'
delete_snapshot 'erp_sku_month_start_qty_snapshot'
snapshot 'erp_sku_month_start_qty','erp_sku_month_start_qty_snapshot'
delete_snapshot 'erp_sku_to_channel_sku_snapshot'
snapshot 'erp_sku_to_channel_sku','erp_sku_to_channel_sku_snapshot'
delete_snapshot 'flink_delivery_confirm_error_log_snapshot'
snapshot 'flink_delivery_confirm_error_log','flink_delivery_confirm_error_log_snapshot'
delete_snapshot 'flink_delivery_confirm_relation_snapshot'
snapshot 'flink_delivery_confirm_relation','flink_delivery_confirm_relation_snapshot'
delete_snapshot 'flink_delivery_lock_relation_snapshot'
snapshot 'flink_delivery_lock_relation','flink_delivery_lock_relation_snapshot'
delete_snapshot 'flink_delivery_order_snapshot'
snapshot 'flink_delivery_order','flink_delivery_order_snapshot'
delete_snapshot 'flink_delivery_order_detail_snapshot'
snapshot 'flink_delivery_order_detail','flink_delivery_order_detail_snapshot'
delete_snapshot 'flink_delivery_order_lock_snapshot'
snapshot 'flink_delivery_order_lock','flink_delivery_order_lock_snapshot'
delete_snapshot 'flink_oms_order_snapshot'
snapshot 'flink_oms_order','flink_oms_order_snapshot'
delete_snapshot 'flink_oms_order_delivery_relation_snapshot'
snapshot 'flink_oms_order_delivery_relation','flink_oms_order_delivery_relation_snapshot'
delete_snapshot 'flink_poas_translate_channel_order_snapshot'
snapshot 'flink_poas_translate_channel_order','flink_poas_translate_channel_order_snapshot'
delete_snapshot 'hb_add_friends_snapshot'
snapshot 'hb_add_friends','hb_add_friends_snapshot'
delete_snapshot 'hb_add_inchat_snapshot'
snapshot 'hb_add_inchat','hb_add_inchat_snapshot'
delete_snapshot 'hb_cart_favorite_need_reflush_snapshot'
snapshot 'hb_cart_favorite_need_reflush','hb_cart_favorite_need_reflush_snapshot'
delete_snapshot 'hb_channel_member_info_snapshot'
snapshot 'hb_channel_member_info','hb_channel_member_info_snapshot'
delete_snapshot 'hb_channel_member_rk_acc_snapshot'
snapshot 'hb_channel_member_rk_acc','hb_channel_member_rk_acc_snapshot'
delete_snapshot 'hb_channel_member_rk_user_snapshot'
snapshot 'hb_channel_member_rk_user','hb_channel_member_rk_user_snapshot'
delete_snapshot 'hb_dim_channel_store_snapshot'
snapshot 'hb_dim_channel_store','hb_dim_channel_store_snapshot'
delete_snapshot 'hb_dim_product_info_snapshot'
snapshot 'hb_dim_product_info','hb_dim_product_info_snapshot'
delete_snapshot 'hb_dim_sigoweb_product_info_snapshot'
snapshot 'hb_dim_sigoweb_product_info','hb_dim_sigoweb_product_info_snapshot'
delete_snapshot 'hb_dim_source_channel_snapshot'
snapshot 'hb_dim_source_channel','hb_dim_source_channel_snapshot'
delete_snapshot 'hb_item_sku_info_snapshot'
snapshot 'hb_item_sku_info','hb_item_sku_info_snapshot'
delete_snapshot 'hb_key_word_need_reflush_snapshot'
snapshot 'hb_key_word_need_reflush','hb_key_word_need_reflush_snapshot'
delete_snapshot 'hb_logic_brand_info_snapshot'
snapshot 'hb_logic_brand_info','hb_logic_brand_info_snapshot'
delete_snapshot 'hb_logic_brand_rel_snapshot'
snapshot 'hb_logic_brand_rel','hb_logic_brand_rel_snapshot'
delete_snapshot 'hb_member_bind_relation_union_snapshot'
snapshot 'hb_member_bind_relation_union','hb_member_bind_relation_union_snapshot'
delete_snapshot 'hb_member_bind_relation_user_snapshot'
snapshot 'hb_member_bind_relation_user','hb_member_bind_relation_user_snapshot'
delete_snapshot 'hb_mes_equipment_info_snapshot'
snapshot 'hb_mes_equipment_info','hb_mes_equipment_info_snapshot'
delete_snapshot 'hb_oms_order_snapshot'
snapshot 'hb_oms_order','hb_oms_order_snapshot'
delete_snapshot 'hb_oms_order_detail_snapshot'
snapshot 'hb_oms_order_detail','hb_oms_order_detail_snapshot'
delete_snapshot 'hb_page_view_need_reflush_snapshot'
snapshot 'hb_page_view_need_reflush','hb_page_view_need_reflush_snapshot'
delete_snapshot 'hb_pay_order_first_snapshot'
snapshot 'hb_pay_order_first','hb_pay_order_first_snapshot'
delete_snapshot 'hb_pay_order_first_product_snapshot'
snapshot 'hb_pay_order_first_product','hb_pay_order_first_product_snapshot'
delete_snapshot 'hb_pay_order_first_union_snapshot'
snapshot 'hb_pay_order_first_union','hb_pay_order_first_union_snapshot'
delete_snapshot 'hb_pay_order_first_user_snapshot'
snapshot 'hb_pay_order_first_user','hb_pay_order_first_user_snapshot'
delete_snapshot 'hb_pay_order_upload_union_snapshot'
snapshot 'hb_pay_order_upload_union','hb_pay_order_upload_union_snapshot'
delete_snapshot 'hb_product_sale_plan_snapshot'
snapshot 'hb_product_sale_plan','hb_product_sale_plan_snapshot'
delete_snapshot 'hb_qy_user_snapshot'
snapshot 'hb_qy_user','hb_qy_user_snapshot'
delete_snapshot 'hb_register_unionid_rel_snapshot'
snapshot 'hb_register_unionid_rel','hb_register_unionid_rel_snapshot'
delete_snapshot 'hb_sdk_channel_snapshot'
snapshot 'hb_sdk_channel','hb_sdk_channel_snapshot'
delete_snapshot 'hb_sku_to_product_code_snapshot'
snapshot 'hb_sku_to_product_code','hb_sku_to_product_code_snapshot'
delete_snapshot 'hb_udf_user_chats_snapshot'
snapshot 'hb_udf_user_chats','hb_udf_user_chats_snapshot'
delete_snapshot 'hb_udf_user_friends_snapshot'
snapshot 'hb_udf_user_friends','hb_udf_user_friends_snapshot'
delete_snapshot 'hb_user_account_order_record_snapshot'
snapshot 'hb_user_account_order_record','hb_user_account_order_record_snapshot'
delete_snapshot 'hb_user_brand_info_snapshot'
snapshot 'hb_user_brand_info','hb_user_brand_info_snapshot'
delete_snapshot 'hb_utm_view_need_reflush_snapshot'
snapshot 'hb_utm_view_need_reflush','hb_utm_view_need_reflush_snapshot'
delete_snapshot 'hb_utm_view_need_reflush_test_snapshot'
snapshot 'hb_utm_view_need_reflush_test','hb_utm_view_need_reflush_test_snapshot'
delete_snapshot 'hb_view_first_snapshot'
snapshot 'hb_view_first','hb_view_first_snapshot'
delete_snapshot 'hb_view_first_union_snapshot'
snapshot 'hb_view_first_union','hb_view_first_union_snapshot'
delete_snapshot 'hb_view_first_user_snapshot'
snapshot 'hb_view_first_user','hb_view_first_user_snapshot'
delete_snapshot 'hb_visitor_accum_snapshot'
snapshot 'hb_visitor_accum','hb_visitor_accum_snapshot'
delete_snapshot 'hb_visitor_user_rel_snapshot'
snapshot 'hb_visitor_user_rel','hb_visitor_user_rel_snapshot'
delete_snapshot 'hb_wecom_add_friends_snapshot'
snapshot 'hb_wecom_add_friends','hb_wecom_add_friends_snapshot'
delete_snapshot 'hb_wecom_add_inchat_snapshot'
snapshot 'hb_wecom_add_inchat','hb_wecom_add_inchat_snapshot'
delete_snapshot 'hb_wecom_first_order_snapshot'
snapshot 'hb_wecom_first_order','hb_wecom_first_order_snapshot'
delete_snapshot 'member_action_detail_snapshot'
snapshot 'member_action_detail','member_action_detail_snapshot'
delete_snapshot 'member_continuous_login_snapshot'
snapshot 'member_continuous_login','member_continuous_login_snapshot'
delete_snapshot 'member_continuous_sign_snapshot'
snapshot 'member_continuous_sign','member_continuous_sign_snapshot'
delete_snapshot 'pay_account_order_table_snapshot'
snapshot 'pay_account_order_table','pay_account_order_table_snapshot'
delete_snapshot 'pay_order_upload_table_snapshot'
snapshot 'pay_order_upload_table','pay_order_upload_table_snapshot'
delete_snapshot 'procure_transit_data_snapshot'
snapshot 'procure_transit_data','procure_transit_data_snapshot'
delete_snapshot 'product_color_dim_snapshot'
snapshot 'product_color_dim','product_color_dim_snapshot'
delete_snapshot 'product_color_sku_dim_snapshot'
snapshot 'product_color_sku_dim','product_color_sku_dim_snapshot'
delete_snapshot 'product_inv_bonded_target_snapshot'
snapshot 'product_inv_bonded_target','product_inv_bonded_target_snapshot'
delete_snapshot 'product_sale_plan_target_snapshot'
snapshot 'product_sale_plan_target','product_sale_plan_target_snapshot'
delete_snapshot 'rebate_cost_snapshot'
snapshot 'rebate_cost','rebate_cost_snapshot'
delete_snapshot 'sigo_web_product_map_snapshot'
snapshot 'sigo_web_product_map','sigo_web_product_map_snapshot'
delete_snapshot 'sigoweb_brand_id_code_map_snapshot'
snapshot 'sigoweb_brand_id_code_map','sigoweb_brand_id_code_map_snapshot'
delete_snapshot 'sku_bonded_stock_cental_detail_snapshot'
snapshot 'sku_bonded_stock_cental_detail','sku_bonded_stock_cental_detail_snapshot'
delete_snapshot 'sku_id_to_code_snapshot'
snapshot 'sku_id_to_code','sku_id_to_code_snapshot'
delete_snapshot 'sku_inv_bonded_target_snapshot'
snapshot 'sku_inv_bonded_target','sku_inv_bonded_target_snapshot'
delete_snapshot 'sku_sale_msg_snapshot'
snapshot 'sku_sale_msg','sku_sale_msg_snapshot'
delete_snapshot 'sku_stock_cental_detail_snapshot'
snapshot 'sku_stock_cental_detail','sku_stock_cental_detail_snapshot'
delete_snapshot 'stock_in_transit_detail_snapshot'
snapshot 'stock_in_transit_detail','stock_in_transit_detail_snapshot'
delete_snapshot 'subscribe_restocked_hb_snapshot'
snapshot 'subscribe_restocked_hb','subscribe_restocked_hb_snapshot'
delete_snapshot 'user_view_pay_snapshot'
snapshot 'user_view_pay','user_view_pay_snapshot'
delete_snapshot 'visitor_view_pay_snapshot'
snapshot 'visitor_view_pay','visitor_view_pay_snapshot'
delete_snapshot 'whid_color_stock_central_target_snapshot'
snapshot 'whid_color_stock_central_target','whid_color_stock_central_target_snapshot'
delete_snapshot 'whid_detail_snapshot'
snapshot 'whid_detail','whid_detail_snapshot'
delete_snapshot 'whid_product_stock_central_target_snapshot'
snapshot 'whid_product_stock_central_target','whid_product_stock_central_target_snapshot'
delete_snapshot 'whid_sku_stock_cental_target_snapshot'
snapshot 'whid_sku_stock_cental_target','whid_sku_stock_cental_target_snapshot'
delete_snapshot 'xhs_ascribe_snapshot'
snapshot 'xhs_ascribe','xhs_ascribe_snapshot'
delete_snapshot 'yjq_company_info_snapshot'
snapshot 'yjq_company_info','yjq_company_info_snapshot'
delete_snapshot 'yjq_page_tag_snapshot'
snapshot 'yjq_page_tag','yjq_page_tag_snapshot'
delete_snapshot 'yjq_page_type_snapshot'
snapshot 'yjq_page_type','yjq_page_type_snapshot'
delete_snapshot 'yjq_product_dim_snapshot'
snapshot 'yjq_product_dim','yjq_product_dim_snapshot'
  1. 在源端环境Linuxshell中运行「snapshot_export.sh」文件,批量完成快照传输。

snapshot_export.sh

#!/bin/bash
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240416000001_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240416000002_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240416000003_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240416000004_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240724000001_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240725000001_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240725000002_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240725000003_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20240730000001_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20250522000001_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot TASK20250523000001_20250726_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot bao_shui_whid_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot channel_to_sale_store_code_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot color_inv_bonded_target_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot color_sku_dim_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dim_week_date_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_channel_erp_product_relation_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_oms_order_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_product_sku_exist_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_search_click_event_data_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_search_event_data_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_utm_visitor_product_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_utm_visitor_session_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot dwd_utm_visitor_user_log_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot erp_sku_color_id_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot erp_sku_month_start_qty_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot erp_sku_to_channel_sku_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_delivery_confirm_error_log_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_delivery_confirm_relation_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_delivery_lock_relation_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_delivery_order_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_delivery_order_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_delivery_order_lock_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_oms_order_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_oms_order_delivery_relation_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot flink_poas_translate_channel_order_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_add_friends_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_add_inchat_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_cart_favorite_need_reflush_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_channel_member_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_channel_member_rk_acc_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_channel_member_rk_user_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_dim_channel_store_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_dim_product_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_dim_sigoweb_product_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_dim_source_channel_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_item_sku_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_key_word_need_reflush_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_logic_brand_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_logic_brand_rel_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_member_bind_relation_union_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_member_bind_relation_user_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_mes_equipment_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_oms_order_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_oms_order_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_page_view_need_reflush_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_pay_order_first_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_pay_order_first_product_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_pay_order_first_union_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_pay_order_first_user_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_pay_order_upload_union_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_product_sale_plan_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_qy_user_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_register_unionid_rel_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_sdk_channel_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_sku_to_product_code_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_udf_user_chats_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_udf_user_friends_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_user_account_order_record_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_user_brand_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_utm_view_need_reflush_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_utm_view_need_reflush_test_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_view_first_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_view_first_union_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_view_first_user_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_visitor_accum_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_visitor_user_rel_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_wecom_add_friends_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_wecom_add_inchat_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hb_wecom_first_order_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot member_action_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot member_continuous_login_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot member_continuous_sign_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot pay_account_order_table_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot pay_order_upload_table_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot procure_transit_data_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot product_color_dim_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot product_color_sku_dim_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot product_inv_bonded_target_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot product_sale_plan_target_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot rebate_cost_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot sigo_web_product_map_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot sigoweb_brand_id_code_map_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot sku_bonded_stock_cental_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot sku_id_to_code_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot sku_inv_bonded_target_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot sku_sale_msg_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot sku_stock_cental_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot stock_in_transit_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot subscribe_restocked_hb_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot user_view_pay_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot visitor_view_pay_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot whid_color_stock_central_target_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot whid_detail_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot whid_product_stock_central_target_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot whid_sku_stock_cental_target_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot xhs_ascribe_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot yjq_company_info_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot yjq_page_tag_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot yjq_page_type_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot yjq_product_dim_snapshot -copy-to hdfs://10.86.5.94:8020/hbase/  -overwrite
  1. 在目标端环境Linuxshell 中运行「snapshot_restore.sh」文件,批量完成快照恢复。

snapshot_restore.sh

#!/bin/bash
echo "disable 'TASK20240416000001_20250726'; restore_snapshot 'TASK20240416000001_20250726_snapshot'; enable 'TASK20240416000001_20250726'" | hbase shell -n 
echo "disable 'TASK20240416000002_20250726'; restore_snapshot 'TASK20240416000002_20250726_snapshot'; enable 'TASK20240416000002_20250726'" | hbase shell -n 
echo "disable 'TASK20240416000003_20250726'; restore_snapshot 'TASK20240416000003_20250726_snapshot'; enable 'TASK20240416000003_20250726'" | hbase shell -n 
echo "disable 'TASK20240416000004_20250726'; restore_snapshot 'TASK20240416000004_20250726_snapshot'; enable 'TASK20240416000004_20250726'" | hbase shell -n 
echo "disable 'TASK20240724000001_20250726'; restore_snapshot 'TASK20240724000001_20250726_snapshot'; enable 'TASK20240724000001_20250726'" | hbase shell -n 
echo "disable 'TASK20240725000001_20250726'; restore_snapshot 'TASK20240725000001_20250726_snapshot'; enable 'TASK20240725000001_20250726'" | hbase shell -n 
echo "disable 'TASK20240725000002_20250726'; restore_snapshot 'TASK20240725000002_20250726_snapshot'; enable 'TASK20240725000002_20250726'" | hbase shell -n 
echo "disable 'TASK20240725000003_20250726'; restore_snapshot 'TASK20240725000003_20250726_snapshot'; enable 'TASK20240725000003_20250726'" | hbase shell -n 
echo "disable 'TASK20240730000001_20250726'; restore_snapshot 'TASK20240730000001_20250726_snapshot'; enable 'TASK20240730000001_20250726'" | hbase shell -n 
echo "disable 'TASK20250522000001_20250726'; restore_snapshot 'TASK20250522000001_20250726_snapshot'; enable 'TASK20250522000001_20250726'" | hbase shell -n 
echo "disable 'TASK20250523000001_20250726'; restore_snapshot 'TASK20250523000001_20250726_snapshot'; enable 'TASK20250523000001_20250726'" | hbase shell -n 
echo "disable 'bao_shui_whid_detail'; restore_snapshot 'bao_shui_whid_detail_snapshot'; enable 'bao_shui_whid_detail'" | hbase shell -n 
echo "disable 'channel_to_sale_store_code'; restore_snapshot 'channel_to_sale_store_code_snapshot'; enable 'channel_to_sale_store_code'" | hbase shell -n 
echo "disable 'color_inv_bonded_target'; restore_snapshot 'color_inv_bonded_target_snapshot'; enable 'color_inv_bonded_target'" | hbase shell -n 
echo "disable 'color_sku_dim'; restore_snapshot 'color_sku_dim_snapshot'; enable 'color_sku_dim'" | hbase shell -n 
echo "disable 'dim_week_date'; restore_snapshot 'dim_week_date_snapshot'; enable 'dim_week_date'" | hbase shell -n 
echo "disable 'dwd_channel_erp_product_relation'; restore_snapshot 'dwd_channel_erp_product_relation_snapshot'; enable 'dwd_channel_erp_product_relation'" | hbase shell -n 
echo "disable 'dwd_oms_order'; restore_snapshot 'dwd_oms_order_snapshot'; enable 'dwd_oms_order'" | hbase shell -n 
echo "disable 'dwd_product_sku_exist'; restore_snapshot 'dwd_product_sku_exist_snapshot'; enable 'dwd_product_sku_exist'" | hbase shell -n 
echo "disable 'dwd_search_click_event_data'; restore_snapshot 'dwd_search_click_event_data_snapshot'; enable 'dwd_search_click_event_data'" | hbase shell -n 
echo "disable 'dwd_search_event_data'; restore_snapshot 'dwd_search_event_data_snapshot'; enable 'dwd_search_event_data'" | hbase shell -n 
echo "disable 'dwd_utm_visitor_product_detail'; restore_snapshot 'dwd_utm_visitor_product_detail_snapshot'; enable 'dwd_utm_visitor_product_detail'" | hbase shell -n 
echo "disable 'dwd_utm_visitor_session'; restore_snapshot 'dwd_utm_visitor_session_snapshot'; enable 'dwd_utm_visitor_session'" | hbase shell -n 
echo "disable 'dwd_utm_visitor_user_log'; restore_snapshot 'dwd_utm_visitor_user_log_snapshot'; enable 'dwd_utm_visitor_user_log'" | hbase shell -n 
echo "disable 'erp_sku_color_id'; restore_snapshot 'erp_sku_color_id_snapshot'; enable 'erp_sku_color_id'" | hbase shell -n 
echo "disable 'erp_sku_month_start_qty'; restore_snapshot 'erp_sku_month_start_qty_snapshot'; enable 'erp_sku_month_start_qty'" | hbase shell -n 
echo "disable 'erp_sku_to_channel_sku'; restore_snapshot 'erp_sku_to_channel_sku_snapshot'; enable 'erp_sku_to_channel_sku'" | hbase shell -n 
echo "disable 'flink_delivery_confirm_error_log'; restore_snapshot 'flink_delivery_confirm_error_log_snapshot'; enable 'flink_delivery_confirm_error_log'" | hbase shell -n 
echo "disable 'flink_delivery_confirm_relation'; restore_snapshot 'flink_delivery_confirm_relation_snapshot'; enable 'flink_delivery_confirm_relation'" | hbase shell -n 
echo "disable 'flink_delivery_lock_relation'; restore_snapshot 'flink_delivery_lock_relation_snapshot'; enable 'flink_delivery_lock_relation'" | hbase shell -n 
echo "disable 'flink_delivery_order'; restore_snapshot 'flink_delivery_order_snapshot'; enable 'flink_delivery_order'" | hbase shell -n 
echo "disable 'flink_delivery_order_detail'; restore_snapshot 'flink_delivery_order_detail_snapshot'; enable 'flink_delivery_order_detail'" | hbase shell -n 
echo "disable 'flink_delivery_order_lock'; restore_snapshot 'flink_delivery_order_lock_snapshot'; enable 'flink_delivery_order_lock'" | hbase shell -n 
echo "disable 'flink_oms_order'; restore_snapshot 'flink_oms_order_snapshot'; enable 'flink_oms_order'" | hbase shell -n 
echo "disable 'flink_oms_order_delivery_relation'; restore_snapshot 'flink_oms_order_delivery_relation_snapshot'; enable 'flink_oms_order_delivery_relation'" | hbase shell -n 
echo "disable 'flink_poas_translate_channel_order'; restore_snapshot 'flink_poas_translate_channel_order_snapshot'; enable 'flink_poas_translate_channel_order'" | hbase shell -n 
echo "disable 'hb_add_friends'; restore_snapshot 'hb_add_friends_snapshot'; enable 'hb_add_friends'" | hbase shell -n 
echo "disable 'hb_add_inchat'; restore_snapshot 'hb_add_inchat_snapshot'; enable 'hb_add_inchat'" | hbase shell -n 
echo "disable 'hb_cart_favorite_need_reflush'; restore_snapshot 'hb_cart_favorite_need_reflush_snapshot'; enable 'hb_cart_favorite_need_reflush'" | hbase shell -n 
echo "disable 'hb_channel_member_info'; restore_snapshot 'hb_channel_member_info_snapshot'; enable 'hb_channel_member_info'" | hbase shell -n 
echo "disable 'hb_channel_member_rk_acc'; restore_snapshot 'hb_channel_member_rk_acc_snapshot'; enable 'hb_channel_member_rk_acc'" | hbase shell -n 
echo "disable 'hb_channel_member_rk_user'; restore_snapshot 'hb_channel_member_rk_user_snapshot'; enable 'hb_channel_member_rk_user'" | hbase shell -n 
echo "disable 'hb_dim_channel_store'; restore_snapshot 'hb_dim_channel_store_snapshot'; enable 'hb_dim_channel_store'" | hbase shell -n 
echo "disable 'hb_dim_product_info'; restore_snapshot 'hb_dim_product_info_snapshot'; enable 'hb_dim_product_info'" | hbase shell -n 
echo "disable 'hb_dim_sigoweb_product_info'; restore_snapshot 'hb_dim_sigoweb_product_info_snapshot'; enable 'hb_dim_sigoweb_product_info'" | hbase shell -n 
echo "disable 'hb_dim_source_channel'; restore_snapshot 'hb_dim_source_channel_snapshot'; enable 'hb_dim_source_channel'" | hbase shell -n 
echo "disable 'hb_item_sku_info'; restore_snapshot 'hb_item_sku_info_snapshot'; enable 'hb_item_sku_info'" | hbase shell -n 
echo "disable 'hb_key_word_need_reflush'; restore_snapshot 'hb_key_word_need_reflush_snapshot'; enable 'hb_key_word_need_reflush'" | hbase shell -n 
echo "disable 'hb_logic_brand_info'; restore_snapshot 'hb_logic_brand_info_snapshot'; enable 'hb_logic_brand_info'" | hbase shell -n 
echo "disable 'hb_logic_brand_rel'; restore_snapshot 'hb_logic_brand_rel_snapshot'; enable 'hb_logic_brand_rel'" | hbase shell -n 
echo "disable 'hb_member_bind_relation_union'; restore_snapshot 'hb_member_bind_relation_union_snapshot'; enable 'hb_member_bind_relation_union'" | hbase shell -n 
echo "disable 'hb_member_bind_relation_user'; restore_snapshot 'hb_member_bind_relation_user_snapshot'; enable 'hb_member_bind_relation_user'" | hbase shell -n 
echo "disable 'hb_mes_equipment_info'; restore_snapshot 'hb_mes_equipment_info_snapshot'; enable 'hb_mes_equipment_info'" | hbase shell -n 
echo "disable 'hb_oms_order'; restore_snapshot 'hb_oms_order_snapshot'; enable 'hb_oms_order'" | hbase shell -n 
echo "disable 'hb_oms_order_detail'; restore_snapshot 'hb_oms_order_detail_snapshot'; enable 'hb_oms_order_detail'" | hbase shell -n 
echo "disable 'hb_page_view_need_reflush'; restore_snapshot 'hb_page_view_need_reflush_snapshot'; enable 'hb_page_view_need_reflush'" | hbase shell -n 
echo "disable 'hb_pay_order_first'; restore_snapshot 'hb_pay_order_first_snapshot'; enable 'hb_pay_order_first'" | hbase shell -n 
echo "disable 'hb_pay_order_first_product'; restore_snapshot 'hb_pay_order_first_product_snapshot'; enable 'hb_pay_order_first_product'" | hbase shell -n 
echo "disable 'hb_pay_order_first_union'; restore_snapshot 'hb_pay_order_first_union_snapshot'; enable 'hb_pay_order_first_union'" | hbase shell -n 
echo "disable 'hb_pay_order_first_user'; restore_snapshot 'hb_pay_order_first_user_snapshot'; enable 'hb_pay_order_first_user'" | hbase shell -n 
echo "disable 'hb_pay_order_upload_union'; restore_snapshot 'hb_pay_order_upload_union_snapshot'; enable 'hb_pay_order_upload_union'" | hbase shell -n 
echo "disable 'hb_product_sale_plan'; restore_snapshot 'hb_product_sale_plan_snapshot'; enable 'hb_product_sale_plan'" | hbase shell -n 
echo "disable 'hb_qy_user'; restore_snapshot 'hb_qy_user_snapshot'; enable 'hb_qy_user'" | hbase shell -n 
echo "disable 'hb_register_unionid_rel'; restore_snapshot 'hb_register_unionid_rel_snapshot'; enable 'hb_register_unionid_rel'" | hbase shell -n 
echo "disable 'hb_sdk_channel'; restore_snapshot 'hb_sdk_channel_snapshot'; enable 'hb_sdk_channel'" | hbase shell -n 
echo "disable 'hb_sku_to_product_code'; restore_snapshot 'hb_sku_to_product_code_snapshot'; enable 'hb_sku_to_product_code'" | hbase shell -n 
echo "disable 'hb_udf_user_chats'; restore_snapshot 'hb_udf_user_chats_snapshot'; enable 'hb_udf_user_chats'" | hbase shell -n 
echo "disable 'hb_udf_user_friends'; restore_snapshot 'hb_udf_user_friends_snapshot'; enable 'hb_udf_user_friends'" | hbase shell -n 
echo "disable 'hb_user_account_order_record'; restore_snapshot 'hb_user_account_order_record_snapshot'; enable 'hb_user_account_order_record'" | hbase shell -n 
echo "disable 'hb_user_brand_info'; restore_snapshot 'hb_user_brand_info_snapshot'; enable 'hb_user_brand_info'" | hbase shell -n 
echo "disable 'hb_utm_view_need_reflush'; restore_snapshot 'hb_utm_view_need_reflush_snapshot'; enable 'hb_utm_view_need_reflush'" | hbase shell -n 
echo "disable 'hb_utm_view_need_reflush_test'; restore_snapshot 'hb_utm_view_need_reflush_test_snapshot'; enable 'hb_utm_view_need_reflush_test'" | hbase shell -n 
echo "disable 'hb_view_first'; restore_snapshot 'hb_view_first_snapshot'; enable 'hb_view_first'" | hbase shell -n 
echo "disable 'hb_view_first_union'; restore_snapshot 'hb_view_first_union_snapshot'; enable 'hb_view_first_union'" | hbase shell -n 
echo "disable 'hb_view_first_user'; restore_snapshot 'hb_view_first_user_snapshot'; enable 'hb_view_first_user'" | hbase shell -n 
echo "disable 'hb_visitor_accum'; restore_snapshot 'hb_visitor_accum_snapshot'; enable 'hb_visitor_accum'" | hbase shell -n 
echo "disable 'hb_visitor_user_rel'; restore_snapshot 'hb_visitor_user_rel_snapshot'; enable 'hb_visitor_user_rel'" | hbase shell -n 
echo "disable 'hb_wecom_add_friends'; restore_snapshot 'hb_wecom_add_friends_snapshot'; enable 'hb_wecom_add_friends'" | hbase shell -n 
echo "disable 'hb_wecom_add_inchat'; restore_snapshot 'hb_wecom_add_inchat_snapshot'; enable 'hb_wecom_add_inchat'" | hbase shell -n 
echo "disable 'hb_wecom_first_order'; restore_snapshot 'hb_wecom_first_order_snapshot'; enable 'hb_wecom_first_order'" | hbase shell -n 
echo "disable 'member_action_detail'; restore_snapshot 'member_action_detail_snapshot'; enable 'member_action_detail'" | hbase shell -n 
echo "disable 'member_continuous_login'; restore_snapshot 'member_continuous_login_snapshot'; enable 'member_continuous_login'" | hbase shell -n 
echo "disable 'member_continuous_sign'; restore_snapshot 'member_continuous_sign_snapshot'; enable 'member_continuous_sign'" | hbase shell -n 
echo "disable 'pay_account_order_table'; restore_snapshot 'pay_account_order_table_snapshot'; enable 'pay_account_order_table'" | hbase shell -n 
echo "disable 'pay_order_upload_table'; restore_snapshot 'pay_order_upload_table_snapshot'; enable 'pay_order_upload_table'" | hbase shell -n 
echo "disable 'procure_transit_data'; restore_snapshot 'procure_transit_data_snapshot'; enable 'procure_transit_data'" | hbase shell -n 
echo "disable 'product_color_dim'; restore_snapshot 'product_color_dim_snapshot'; enable 'product_color_dim'" | hbase shell -n 
echo "disable 'product_color_sku_dim'; restore_snapshot 'product_color_sku_dim_snapshot'; enable 'product_color_sku_dim'" | hbase shell -n 
echo "disable 'product_inv_bonded_target'; restore_snapshot 'product_inv_bonded_target_snapshot'; enable 'product_inv_bonded_target'" | hbase shell -n 
echo "disable 'product_sale_plan_target'; restore_snapshot 'product_sale_plan_target_snapshot'; enable 'product_sale_plan_target'" | hbase shell -n 
echo "disable 'rebate_cost'; restore_snapshot 'rebate_cost_snapshot'; enable 'rebate_cost'" | hbase shell -n 
echo "disable 'sigo_web_product_map'; restore_snapshot 'sigo_web_product_map_snapshot'; enable 'sigo_web_product_map'" | hbase shell -n 
echo "disable 'sigoweb_brand_id_code_map'; restore_snapshot 'sigoweb_brand_id_code_map_snapshot'; enable 'sigoweb_brand_id_code_map'" | hbase shell -n 
echo "disable 'sku_bonded_stock_cental_detail'; restore_snapshot 'sku_bonded_stock_cental_detail_snapshot'; enable 'sku_bonded_stock_cental_detail'" | hbase shell -n 
echo "disable 'sku_id_to_code'; restore_snapshot 'sku_id_to_code_snapshot'; enable 'sku_id_to_code'" | hbase shell -n 
echo "disable 'sku_inv_bonded_target'; restore_snapshot 'sku_inv_bonded_target_snapshot'; enable 'sku_inv_bonded_target'" | hbase shell -n 
echo "disable 'sku_sale_msg'; restore_snapshot 'sku_sale_msg_snapshot'; enable 'sku_sale_msg'" | hbase shell -n 
echo "disable 'sku_stock_cental_detail'; restore_snapshot 'sku_stock_cental_detail_snapshot'; enable 'sku_stock_cental_detail'" | hbase shell -n 
echo "disable 'stock_in_transit_detail'; restore_snapshot 'stock_in_transit_detail_snapshot'; enable 'stock_in_transit_detail'" | hbase shell -n 
echo "disable 'subscribe_restocked_hb'; restore_snapshot 'subscribe_restocked_hb_snapshot'; enable 'subscribe_restocked_hb'" | hbase shell -n 
echo "disable 'user_view_pay'; restore_snapshot 'user_view_pay_snapshot'; enable 'user_view_pay'" | hbase shell -n 
echo "disable 'visitor_view_pay'; restore_snapshot 'visitor_view_pay_snapshot'; enable 'visitor_view_pay'" | hbase shell -n 
echo "disable 'whid_color_stock_central_target'; restore_snapshot 'whid_color_stock_central_target_snapshot'; enable 'whid_color_stock_central_target'" | hbase shell -n 
echo "disable 'whid_detail'; restore_snapshot 'whid_detail_snapshot'; enable 'whid_detail'" | hbase shell -n 
echo "disable 'whid_product_stock_central_target'; restore_snapshot 'whid_product_stock_central_target_snapshot'; enable 'whid_product_stock_central_target'" | hbase shell -n 
echo "disable 'whid_sku_stock_cental_target'; restore_snapshot 'whid_sku_stock_cental_target_snapshot'; enable 'whid_sku_stock_cental_target'" | hbase shell -n 
echo "disable 'xhs_ascribe'; restore_snapshot 'xhs_ascribe_snapshot'; enable 'xhs_ascribe'" | hbase shell -n 
echo "disable 'yjq_company_info'; restore_snapshot 'yjq_company_info_snapshot'; enable 'yjq_company_info'" | hbase shell -n 
echo "disable 'yjq_page_tag'; restore_snapshot 'yjq_page_tag_snapshot'; enable 'yjq_page_tag'" | hbase shell -n 
echo "disable 'yjq_page_type'; restore_snapshot 'yjq_page_type_snapshot'; enable 'yjq_page_type'" | hbase shell -n 
echo "disable 'yjq_product_dim'; restore_snapshot 'yjq_product_dim_snapshot'; enable 'yjq_product_dim'" | hbase shell -n