Primary/secondary disaster recovery
ApsaraDB for HBase ensures service and data availability through features such as automatic master node switchovers, two data replicas, and fast automatic failovers. However, a single HBase cluster cannot meet high availability (HA) requirements in the event of zone-level power outages, network failures, or other major faults. To address this, ApsaraDB for HBase provides a cross-zone, primary/secondary disaster recovery solution. This solution offers one-click, transparent, and fast switchover capabilities for hybrid scenarios, such as cloud-to-cloud, cloud-to-on-premises, and on-premises-to-on-premises deployments.
Benefits
Supports hybrid primary/secondary setups for ApsaraDB for HBase, EMR HBase, and on-premises HBase.
Requires zero code changes and supports multiple versions, including 1.x and 2.x.
Provides automatic data synchronization management and efficient two-way synchronization with latency in the hundreds of milliseconds.
Features automatic fault tolerance.
Includes a primary/secondary console that displays important information such as synchronization latency, number of client connections, and primary/secondary status.
Architecture
Primary and secondary HBase instances are deployed in different zones. Lindorm Tunnel Service (LTS) provides real-time, incremental, two-way data synchronization between the primary and secondary instances. Primary/secondary switchovers are also performed using LTS. You can access the HBase instance through `alihbase-connector`. The `alihbase-connector` listens for switchover events from the ZooKeeper clusters of the primary and secondary instances to perform the instance switchover. The switchover process is transparent to the application.
Limits
Currently, only cross-zone deployment within the same VPC is supported.
This feature is available on BDS services purchased after September 18, 2019.
Data synchronization between primary and secondary instances currently uses the eventual consistency model. The strong consistency model is not yet supported.
Currently, only HBase service switchovers are supported. Phoenix and Solr switchovers are not supported.
Procedure
Purchase an HBase cluster. For more information, see Purchase a cluster. If you already have a cluster, you can skip this step.
Log on to the ApsaraDB for HBase console.
On the Clusters page, click a cluster name to open its details page.
On the instance information page, in the navigation pane on the left, choose Database Connection to view the VPC ID.
Purchase a Lindorm Tunnel Service (LTS) instance. If you already have an instance, you can skip this step. Otherwise, go to the purchase page to purchase an LTS instance. The LTS instance must be in the same VPC as the HBase instance but in a different zone. If no other zones are available, you can purchase the LTS instance in the same zone as the secondary HBase instance. The following figure shows a purchased LTS instance.
Add to the whitelist
Add the client to the whitelists of the primary and secondary HBase instances and the LTS instance. Otherwise, the client cannot access the instances. For more information about how to configure a whitelist, see Set a whitelist.
Configure network access for on-premises instances
If you associate an on-premises instance, LTS must be able to access it. This section uses an EMR HBase instance as an example. The process is the same if you use your own ECS instances to build an HBase cluster. You must grant permissions to the LTS nodes in the security group of the ECS instance where HBase is deployed. On the LTS page, choose **Configuration Management** and click **Generate Security Group Rules**. A file named `xxxx_securityrule.json` is downloaded to your local computer.In the EMR console, go to **Cluster Management**, find the target cluster, and click **Details**.Find the security group and click its name. If you have a fully self-managed HBase cluster, you can find the security group of the ECS instance.In the security group, choose **Import**.Select the `xxxx_securityrule.json` file that you downloaded and start importing the security group rules.After the import is complete, LTS has permission to access the relevant ports of the on-premises HBase instance. You must add the host information of the on-premises HBase instance nodes in LTS. On the LTS page, choose **Configuration Management**, configure the `bds.conf.hosts` parameter, and then click **Modify**.To obtain the hosts information for EMR, you can log on to an ECS instance in the EMR cluster and view the `/etc/hosts` file to find the IP addresses and domain names. The domain names typically follow the format `emr-header-x.cluster-xxx` or `emr-worker-x.cluster-xxx`.
Add an HA instance in the console
Go to the **Clusters** page and click the newly purchased LTS instance.Choose **HA Management** and add an HA instance.If both the primary and secondary instances are ApsaraDB for HBase instances, you can select their names from the drop-down list. For ApsaraDB for HBase Performance-enhanced Edition instances, you must enter a username and password. If the secondary instance is a new instance, the HA service supports one-click migration. You can set the synchronization type to **Instance Level** to synchronize all tables in the primary instance, or **Table Level** to specify the tables to synchronize. If you select **Table Level**, you must enter the table names separated by commas. If the secondary instance is an existing instance and real-time synchronization is already enabled, you can select **No Sync** to skip data synchronization.If the primary or secondary instance is an on-premises instance, you must enter its connection information. This information includes the connection endpoint in the `zk1,zk2:port:/hbase` format, a user-defined instance name, the HDFS address in the `hdfs://namenode1:port,namenode2:port` format, and the HBase root directory, which is the `hbase.rootdir` configuration in the `hbase-site.xml` file.Click **Confirm**. After the instance is added, the new HA instance and its endpoint are displayed.
Switchover management
Click **Manage** to open the management page. You can set the username and password by selecting Database Connection > Reset UI Access Password. If you selected data synchronization, the management page displays the real-time, two-way data synchronization points for the primary and secondary instances. You can also view the progress of historical data synchronization. If historical data synchronization is complete and the real-time points are updating correctly, the data migration is successful.You can click **Switch** to perform a switchover between instances. After a successful operation, you can see that the **Active** status has changed.
Access using the Java API
Prerequisites
On the **HA Management** page, obtain the HA endpoint and HA instance ID.
To use the client for HBase 2.x, add the alihbase-endpoint dependency.
<dependency>
<groupId>com.aliyun.hbase</groupId>
<artifactId>alihbase-endpoint</artifactId>
<version>2.8.3</version>
</dependency>Configure client parameters
You can configure client parameters in one of the following two ways.
Method 1: Configuration file
<configuration>
<!-- HA endpoint -->
<property>
<name>hbase.zookeeper.quorum</name>
<value>HOST:PORT</value>
</property>
<!--
HA instance ID
-->
<property>
<name>haclient.cluster.id</name>
<value>ha-xxxxxx</value>
</property>
<!--
To connect to ApsaraDB for HBase Performance-enhanced Edition, set the username and password. The default for both is root. Adjust them as needed.
-->
<property>
<name>hbase.client.username</name>
<value>testuser</value>
</property>
<property>
<name>hbase.client.password</name>
<value>password</value>
</property>
</configuration>You can obtain the configuration information from the **HA Management** page.
Method 2: Code
Create a `Configuration` object in your code and add the required configurations.
// Create a new Configuration object.
Configuration conf = HBaseConfiguration.create();
// HA endpoint
conf.set("hbase.zookeeper.quorum", "HOST:PORT");
// HA instance ID
conf.set("haclient.cluster.id", "ha-xxxxxx");
// Set the username and password. The default for both is root. Adjust them as needed.
conf.set("hbase.client.username", "testuser")
conf.set("hbase.client.password", "password")
// One-click switchover API
ClusterSwitcher clusterSwitcher = new ClusterSwitcher(conf);
// Switch to standby.
clusterSwitcher.switchToStandby();Create a connection
Create a `Connection` object using the `conf` configuration to access HBase.
// Create an HBase connection. This only needs to be done once during the program's lifecycle. The connection is thread-safe and can be shared by all threads.
// After the program finishes, close the Connection object to prevent connection leaks.
// You can also use a try-finally block to prevent leaks.
Connection connection = ConnectionFactory.createConnection(conf);Create a table
Using `haclient`, you can use the API to create tables on both the primary and secondary instances. The table is created on the active instance in real time. An asynchronous task is generated to create the table on the standby instance, and the real-time sync task is updated. You can view the status on the HA instance management page.