In HBase, data is stored in tables with rows and columns. This model is similar to a relational database (RDBMS), but it is loosely structured and functions as a multi-dimensional, sorted map. The index sort key consists of a row, a column, and a timestamp. An HBase table can be considered a sparse, distributed, persistent, and multi-dimensional sorted map.
Glossary
Namespace: A logical grouping of tables, similar to the concept of a database in a relational database. Namespaces help you isolate resources and data in multi-tenant scenarios.
Table: HBase organizes data into tables. An HBase table consists of multiple rows.
Row: A row in HBase contains a rowkey and one or more columns with their associated values. Rows are sorted alphabetically by rowkey when they are stored. Therefore, the design of the rowkey is very important. The goal is to store related rows close to each other. A common rowkey pattern is to use website domains. If your rowkeys are domain names, you should store them in reverse order. This keeps all domains from a single site close to each other, rather than being distributed based on the first letter of the subdomain.
Column: A column in HBase consists of a column family and a column qualifier, which are separated by a colon (:) character.
Column Family: For performance reasons, a column family physically groups a set of columns and their values. In HBase, each column family has a set of storage properties, such as whether its values should be cached in memory, how data is compressed, or how its rows are encoded. Every row in a table has the same column families, but a given row might not store anything in a given column family. Because a column family affects the physical storage structure of HBase, it cannot be easily modified after it is defined. However, you can add or delete column qualifiers and their corresponding values dynamically.
Column Qualifier: A column qualifier is added to a column family to provide an index for a given piece of data. For a column family named content, one column qualifier might be content:html, and another might be content:pdf. Although column families are fixed when the table is created, column qualifiers are variable and can differ greatly between rows.
Cell: A cell is the combination of a row, column family, and column qualifier. It contains a value and a timestamp, which represents the version of that value.
Timestamp: A timestamp is written with each value and is the identifier for a given version of that value. By default, the timestamp represents the time on the RegionServer when the data was written. You can specify a different timestamp value when you put data into a cell.
Conceptual view
The example in this section is a modified version of the example from the BigTable paper. This example has a table named webtable. It contains two rows (com.test.www and com.example.www) and three column families: contents, anchor, and people. In this example, for the first row (com.test.www), the anchor family contains two columns (anchor:example.com and anchor:test.ca), and the contents family contains one column (contents:html). This example contains five versions of the row with the rowkey com.test.www, and one version of the row with the rowkey com.example.www. The contents:html column qualifier contains the entire HTML for a given website. The column qualifiers for the anchor column family each contain an external site that links to the site represented by the row, and the text used in the anchor for that link. The people column family represents people associated with the website.
Column names: By convention, a column name consists of its column family prefix and a qualifier. For example, the column contents:html consists of the column family contents and the qualifier html, separated by a colon (:).
The webtable table is shown below:
Row Key | Time Stamp | ColumnFamily contents | ColumnFamily anchor | ColumnFamily people |
"com.test.www" | T9 | (-) | anchor:example1.com = "CNNtest" | (-) |
"com.test.www" | T8 | (-) | anchor:test.ca = "CNNtest.com" | (-) |
"com.test.www" | T6 | contents:html = "..." | (-) | (-) |
"com.test.www" | T5 | contents:html = "..." | (-) | (-) |
"com.test.www" | T3 | contents:html = "..." | (-) | (-) |
"com.example.www" | T5 | contents:html = "..." | (-) | people:author = "John Doe" |
Cells shown as - in this table do not take up space or actually exist in HBase. This is what makes HBase "sparse". The table view is not the only possible way to view HBase data, nor is it the most accurate. The following represents the same information as a multi-dimensional map. This is only a simulation for illustrative purposes and may not be strictly accurate.
{
"com.test.www": {
contents: {
t6: contents:html: "<html>..."
t5: contents:html: "<html>..."
t3: contents:html: "<html>..."
}
anchor: {
t9: anchor:example1.com = "CNNtest"
t8: anchor:test.ca = "CNNtest.com"
}
people: {}
}
"com.example.www": {
contents: {
t5: contents:html: "<html>..."
}
anchor: {}
people: {
t5: people:author: "John Doe"
}
}
}Physical view
Although tables are viewed as a sparse set of rows conceptually, they are physically stored by column family. You can add new column qualifiers (column_family:column_qualifier) to an existing column family at any time.
ColumnFamily anchor:
Row Key | Time Stamp | ColumnFamily anchor |
"com.test.www" | T9 | anchor:example1.com = "CNNtest" |
"com.test.www" | T8 | anchor:test.ca = "CNNtest.com" |
ColumnFamily contents:
Row Key | Time Stamp | ColumnFamily contents |
"com.test.www" | T6 | contents:html = "..." |
"com.test.www" | T5 | contents:html = "..." |
"com.test.www" | T3 | contents:html = "..." |
The empty cells shown in the conceptual view are not stored. Therefore, a request for the value of the contents:html column at timestamp t8 returns no value. Similarly, a request for a value for anchor:test.ca at timestamp t9 returns no value. However, if you do not provide a timestamp, the latest value for a specific column is returned. Because timestamps are stored in descending order, the most recent version is the first one found when multiple versions exist. Therefore, if you do not specify a timestamp, a request for the values of all columns in the row com.test.www returns the following: the value of contents:html at timestamp t6, the value of anchor:example1.com at timestamp t9, and the value of anchor:test.ca at timestamp t8.
Data sorting
All data model operations in HBase return data in a sorted order. Data is sorted first by row, then by ColumnFamily, then by column qualifier, and finally by timestamp. Timestamps are sorted in reverse order, so the newest record is returned first.
Column metadata
Column metadata is not stored outside of the KeyValue instances within a ColumnFamily. Therefore, although HBase can support many columns per row and maintain a heterogeneous set of columns across rows, you are responsible for tracking the column names.
The only way to obtain a complete set of columns that exist for a ColumnFamily is to process all rows.
ACID
ACID is an acronym for the four basic elements of a correctly executed database transaction: atomicity, consistency, isolation, and durability.
HBase supports ACID for single-row operations. This means that Put operations on the same row are guaranteed to be fully ACID-compliant.