The Data Catalog provides a unified interface to manage and operate DLF Catalog metadata. This topic describes how to create and manage database and table metadata in the Data Catalog.
Go to the DLF Catalog page
Go to the Workspaces page in the DataWorks console. In the top navigation bar, select a desired region. Find the desired workspace and choose in the Actions column.
In the navigation pane on the left, click the
icon. In the Data Directory tree, click DLF Catalog to open the DLF Catalog management page.
Add a DLF Catalog
On the DLF Catalog management page, add an existing DLF Catalog to the list.
Click the
icon to the right of DLF Catalog to open the Add DLF Catalog page.On the tab that corresponds to your DLF version, add the DLF Catalog to the list.
DLF version
Instructions
DLF 1.0
On the DLF 1.0 tab, click Create DLF Catalog to go to the Data Lake Formation 1.0 console and create a DLF Catalog. After you create the catalog, add it to the list.
Add a single catalog: In the list on the DLF 1.0 tab, find the DLF Catalog that you want to add and click Add in the Operation column.
Add multiple catalogs: In the list on the DLF 1.0 tab, select multiple DLF Catalogs and click Batch Add below the list.
DLF 2.0
Creating new DLF Catalogs is not supported.
(Recommended) DLF 2.5 and later
On the DLF 2.5 and later tab, click Create DLF Catalog to go to the Data Lake Formation 2.5 console and create a DLF Catalog. After you create the catalog, add it to the list.
Add a single catalog: In the list on the DLF2.5 tab, find the DLF Catalog that you want to add and click Add in the Operation column.
Add multiple catalogs: In the list on the DLF2.5 tab, select multiple DLF Catalogs and click Batch Add below the list.
Manage DLF Catalog data
Add and manage databases and tables in the DLF Catalog.
Manage databases
Find the target DLF Catalog and click the
icon to the right of the DLF Catalog.On the Create Database page, enter the Database Name that you want to create.
Click Save in the upper toolbar to complete the creation.
View databases: After a database is added to the DLF Catalog, you can click the
icon to the left of the DLF Catalog to view the added databases.Delete a database: If you no longer need to manage a database, right-click the database and select Delete from the context menu.
ImportantDeleted databases cannot be recovered. Proceed with caution.
Before you delete a database, delete all tables in the database first.
Manage tables
Click the
icon to the right of the target database to go to the Create Table page.Use one of the following methods to generate the basic information and column information for the table.
Copilot table creation:
Click Copilot table creation in the upper toolbar to open the Copilot Chat interface.
Enter a table creation instruction (for example,
Create a user table).Click Generate and Replace. The system generates a default table name and column information based on your instruction.
If the table name and columns meet your expectations, click Accept.
NoteIf you need to modify some table information, click Accept first and then manually edit the system-generated table information.
Manual table creation:
Create a table based on the following parameter descriptions.
Parameter
Description
Basic information
Specify Table Name, Description, and other information.
Column information
Edit column names and column comments.
Manual editing: Click Insert above the column list, specify the number of rows to insert, and then edit column names, data types, and other information.
Copilot intelligent editing: Click Generate Field or Generate field descriptions above the column list. The system generates relevant columns and descriptions based on the table name and description that you specified.
(Optional) Configure partition information.
To create a partitioned table, go to the Partition Field section, specify the number of partition column rows (multiple partitions are supported), and click Insert. In the partition column list, configure the partition Field Name, Data Type, and other information based on your business requirements.
(Optional) Configure advanced settings.
DLF version
Parameter
Description
DLF 2.5 and later
DLF 2.0
Custom properties
You can configure custom property parameters to set the table storage location, storage format, and other information. For more information, see Create a table in Data Lake Formation.
DLF 1.0
Storage location
Click Change next to the storage location. In the OSS Path dialog, select a storage location for the table. Metadata is stored in Object Storage Service (OSS) to ensure data security, reliability, and centralized management.
Data format
You can set the storage format to CSV, PARQUET, ORC, AVRO, JSON, or SELE_DEFINE. The system automatically defines the input and output formats, as well as the serialization and deserialization methods based on the selected storage format.
CSV: A comma-separated text file format suitable for simple data structures.
PARQUET: A columnar storage format with a high compression ratio, suitable for big data analytics.
ORC: An optimized columnar storage format with excellent performance that supports complex data types.
AVRO: A binary format that supports schema evolution, suitable for dynamic data structures.
JSON: Supports nested structures, suitable for semi-structured data.
SELE_DEFINE: Allows you to customize serialization and deserialization logic.
After the configuration is complete, click Publish in the upper toolbar to create the table.
View tables: After you create a table in the DLF Catalog, click the
icon to the left of the DLF Catalog, and then click Table to view basic table information. You can also click a specific table name to view the table Details and Basic information.Delete a table: Right-click the table that you want to delete and select Delete from the context menu.
ImportantDeleted tables cannot be recovered. Proceed with caution.
View and remove a DLF Catalog
If you no longer need a DLF Catalog, you can view or remove it.
View a catalog.
After an instance is added to the DLF Catalog list, you can click the
icon to the left of the DLF Catalog to view the added DLF Catalogs.Hover over a DLF Catalog to view its Version, Creation time, and other information.
Remove a catalog.
If you no longer need to manage a DLF Catalog, right-click the DLF Catalog and select Remove from the context menu.