Databases, tables, and functions
Metadata management is a critical part of building a data lake. Effective, centralized metadata management helps you maximize the value of your data assets. This topic describes how to manage databases, tables, and functions in Data Lake Formation (DLF).
Databases
Create a database
Log on to the Data Lake Formation (DLF) console.
In the left-side navigation pane, select .
Click the Database tab, select the target Catalogs, and then click Create Database.
Configure the parameters for the database and click OK.
Parameter
Description
Catalog
Select the data catalog where you want to create the database.
Name
Enter a name for the database.
Description
Optional. Enter a description for the database.
Select the path.
Specify an Object Storage Service (OSS) path to store the metadata. Storing metadata in OSS ensures data security and reliability, and simplifies management and maintenance.
NoteOnly OSS buckets that use Standard storage are supported. If a Standard storage bucket does not exist in the current region, create one in the OSS console.
View database details
Click the Database tab.
In the database list, click a database name to view its Basic Information.
Click the corresponding tabs to view the Tables, Functions, Data Permission, Overview, and Storage Rule for the database.
Edit a database
Click the Database tab and select the target Catalogs.
In the Actions column for the database, click Modify.
Modify the information and click Update.
Description: Enter a description for the database.
Select the path.: Enter the location of the database.
Delete a database
Click the Database tab and select the target Catalogs.
In the Actions column for the database, click Delete.
In the confirmation dialog box, click Delete.
DLF creates the
defaultdatabase, which cannot be deleted. We recommend that you use other databases for production environments.Deleting a database in the DLF console removes its metadata but does not delete the data stored in OSS. To delete the underlying data, you must run a
Drop Databaseoperation from E-MapReduce or MaxCompute.
Tables
Create a table
After you create a database, click the Data Table tab. Select the target Catalogs and Database Name, and then click Create Table.
Configure the parameters for the table and click OK.
Parameter
Description
Table Name
Enter a name for the table.
Catalog
Select the data catalog for the table.
Database:
Select a database within the data catalog.
Table Description
Optional. Enter a description for the table.
Data Storage Location
Select the storage location for the table's data.
The recommended default storage location is
oss://[database_location]/[table_name].Format and Serialization
Select the table's data format. Supported formats include Avro, CSV, JSON, Parquet, and ORC.
Delimiter
Optional. If you select the CSV format, specify a delimiter.
Common Column
Manually define the table's columns and partition keys. Specify the name, data type, and description for each.
Partition Key Column
View table details
Click the Data Table tab.
In the table list, click a table name to view its Basic Information.
Click the corresponding tabs to view the Data Permission, Overview, Version Management, and Storage Rule for the table.
For more information about the Overview of a table, see Table - data overview.
Edit a table
Click the Data Table tab, and select the target Catalogs and Database Name.
In the Actions column for the table, click Modify.
Modify the table information and click Update.
Delete a table
Click the Data Table tab, and select the target Catalogs and Database Name.
In the Actions column for the table, click Delete.
In the confirmation dialog box, click Delete.
Deleting a table in the DLF console removes its metadata but does not delete the data stored in OSS. To delete the underlying data, you must run a Drop Table operation from E-MapReduce or MaxCompute.
Functions
Create a function
In the left-side navigation pane, select .
Click the Function tab. Select the target Catalogs and Database Name, and then click Create Function.
Configure the parameters for the function and click OK.
Parameter
Description
Function Name
Enter a name for the function.
Function Type
Only JAVA is supported.
Catalog
Select the data catalog for the function.
Database:
Select a database within the data catalog.
Class Name
Optional. Enter the class name.
Resource URI
Optional. Click Add, and then enter the Type and URI. Supported resource types are JAR, FILE, and ARCHIVE.
View function details
Click the Function tab, and select the target Catalogs and Database Name.
In the function list, click a function name to view its Basic Information.
Edit a function
Click the Function tab, and select the target Catalogs and Database Name.
In the Actions column for the function, click Modify.
Modify the Class Name and click OK.
Delete a function
Click the Function tab, and select the target Catalogs and Database Name.
In the Actions column for the function, click Delete.
In the confirmation dialog box, click Delete.