The data catalog is your unified metadata workbench for data analysis. It integrates metadata from various data sources, such as MaxCompute, Hologres, and DLF. This lets you complete all preparatory work, such as creating tables and managing views, without leaving DataWorks so you can focus on your core analysis.
Access the feature
Log on to DataWorks DataAnalysis, switch to the target region, and then click Go to DataAnalysis.
If Switch to New DataAnalysis appears in the navigation bar, click it to switch to the new version.
If Return to Legacy DataAnalysis appears in the navigation bar, you are in the new version.
Add a data catalog
If you are a new user of DataAnalysis, follow the instructions for new users. Otherwise, follow the instructions for existing users.
Add a data catalog for new users
You can customize the catalog tree to meet your needs and improve your experience.
In the data catalog, find the data source type that you want to add. Click the
icon next to the data source type name to open the page for adding a data catalog.-
On the Add Data Catalog page, find the instance or data source that you want to add. In the Operation column, click Add.
You can remove an instance or data source if you no longer need it.
Add a data catalog for existing users
You can add system-recommended catalogs and data sources from your workspace to the data catalog. This lets you quickly view the data tables in the corresponding catalog, view table schemas, and generate SQL query statements.
Click the icon in the upper-right corner of the catalog to open the Add Data Catalog page.
-
DataMap - Metadata: Adds table metadata collected in Data Map. Each data source or computing resource can be added as a dataset.
-
DataMap - Data Album: Adds data albums from Data Map that group tables by specific subjects. Each data album can be added as a dataset.
-
My Favorites: Adds tables that you have favorited in the data catalog.
-
My MaxCompute Tables: Adds all MaxCompute tables owned by the currently logged-in account.
-
Public Data: Adds public datasets provided by DataWorks, allowing you to quickly generate test SQL queries for EMR Spark SQL, MaxCompute, and Hologres.
-
Alibaba Cloud Bill Data: Adds your Alibaba Cloud bill data. This lets you perform bill analysis.
You can add a maximum of 12 datasets. If you reach the limit, you must remove a dataset before you can add a new one.
Manage a data catalog
The following information applies only to the data catalog for new users.
You can use the data catalog to manage data objects such as tables, views, external tables, resources, and functions.
MaxCompute: Manage a MaxCompute data catalog.
Hologres: Manage a Hologres data catalog.
DLF: Manage a DLF catalog.
Generate query SQL
You can quickly generate a query SQL statement based on a data table, and then configure and execute the statement. For more information, see Create an SQL query.
In the data catalog, find the data source that you want to query. Click the
icon next to the data source and find the table that you want to query in the Table list.-
Right-click the target table and select Generate SQL File for Query.
You can adjust the generated SQL script on the SQL editing page.
-
After you modify the script, click Run Configuration in the right-side panel. Set the Computing Resource and Script Parameters, then run the query.
Appendix
MaxCompute authentication and authorization
If you use a Resource Access Management (RAM) user or a RAM role to view MaxCompute data in the data catalog, you must first obtain the required MaxCompute permissions. If the Layer 3 model is enabled for the added MaxCompute data source or project, you must also grant the RAM user or RAM role permission to view schema metadata.
If a MaxCompute project contains multiple schemas, you must grant metadata permissions for all schemas to display the complete schema list on the project details page in the data catalog.
Grant permissions to a RAM user:
GRANT DESCRIBE ON SCHEMA <Schema_Name> TO USER RAM$<Alibaba_Cloud_Account_Name>:<RAM_User_Name>;Grant permissions to a RAM role:
GRANT DESCRIBE ON SCHEMA <Schema_Name> TO USER `RAM$<Alibaba_Cloud_Account_Name>:role/<RAM_Role_Name>`;