DocsICP FilingConsole
Documentation
Home Page DataWorks User Guide Data Governance Data Map Metadata retrieval

Metadata retrieval

Updated at:
Copy as MD
Product
My Favorites

DataWorks Data Map is a centralized data asset management and discovery platform. It provides a comprehensive view of your enterprise data landscape and lets you locate the data you need quickly and accurately. Core features include:

  • Data Overview: Provides statistics about your data assets for a quick overview of their current state.

  • Global keyword search: Allows you to quickly locate tables, fields, or business terms of interest from a single entry point.

  • Multi-dimensional filtering and browsing: Enables refined and exploratory data discovery by combining various conditions.

Limits

  • Tenants with an existing PAI workspace can search for and view related AI assets, such as datasets, AI models, algorithm tasks, and model services.

  • Code search is available only to users of DataWorks Standard Edition or later.

Access data map

Log on to the DataWorks console. In the target region, click Data Governance > Data Map in the left-side navigation pane. On the page that appears, click Go to Data Map.

Data overview: Gain macro-level insights

The Data Overview page provides a high-level summary of how your data sources are used.

  1. On the Data Map Homepage, click Data Overview next to Metadata Collection.

  2. On the Data Overview page, view all engine resources in the current region.

    1. MaxCompute key metrics:

      Name

      Description

      Total Number of Projects

      The total number of MaxCompute projects in the current region. This is near-real-time data.

      Total Tables

      The total number of MaxCompute tables in the current region. This data is collected offline and has a T+1 delay.

      Storage

      The total logical storage size of all tables in the current region, including temporary files generated by scheduling tasks and storage that has not been released after tables are deleted. This data is collected offline and has a T+1 delay.

      Total APIs

      The total number of MaxCompute APIs that have been published to API Gateway in the current region.

      Storage Trend Chart

      A trend chart of the total logical storage of MaxCompute projects in the current region. This data includes temporary files generated by scheduling tasks and storage that has not been released after tables are deleted. This data is collected offline and has a T+1 delay.

      Top Projects by Storage

      A ranking of MaxCompute projects by logical storage size in the current region. This data is collected offline and has a T+1 delay. You can click a project to view its metadata.

      Important

      In addition to table storage, project storage also includes resource storage, recycle bin storage, and other system file storage. Therefore, project storage is larger than table storage.

      Top Tables by Storage

      A ranking of MaxCompute tables by size. This data is collected offline and has a T+1 delay. You can click a table to view its metadata.

      Important

      Table storage is calculated based on logical storage rather than physical storage.

      Most Frequently Viewed Tables

      The most frequently viewed tables in the last 30 days, ranked by PV (page views) of table detail pages in Data Map. This is near-real-time data.

    2. Other engine metrics: Displays information such as the total number of databases, total number of tables, and total number of clusters.

How to find data: From search to discovery

Data Map provides two complementary search methods to meet your different data discovery needs.

Global keyword search

When you have clear clues about the data you are looking for, global search is the most efficient method. The homepage also provides quick access to Recent Views, Followed Collections, and Managed Collections.

  1. Go to the search bar: At the top of the Data Map homepage, locate the prominent search box.

  2. Enter keywords: Select a metadata type and enter a table name, field name, Chinese comment, or other Keyword. For example, enter "user", "order", or "user_info".

  3. Run the search: Press Enter. The system returns a list of all related data assets. Click an item to view its details.

Multi-dimensional filtering and browsing

When you need to narrow down results more precisely or explore data by business category, go to the search page for advanced search. The global keyword search bar is still available at the top of the navigation bar.

  1. Go to the search page: Click the search icon image in the left-side navigation pane to open the advanced search page.

  2. Combine filter conditions: On the left side of the page, select different filter conditions to progressively narrow down results:

    • Type: Select a metadata type. Supported types include Table, Code, Index, API, Dataset, Workspace, and more. If your tenant has a PAI workspace, you can also filter by AI asset types such as AI Model, Algorithm Task, and Model Service (currently in beta).

    • Data source: When you select tables, you can limit the search to a specific engine, such as MaxCompute, EMR Hive, or Hologres.

      Important

      If the number of data sources exceeds a certain threshold, click the More button in the upper-right corner to display hidden data sources.

    • Filter conditions: You can filter by project, owner, environment, cluster, data catalog, database, and more.

      The available filter conditions vary depending on the metadata type.
      Note

      In addition to basic attributes, table objects can be filtered by tags and categories, while workspaces and datasets support filtering by custom attributes.

      • Tags: You can filter by tags applied during data governance. For more information, see Tags.

      • Category: You can filter by business category. For more information, see Manage categories.

      • Custom attributes: You can filter by enumeration-type custom attributes. For more information, see Custom attributes.

Usage example:

I want to find all production tables in MaxCompute that are owned by "Zhang San". Steps: Select MaxCompute in "Data source" -> Enter Zhang San in "Owner" -> Select Production in "Environment".

AI Search

When you find it difficult to describe the data you are looking for with a single keyword, or when you want to get recommendations based on business scenarios, you can use AI Search to ask questions in natural language. Data Map intelligently locates assets, recommends related tables, and explains the reasoning behind its recommendations.

  • Access from the search page: In the upper-right corner of the Data Map search page, switch to the AI Search tab to enter the conversational search experience.

  • Access from the homepage: In the search box on the Data Map homepage, switch to AI Q&A mode, enter your question, and the system redirects you to the AI search conversation page to continue the dialogue.

How to use: Describe your requirements in natural language in the input box. For example:

Find fact tables related to the transaction business; Query datasets added in the last 7 days; List MaxCompute projects that belong to the "Data Governance" business department.

Session management: A question submitted from the homepage creates a new session. When you enter the AI search page without specifying a session, your most recent session is resumed. You can create a new session or return to a historical session as needed.

  • Quick question templates: The bottom of the AI search conversation page provides commonly used question templates. Click a template to send it, which helps you get started quickly.

  • Relationship with comprehensive search: Comprehensive search matches metadata precisely by keywords and filter conditions, while AI search understands your requirements through natural language and semantic context and provides recommendations. The two approaches are complementary. Use comprehensive search when you have a clear target object, and use AI search when you want to describe business requirements.

Note

AI search is currently available through a gradual rollout on a per-tenant basis. It is visible only if your tenant has this capability enabled. To enable it, contact your administrator or submit a ticket.

FAQ

  • Q: Cannot find the table I am looking for?

    A: If you cannot find a target table in the DataWorks Data Map, the issue is typically caused by the following reasons. You can troubleshoot them one by one and refer to the corresponding solutions.

    • Search keyword mismatch: Search supports retrieval by table name, description, and field name. Try using more precise or partial keywords.

    • Metadata not synchronized: After you bind MaxCompute or DLF to DataWorks, metadata is automatically collected. However, there may be a delay after a new table is created or a table structure is changed. To manually trigger synchronization, perform the following steps:

      • Go to My Data > Refresh Table Metadata, enter the GUID of the target table in the GUID input box in the format project.table or odps.project.table, and click Sync.

      After the operation succeeds, search again to find the target table.

    • Table is hidden: The table may be set to the "Hidden" state, which makes it undiscoverable through search. When a table is set to "Hidden", it is invisible to everyone. When it is set to "Project Only", it is visible only to members of the current workspace.

    • Insufficient permissions: By default, search covers only the current workspace and authorized projects. If the target table belongs to another account or an unauthorized workspace, it cannot be found even if it exists. In addition, if you do not have query permissions on the table, you cannot view it in Data Map by default (especially in standard mode workspaces).

      Solution: Contact the table owner or administrator to request permissions, or verify whether you have joined the corresponding project.

Previous：External catalog collectionNext：Metadata details