API-Data Lake Formation(DLF)-阿里云帮助中心

The metadata API is a standard, open-source interface provided by Paimon. Its lightweight SDK includes only Paimon-related classes and does not conflict with your application's dependencies. You can access the metadata API with low latency through a Data Lake Formation (DLF) VPC endpoint, or over the public internet via an Alibaba Cloud OpenAPI endpoint for use cases such as cross-region access and local debugging. Signature support for the OpenAPI endpoint requires a Paimon 1.4 or later client.

Before you begin

Configure a VPC whitelist

By default, the metadata API can be accessed only from whitelisted VPCs. When you enable the Data Lake Formation (DLF) service, the system automatically syncs the VPC IDs in your current region to your user-level whitelist. If you need to add a new VPC, you can manually add its VPC ID in the DLF console. To do so, follow these steps:

Log on to the DLF console.
In the navigation pane on the left, click System & Security.
Click the System Security tab, and then click Add VPC ID.
In the dialog box that appears, enter the VPC ID and click OK.

Grant DLF permissions to a role

Grant Resource Access Management (RAM) permissions to the ECS role. For example, for an EMR cluster, the role is AliyunECSInstanceForEMRRole.
1. Log on to the RAM console with your Alibaba Cloud account or as a RAM administrator.
2. In the navigation pane on the left, choose Identities > Roles, and find the target ECS role.
3. In the Actions column, click Add Permissions.
4. Under Policies, search for and select AliyunDLFFullAccess, and then click OK.
Grant the ECS role DLF permissions.
1. Log on to the DLF console.
2. On the Catalogs page, click the name of your target catalog.
3. Click the Permissions tab, and then click Grant.
4. On the authorization page, configure the following settings and click OK.
  - User/Role: Select RAM user/RAM role.
  - Select object to authorize: From the drop-down list, select the ECS role.
    
    Note
    If the ECS role is not in the drop-down list, go to the System and Security > Access Control > User page and click Sync.
  - Pre-defined Permission Type: Select Data Editor.

Add a Maven dependency

To reference the API SDK in your Java project, add the following Maven dependency.

<dependency>
  <groupId>org.apache.paimon</groupId>
  <artifactId>paimon-api</artifactId>
  <version>1.3.0</version>
</dependency>

You can also download the JAR file directly: paimon-api-1.3.0.jar .

Create a REST API client

We recommend using an ECS role for DLF REST authentication.

import org.apache.paimon.options.Options;
import org.apache.paimon.rest.RESTApi;
import static org.apache.paimon.options.CatalogOptions.WAREHOUSE;
import static org.apache.paimon.rest.RESTCatalogOptions.DLF_TOKEN_LOADER;
import static org.apache.paimon.rest.RESTCatalogOptions.TOKEN_PROVIDER;
import static org.apache.paimon.rest.RESTCatalogOptions.URI;
public class RESTApiExample {
    public static void main(String[] args) {
        Options options = new Options();
        options.set(URI, "http://cn-hangzhou-vpc.dlf.aliyuncs.com");
        options.set(WAREHOUSE, "dlf_test");
        options.set(TOKEN_PROVIDER, "dlf");
        options.set(DLF_TOKEN_LOADER, "ecs");
        RESTApi api = new RESTApi(options);
        System.out.println(api.listTables("my_database"));
    }
}

The following table describes the parameters.

Parameter	Description	Required	Example
URI	The URI for accessing the DLF REST catalog server. The format is `http://[region-id]-vpc.dlf.aliyuncs.com`. For a list of region IDs and OpenAPI addresses, see Endpoints.	Yes	http://cn-hangzhou-vpc.dlf.aliyuncs.com
WAREHOUSE	The name of the DLF catalog.	Yes	dlf_test
TOKEN_PROVIDER	The token provider. Set this to `dlf`.	Yes	dlf
DLF_TOKEN_LOADER	Specifies that an ECS instance issues a temporary token. Set this to `ecs`. Note Instead of using the ECS authentication method, you can also connect directly to the DLF REST service by configuring `RESTCatalogOptions` with `DLF_ACCESS_KEY_ID` and `DLF_ACCESS_KEY_SECRET`.	No	ecs