DataLake SDK for Java reference

Prerequisites

Before you begin, make sure you have:

• An AccessKey pair (AccessKey ID and AccessKey secret)

• JDK 1.7 or later installed

Install the SDK

Add the following Maven dependency to your pom.xml:

<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>datalake20200710</artifactId>
  <version>2.0.12</version>
</dependency>

To find the latest version, search for datalake20200710 in the Maven repository.

Initialize the client

All API calls require an authenticated Client instance. Read credentials from environment variables — never hardcode your AccessKey ID or AccessKey secret in source code.

Set the following environment variables before running your application:

Config authConfig = new Config();
authConfig.accessKeyId     = System.getenv("AK_ENV");
authConfig.accessKeySecret = System.getenv("SK_ENV");
authConfig.type            = "access_key";
authConfig.endpoint        = "dlf.cn-shanghai.aliyuncs.com";
authConfig.regionId        = "cn-shanghai";

Client authClient = new Client(authConfig);

Key parameters:

For the full list of supported regions and endpoints, see Supported regions and endpoints.

Use a RAM user's AccessKey pair instead of your Alibaba Cloud account's credentials. Account credentials have unrestricted access to all API operations, which increases security risk. To create a RAM user, go to the RAM console.

Make an API call

Every API call follows three steps: build a request object, set parameters, and call the operation.

The example below creates a metadatabase (a database in the DLF data catalog) using the CreateDatabase operation.

Build the request:

CreateDatabaseRequest request = new CreateDatabaseRequest();
request.catalogId = "";

DatabaseInput input = new DatabaseInput();
input.name        = "example";
input.locationUri = "oss://test";
input.description = "";

request.databaseInput = input;

Call the operation and handle the response:

CreateDatabaseResponseBody response = authClient.createDatabase(request).body;

Complete example

The following runnable example combines all three steps. Save it as src/main/java/com/aliyun/datalake/examples/SchemaExample.java in a Maven project with the dependency added to pom.xml.

package com.aliyun.datalake.examples;

import com.aliyun.datalake20200710.Client;
import com.aliyun.datalake20200710.models.CreateDatabaseRequest;
import com.aliyun.datalake20200710.models.CreateDatabaseResponseBody;
import com.aliyun.datalake20200710.models.DatabaseInput;
import com.aliyun.teaopenapi.models.Config;
import com.google.gson.Gson;

public class SchemaExample {

    public static void main(String[] args) throws Exception {
        // 1. Initialize the client
        Config authConfig = new Config();
        // Store credentials in environment variables, not in source code.
        authConfig.accessKeyId     = System.getenv("AK_ENV");
        authConfig.accessKeySecret = System.getenv("SK_ENV");
        authConfig.type            = "access_key";
        authConfig.endpoint        = "dlf.cn-shanghai.aliyuncs.com";
        authConfig.regionId        = "cn-shanghai";

        Client authClient = new Client(authConfig);

        // 2. Build the request
        CreateDatabaseRequest request = new CreateDatabaseRequest();
        request.catalogId = "";

        DatabaseInput input = new DatabaseInput();
        input.name        = "example";
        input.locationUri = "oss://test";
        input.description = "";

        request.databaseInput = input;

        // 3. Call the operation and print the response
        CreateDatabaseResponseBody response = authClient.createDatabase(request).body;
        System.out.println(new Gson().toJson(response));
    }
}

Success response:

{"code":"OK","message":"","requestId":"1739F0B0-A94E-49AC-95FC-C1CE5E4171FA","success":true}

Error response — resource conflict (HTTP 409):

The SDK packages the status code and error message as a TeaException:

Exception in thread "main" com.aliyun.tea.TeaException: code: 409, Database example already exists request id: 598B1E2F-9AEF-4B13-AE4D-EB8733B643EB
 at com.aliyun.teaopenapi.Client.doROARequest(Client.java:303)
 at com.aliyun.datalake20200710.Client.createDatabaseWithOptions(Client.java:790)
 at com.aliyun.datalake20200710.Client.createDatabase(Client.java:772)
 at com.aliyun.datalake.examples.SchemaExample.main(SchemaExample.java:34)

Error response — network error:

Unknown errors such as DNS resolution failures are returned without modification:

Exception in thread "main" com.aliyun.tea.TeaException
 at com.aliyun.tea.Tea.doAction(Tea.java:67)
 at com.aliyun.teaopenapi.Client.doROARequest(Client.java:292)
 at com.aliyun.datalake20200710.Client.createDatabaseWithOptions(Client.java:790)
 at com.aliyun.datalake20200710.Client.createDatabase(Client.java:772)
 at com.aliyun.datalake.examples.SchemaExample.main(SchemaExample.java:34)
Caused by: java.net.UnknownHostException: dlf.cn-shanghai.aliyuncs.com
 at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
 at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
 at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
 at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
 at java.net.InetAddress.getAllByName(InetAddress.java:1192)
 at java.net.InetAddress.getAllByName(InetAddress.java:1126)
 at okhttp3.Dns$1.lookup(Dns.java:39)
 at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:171)
 at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:137)
 at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:82)
 at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:171)
 at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
 at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)
 at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
 at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
 at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
 at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
 at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
 at okhttp3.RealCall.execute(RealCall.java:69)
 at com.aliyun.tea.Tea.doAction(Tea.java:64)
 ... 4 more

If you see UnknownHostException, verify that your endpoint value is correct and that your network can reach it.

Best practices

public class AbstractAPI {

    protected final Client client;

    public AbstractAPI(Client client) {
        this.client = client;
    }

    public <M, V extends ResultModel<M>> ResultModel<M> call(Callable<V> c) throws Exception {
        try {
            return c.call();
        } catch (TeaException e) {
            Map<String, Object> data = e.getData();
            if (data != null && data.get("Code") != null) {
                return TeaModel.toModel(data, new ResultModel<M>());
            } else {
                throw e;
            }
        }
    }
}

public class DatabaseAPI extends AbstractAPI {

    public DatabaseAPI(Client client) {
        super(client);
    }

    public ResultModel<Void> createDatabase(String catalogId, String databaseName, String description,
                                            String locationUri, Map<String, String> parameters,
                                            String ownerName, String ownerType, PrincipalPrivilegeSet privileges) throws Exception {
        return call(() -> {
            CreateDatabaseRequest request = new CreateDatabaseRequest();
            request.catalogId = catalogId;

            DatabaseInput input = new DatabaseInput();
            input.name        = databaseName;
            input.description = description;
            input.locationUri = locationUri;
            input.parameters  = parameters;
            input.ownerName   = ownerName;
            input.ownerType   = ownerType;
            input.privileges  = privileges;

            request.databaseInput = input;

            CreateDatabaseResponseBody response = client.createDatabase(request).body;
            return new ResultModel<>(response.success, response.code, response.message,
                    response.requestId);
        });
    }
}

ResultModel<Void> response = new DatabaseAPI(authClient).createDatabase("", "example3", "",
        "oss://test", null, null, null, null);
System.out.println(new Gson().toJson(response));

{"success":true,"code":"OK","message":"","requestId":"50778D55-696D-45FA-8328-B01983F6CEB1"}

{"success":false,"code":"AlreadyExists","message":"Database example3 already exists","requestId":"94617169-DA17-4020-9027-7D8F89160682","httpStatusCode":409}

What's next

• Supported regions and endpoints — find the endpoint for your region

• OpenAPI Explorer — call API operations interactively and generate SDK sample code