Example of simple data uploads-MaxCompute(MaxCompute)-阿里云帮助中心

Tunnel SDK lets you bulk-load offline datasets into MaxCompute tables programmatically using Java. Use Tunnel SDK when you need to:

Write records from application code directly into partitioned or non-partitioned tables
Integrate MaxCompute data ingestion into Java-based ETL or data pipeline workflows
Upload large offline datasets at high throughput without using DataWorks or the command-line tool

Prerequisites

Before you begin, make sure you have:

The MaxCompute Java SDK added to your project. Add the following dependency to your pom.xml:

<dependency>
    <groupId>com.aliyun.odps</groupId>
    <artifactId>odps-sdk-core</artifactId>
    <version><!-- use the latest version --></version>
</dependency>

A MaxCompute project with a target table created. For partitioned tables, create the target partition before uploading
An AccessKey ID and AccessKey secret stored as environment variables:
```
export ALIBABA_CLOUD_ACCESS_KEY_ID=<your-access-key-id>
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=<your-access-key-secret>
```
Use a RAM user with the minimum permissions required. Avoid using your Alibaba Cloud account credentials directly. To create a RAM user, go to the RAM console

Key limits

Review the following constraints before writing upload code. Exceeding these limits causes upload failures or performance degradation.

Resource	Limit	Notes
Session lifecycle	24 hours	A session expires after 24 hours regardless of upload status
Blocks per session	20,000	Block IDs are numbered 0–19,999. If you exceed 20,000 blocks, commit the session and start a new one
Minimum block size	64 MB (recommended)	Writing less than 64 MB per block generates many small files, which degrades query performance
Maximum block size	100 GB
Writer idle timeout	2 minutes	If a Writer writes less than 4 KB in 2 minutes after creation, the connection is closed automatically
Data flush interval	Every 8 KB	Data is transmitted to the server each time 8 KB is written
Server auto-disconnect	120 seconds	If no data is transmitted for 120 seconds, the server closes the Writer connection

Prepare data in memory before creating a Writer. Session creation takes several seconds and allocates server-side temporary directories — upload all records for the same partition in a single session.

Upload data using Tunnel SDK

The upload flow uses five core objects in sequence: TableTunnel → UploadSession → RecordWriter → Record → commit.

Step 1: Initialize the client

import com.aliyun.odps.account.Account;
import com.aliyun.odps.account.AliyunAccount;
import com.aliyun.odps.Odps;
import com.aliyun.odps.tunnel.TableTunnel;

// Read credentials from environment variables — do not hardcode AccessKey values in source code
String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");

String odpsUrl = "http://service.odps.aliyun.com/api";
String project = "<your-project>";

Account account = new AliyunAccount(accessId, accessKey);
Odps odps = new Odps(account);
odps.setEndpoint(odpsUrl);
odps.setDefaultProject(project);

TableTunnel tunnel = new TableTunnel(odps);

By default, data is transmitted over the Internet. To transmit over an intranet, set the Tunnel endpoint to match your network environment. The example below uses the cloud product interconnection network endpoint for the China (Shanghai) region:

// Intranet endpoint for China (Shanghai) — cloud product interconnection network
tunnel.setEndpoint("https://dt.cn-shanghai-intranet.maxcompute.aliyun-inc.com");

For all available Tunnel endpoints by region and network type, see Endpoints.

Step 2: Create an upload session

Create one session per partition. Each session has a 24-hour lifecycle and supports up to 20,000 blocks.

import com.aliyun.odps.PartitionSpec;
import com.aliyun.odps.tunnel.TableTunnel.UploadSession;

String table = "<your-table-name>";
String partition = "pt='XXX',ds='XXX'"; // PartitionSpec format: key='value'

PartitionSpec partitionSpec = new PartitionSpec(partition);
UploadSession uploadSession = tunnel.createUploadSession(project, table, partitionSpec);

System.out.println("Session status: " + uploadSession.getStatus());

PartitionSpec(String spec) accepts a comma-separated string of key-value pairs, for example pt='1',ds='2'.

Step 3: Write records

Open a block by ID (starting at 0), write records, then close the Writer. A successfully closed block cannot be re-uploaded. If the upload fails, retry that block using the same block ID.

import com.aliyun.odps.Column;
import com.aliyun.odps.TableSchema;
import com.aliyun.odps.data.Record;
import com.aliyun.odps.data.RecordWriter;

import java.util.Date;

TableSchema schema = uploadSession.getSchema();

// Prepare data before opening the Writer — the Writer times out if idle for 2 minutes
RecordWriter recordWriter = uploadSession.openRecordWriter(0); // block ID 0
Record record = uploadSession.newRecord();

for (int i = 0; i < schema.getColumns().size(); i++) {
    Column column = schema.getColumn(i);
    switch (column.getType()) {
        case BIGINT:   record.setBigint(i, 1L);           break;
        case BOOLEAN:  record.setBoolean(i, true);        break;
        case DATETIME: record.setDatetime(i, new Date()); break;
        case DOUBLE:   record.setDouble(i, 0.0);          break;
        case STRING:   record.setString(i, "sample");     break;
        default:
            throw new RuntimeException("Unknown column type: " + column.getType());
    }
}

// Write 10 records to block 0
for (int i = 0; i < 10; i++) {
    recordWriter.write(record);
}

recordWriter.close(); // Block 0 is uploaded when close() succeeds

Write at least 64 MB of data per block. Smaller blocks generate many small files that slow down downstream queries.

Step 4: Commit the session

After all blocks are closed, commit the session to finalize the upload. Pass the block IDs you uploaded.

uploadSession.commit(new Long[]{0L}); // Commit block 0
System.out.println("Upload complete.");

Error handling

Catch TunnelException and IOException separately and retry failed operations:

} catch (TunnelException e) {
    // Retry the failed block or session
    e.printStackTrace();
} catch (IOException e) {
    // Retry the Writer operation
    e.printStackTrace();
}

TunnelException: indicates a server-side Tunnel error. Retry at the session or block level.
IOException: indicates a network or I/O error. Retry the Writer operation for the affected block.

Complete example

The following is the full upload example combining all steps above.

import java.io.IOException;
import java.util.Date;

import com.aliyun.odps.Column;
import com.aliyun.odps.Odps;
import com.aliyun.odps.PartitionSpec;
import com.aliyun.odps.TableSchema;
import com.aliyun.odps.account.Account;
import com.aliyun.odps.account.AliyunAccount;
import com.aliyun.odps.data.Record;
import com.aliyun.odps.data.RecordWriter;
import com.aliyun.odps.tunnel.TableTunnel;
import com.aliyun.odps.tunnel.TableTunnel.UploadSession;
import com.aliyun.odps.tunnel.TunnelException;

public class UploadSample {

    private static String accessId  = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
    private static String accessKey  = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
    private static String odpsUrl    = "http://service.odps.aliyun.com/api";
    private static String tunnelUrl  = "https://dt.cn-shanghai-intranet.maxcompute.aliyun-inc.com";
    private static String project    = "<your-project>";
    private static String table      = "<your-table-name>";
    private static String partition  = "pt='XXX',ds='XXX'";

    public static void main(String[] args) {
        Account account = new AliyunAccount(accessId, accessKey);
        Odps odps = new Odps(account);
        odps.setEndpoint(odpsUrl);
        odps.setDefaultProject(project);

        try {
            TableTunnel tunnel = new TableTunnel(odps);
            tunnel.setEndpoint(tunnelUrl);

            PartitionSpec partitionSpec = new PartitionSpec(partition);
            UploadSession uploadSession = tunnel.createUploadSession(project, table, partitionSpec);
            System.out.println("Session status: " + uploadSession.getStatus());

            TableSchema schema = uploadSession.getSchema();

            // Prepare data before creating the Writer
            RecordWriter recordWriter = uploadSession.openRecordWriter(0);
            Record record = uploadSession.newRecord();

            for (int i = 0; i < schema.getColumns().size(); i++) {
                Column column = schema.getColumn(i);
                switch (column.getType()) {
                    case BIGINT:   record.setBigint(i, 1L);           break;
                    case BOOLEAN:  record.setBoolean(i, true);        break;
                    case DATETIME: record.setDatetime(i, new Date()); break;
                    case DOUBLE:   record.setDouble(i, 0.0);          break;
                    case STRING:   record.setString(i, "sample");     break;
                    default:
                        throw new RuntimeException("Unknown column type: " + column.getType());
                }
            }

            for (int i = 0; i < 10; i++) {
                recordWriter.write(record);
            }

            recordWriter.close();
            uploadSession.commit(new Long[]{0L});
            System.out.println("Upload complete.");

        } catch (TunnelException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Replace the following placeholders before running the example:

Placeholder	Description	Example
`<your-project>`	Your MaxCompute project name	`my_project`
`<your-table-name>`	The target table name	`sales_data`
`pt='XXX',ds='XXX'`	The partition spec for the target partition	`pt='2024',ds='0101'`

What's next

For all Tunnel endpoints organized by region and network type, see Endpoints.
To download data using Tunnel SDK, see the download example in the Tunnel SDK documentation.