Iceberg REST Catalog API compatibility

更新时间:
复制 MD 格式

OSS Tables provides API endpoints compatible with the Apache Iceberg REST Catalog and AWS S3 Tables. This topic details differences between OSS Tables and the Iceberg REST standard protocol in request parameters, response fields, and expected behavior across three dimensions: endpoints, parameters, and authentication. Use this information to assess compatibility when building your own client or selecting a compute engine.

Compatibility design principles

OSS Tables follows these three principles for its Iceberg REST Catalog interface:

  • AWS S3 Tables compatibility: Where the Iceberg standard protocol is ambiguous or allows multiple implementations, OSS Tables prioritizes compatibility with S3 Tables behavior. This simplifies cross-cloud client reuse and code migration.

  • Mainstream open source SDK compatibility: Without breaking S3 Tables compatibility, OSS Tables writes certain fields (such as pagination tokens) in two formats. This ensures the official Apache Iceberg Java and C++ SDKs work directly without modification.

  • Clearly defined unsupported features: These include views, CTAS (CREATE TABLE AS SELECT), multi-level namespaces, Plan API, OAuth2, and BearerAuth authentication.

Endpoint compatibility

The following table covers all 13 endpoints exposed by the OSS Tables Iceberg REST Catalog. The "S3 Tables behavior" column describes specific deviations from the Iceberg specification. The "OSS Tables compatibility" column explains how OSS Tables handles each case and its impact on clients.

OSS Tables provides an Iceberg REST Catalog endpoint that Iceberg clients use to manage table metadata. The endpoint format is:

  • Private network: https://{region}-internal.oss-tables.aliyuncs.com/iceberg

  • Public network: https://{region}.oss-tables.aliyuncs.com/iceberg

In the table below, the {prefix} parameter uses the URL-encoded form of the table bucket ARN. The ARN format is acs:osstables:{region}:{userId}:bucket/{tableBucketName}.

Path

Method

Operation

Compatibility

/v1/config

GET

getConfig

The response does not include the endpoints field. Clients will use the default endpoint list defined in the Iceberg specification, which does not fully match the actual list supported by this endpoint.

/v1/{prefix}/namespaces

GET

listNamespaces

The response includes both nextPageToken and next-page-token fields. The token field is omitted in the last page response.

/v1/{prefix}/namespaces

POST

createNamespace

Only the owner property is accepted. All other properties are rejected.

/v1/{prefix}/namespaces/{namespace}

GET

loadNamespaceMetadata

The response does not include the properties field.

/v1/{prefix}/namespaces/{namespace}

HEAD

namespaceExists

Complies with the Iceberg specification.

/v1/{prefix}/namespaces/{namespace}

DELETE

dropNamespace

Complies with the Iceberg specification.

/v1/{prefix}/namespaces/{namespace}/tables

GET

listTables

Supports pageToken pagination. The response includes both nextPageToken and next-page-token fields. The token field is omitted in the last page response.

/v1/{prefix}/namespaces/{namespace}/tables

POST

createTable

Does not support stage-create=true. Set this field to false in requests. Therefore, CREATE TABLE AS SELECT (CTAS) syntax is not supported.

/v1/{prefix}/namespaces/{namespace}/tables/{table}

GET

loadTable

Supports the snapshots query parameter (default all; valid values are all and refs, case-sensitive). Supports the If-None-Match request header (case-insensitive).

/v1/{prefix}/namespaces/{namespace}/tables/{table}

POST

updateTable

Complies with the Iceberg specification. For details on supported requirements and updates types, see updateTable compatibility.

/v1/{prefix}/namespaces/{namespace}/tables/{table}

DELETE

dropTable

You must explicitly set purgeRequested=true. Otherwise, the request returns 400 Bad Request. In the Iceberg specification, this parameter defaults to false.

/v1/{prefix}/namespaces/{namespace}/tables/{table}

HEAD

tableExists

Complies with the Iceberg specification.

/v1/{prefix}/tables/rename

POST

renameTable

Complies with the Iceberg specification.

updateTable compatibility

The updateTable endpoint submits transactions using two arrays: requirements (optimistic locking preconditions) and updates (metadata change actions).

Supported requirements

Supports all 8 requirement types defined in the Iceberg specification:

  • assert-create

  • assert-table-uuid

  • assert-ref-snapshot-id

  • assert-last-assigned-field-id

  • assert-current-schema-id

  • assert-last-assigned-partition-id

  • assert-default-spec-id

  • assert-default-sort-order-id

Supported updates (metadata change actions)

  • assign-uuid

  • upgrade-format-version

  • add-schema

  • set-current-schema

  • add-spec

  • set-default-spec

  • set-partition-statistics

  • remove-partition-statistics

  • add-sort-order

  • set-default-sort-order

  • add-snapshot

  • set-snapshot-ref

  • remove-snapshot-ref

  • remove-snapshots

  • set-properties

  • remove-properties

  • set-statistics

  • remove-statistics

Unsupported updates

  • remove-schemas

  • remove-partition-specs

  • set-location

  • add-encryption-key

  • remove-encryption-key

  • add-view-version

  • set-current-view-version

General rules

  1. Strict field type matching: All JSON field types must be correct (do not pass numbers as strings or arrays as objects). Otherwise, the request fails.

  2. The action field must match: The action field value in each request must exactly match the corresponding action name.

  3. Semantics of ID -1: In set-current-schema, set-default-spec, and set-default-sort-order, a *-id value of -1 means "point to the most recently added corresponding object," per the Iceberg protocol convention.

  4. Idempotence: Operations such as add-schema, add-spec, add-sort-order, and add-snapshot skip execution if the target object already exists. They do not add duplicates.

Parameter compatibility

The following table covers compatibility for common path parameters, query parameters, and request headers defined in the Iceberg REST standard protocol.

Parameter

Compatibility

namespace

Multi-level namespaces are not supported. Requests with multi-level namespaces fail.

prefix

Supported.

table

Supported.

plan-id

Not supported. The Plan API is not implemented.

view

Views are not supported.

X-Iceberg-Access-Delegation

Not supported. The request header is ignored.

parent

Multi-level namespaces are not supported. Requests with this parameter fail.

pageToken

Supported. Pass it as a URL-encoded query string parameter. The field is not returned in the last page response.

pageSize

Supported.

ETag

Supported.

Idempotency-Key

Not supported. The request header is ignored.

Authentication compatibility

OSS Tables Iceberg REST Catalog supports only SigV4 signature authentication (with service name osstables). Other authentication methods defined in the Iceberg specification are not supported:

Authentication method

Compatibility

SigV4

Supported (recommended). The service name for signing is osstables.

OAuth2

Not supported. The Iceberg specification marks this as DEPRECATED for REMOVAL.

BearerAuth

Not supported.

Integration recommendations

Before integrating with OSS Tables, note these three key impacts:

  • Client selection: Use the official Apache Iceberg Java or C++ SDK.

  • Table creation and deletion: CTAS (CREATE TABLE AS SELECT) is not supported. First call createTable to create an empty table, then write data. When deleting a table, you must explicitly include purgeRequested=true. Otherwise, the request returns a 400 error.

  • Unsupported features: These include views, multi-level namespaces, Plan API, Idempotency-Key, X-Iceberg-Access-Delegation, OAuth2, and BearerAuth authentication.