DocsICP FilingConsole
Documentation
Home Page MaxCompute User Guide Security and Compliance Data security configuration Authorization for external data sources

Authorization for external data sources

Updated at:
Copy as MD
Product
My Favorites

MaxCompute integrates with RAM and STS to enable secure access to external data sources. This topic describes authorization methods that use RAM and STS for different scenarios.

OSS/Tablestore: RAM role authorization

To allow a MaxCompute external table to access data in OSS or Tablestore, its owner must associate a RAM role that has the required data access permissions with the external table. After the role is associated, the owner can grant permissions to users who need to access this external table.

Choose an authorization method based on your business requirements. Use custom authorization for more fine-grained permission control.

Same-account authorization

When MaxCompute and OSS/Tablestore are owned by the same Alibaba Cloud account, MaxCompute supports one-click authorization and custom authorization.

  • One-click authorization: After you log on to your Alibaba Cloud account, click here to complete one-click authorization. This method creates a role named AliyunODPSDefaultRole by default. This role has broad permissions and is suitable for testing.

  • Custom authorization: Create a new role in the RAM console and configure appropriate access policies and a trust policy for it. This method allows for fine-grained permission control. Perform the following steps:

    1. Create a RAM role

      Log on to the RAM console, and create a RAM role, such as test-admin. For more information, see Create a RAM role for a trusted Alibaba Cloud account.

    2. Modify the trust policy of the RAM role.

      1. Log in to the RAM console.

      2. In the left navigation bar, select Identities > Roles.

      3. On the Roles page, click the target Role Name to go to the role details page.

      4. On the Trust Policy tab, click Edit Trust Policy, and on the Edit Trust Policy page, select the JSON Editor tab.

        Configure the following policy:

        {
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "odps.aliyuncs.com"
 ]
 }
 }
 ],
 "Version": "1"
}

      5. When you are finished, click OK.

    3. Create an access policy.

      On the Policies page in the RAM console, create a custom access policy. For example, you can name it AliyunODPSRolePolicy.

      The following policies provide common permissions. You can customize them as needed. For more information, see Create a custom access policy.

      OSS access policy

      {
"Version": "1",
"Statement": [
{
 "Action": [
        "oss:ListBuckets",
        "oss:GetObject",
        "oss:ListObjects",
        "oss:PutObject",
        "oss:DeleteObject",
        "oss:AbortMultipartUpload",
        "oss:ListParts",
        "oss:GetBucketInfo",
        "oss:PostDataLakeStorageFileOperation",
        "oss:PostDataLakeStorageAdminOperation"
 ],
 "Resource": "*",
 "Effect": "Allow"
}
]
}

      Tablestore access policy

      {
"Version": "1",
"Statement": [
{
 "Action": [
 "ots:ListTable",
 "ots:DescribeTable",
 "ots:GetRow",
 "ots:PutRow",
 "ots:UpdateRow",
 "ots:DeleteRow",
 "ots:GetRange",
 "ots:BatchGetRow",
 "ots:BatchWriteRow",
 "ots:ComputeSplitPointsBySize"
 ],
 "Resource": "*",
 "Effect": "Allow"
}
]
}

    4. Attach the custom access policy to the new RAM role.

      1. Log in to the RAM console.

      2. In the left navigation bar, select Identities > Roles.

      3. On the Roles page, click the target Role Name to go to the role details page.

      4. On the Manage Permissions tab, click Create Authorization. In the Create Authorization panel that appears, select the policies to grant to the role, and click OK.

        For more information, see Manage the permissions of a RAM role.

Cross-account authorization

If MaxCompute and OSS/Tablestore are owned by different Alibaba Cloud accounts, only custom authorization is supported.

The owner of the Alibaba Cloud account that owns OSS or Tablestore must create a role in the RAM console and configure its access and trust policies as follows:

  1. Create a RAM role.

    Log in to the RAM console and create a RAM role. For example, a role named test-admin. For more information about how to create a role, see Create a RAM role for a trusted Alibaba Cloud account.

  2. Modify the trust policy of the RAM role.

    1. Log in to the RAM console.

    2. In the left navigation bar, select Identities > Roles.

    3. On the Roles page, click the target Role Name to go to the role details page.

    4. On the Trust Policy tab, click Edit Trust Policy, and on the Edit Trust Policy page, select the JSON Editor tab.

      Configure the following policy. Replace <MaxCompute_owner_account_ID> with the ID of the Alibaba Cloud account that owns the MaxCompute project.

      {
"Statement": [
{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "<MaxCompute_owner_account_ID>@odps.aliyuncs.com"
 ]
 }
}
],
"Version": "1"
}

    5. When you are finished, click OK.

  3. On the Policies page in the RAM console, create a custom access policy. For example, you can name it AliyunODPSRolePolicy.

    The following policies provide common permissions. You can customize them as needed. For more information, see Create a custom access policy.

    OSS access policy

    {
"Version": "1",
"Statement": [
{
 "Action": [
        "oss:ListBuckets",
        "oss:GetObject",
        "oss:ListObjects",
        "oss:PutObject",
        "oss:DeleteObject",
        "oss:AbortMultipartUpload",
        "oss:ListParts",
        "oss:GetBucketInfo",
        "oss:PostDataLakeStorageFileOperation",
        "oss:PostDataLakeStorageAdminOperation"
 ],
 "Resource": "*",
 "Effect": "Allow"
}
]
}

    Tablestore access policy

    {
"Version": "1",
"Statement": [
{
 "Action": [
 "ots:ListTable",
 "ots:DescribeTable",
 "ots:GetRow",
 "ots:PutRow",
 "ots:UpdateRow",
 "ots:DeleteRow",
 "ots:GetRange",
 "ots:BatchGetRow",
 "ots:BatchWriteRow",
 "ots:ComputeSplitPointsBySize"
 ],
 "Resource": "*",
 "Effect": "Allow"
}
]
}

  4. Attach the custom access policy to the new RAM role.

    1. Log in to the RAM console.

    2. In the left navigation bar, select Identities > Roles.

    3. On the Roles page, click the target Role Name to go to the role details page.

    4. On the Manage Permissions tab, click Create Authorization. In the Create Authorization panel that appears, select the policies to grant to the role, and click OK.

      For more information, see Manage the permissions of a RAM role.

Hologres: RAM role authorization

To allow a MaxCompute external table to directly access data in Hologres, the table owner must associate a RAM role that has the necessary Hologres data access permissions with the external table. After the role is associated, the owner can grant permissions to users who need to access this external table. You can choose an authorization method based on your business requirements.

Same-account authorization

When MaxCompute and Hologres are owned by the same Alibaba Cloud account, grant permissions as follows:

  1. Create a RAM role

    Create a RAM role to obtain its Alibaba Cloud Resource Name (ARN). The ARN is required for STS authentication when you create the external table. Select a trusted entity type based on your business requirements. In this example, the trusted entity type is an Alibaba Cloud account.

    Perform the following steps:

    1. Log in to the RAM console.

    2. In the left navigation bar, select Identities > Roles.

    3. On the Roles page, click Create Role.

  2. Modify the trust policy

    1. On the Roles page, click the target Role Name to go to the role details page.

    2. On the Trust Policy tab, click Edit Trust Policy, and on the Edit Trust Policy page, select the JSON Editor tab.

    3. Modify the trust policy as shown in the following examples.

      The content of the trust policy configuration depends on the selected trusted entity type. In the policy, <UID> is the Alibaba Cloud account ID. You can obtain the ID from the User Information page.

      Trusted entity: Alibaba Cloud account

      {
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "RAM": [
 "acs:ram::<UID>:root"
 ]
 }
 },
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "<UID>@odps.aliyuncs.com"
 ]
 }
 }
 ],
 "Version": "1"
}

      Trusted entity: Identity provider

      {
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Condition": {
 "StringEquals": {
 "saml:recipient": "https://signin.aliyun.com/saml-role/sso"
 }
 },
 "Effect": "Allow",
 "Principal": {
 "Federated": [
 "acs:ram::<UID>:saml-provider/IDP"
 ]
 }
 },
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "<UID>@odps.aliyuncs.com"
 ]
 }
 }
 ],
 "Version": "1"
}

  3. Add the RAM role to a Hologres instance and grant permissions

    The RAM role requires developer permissions on the Hologres instance to access resources within its authorization scope.

    By default, RAM roles cannot view or manage instances in the Hologres management console. The Alibaba Cloud account owner must grant the necessary permissions before you proceed.

    Add the RAM role to a Hologres instance

    Hologres management console

    1. Log on to the Hologres management console. In the upper-left corner, select a region.

    2. In the navigation pane on the left, click Instances.

    3. On the Instances page, click the name of the target instance.

    4. On the instance details page, in the navigation pane on the left, click Account Management.

    5. On the Users page, click Add User.

    6. In the Add User dialog box, configure the settings and click OK.

      • For Role, select the RAM role that you created in Step 1.

      • For Member Role, select Normal User.

    7. On the Users page, in the navigation pane on the left, click DB authorization.

    8. On the DB authorization page, click Create Database.

    9. In the Create Database dialog box, configure the settings and click OK.

      For Policies, select SPM.

    10. On the DB authorization page, find the target database and click Authorize User in the Actions column.

    11. In the upper-right corner, click Create Authorization. In the Create Authorization dialog box, configure the User and User Group.

      • Users Select the RAM user to configure.

      • For Groups, select Developer.

    RAM console

    Attach the AliyunODPSRolePolicy access policy to the new RAM role.

    1. Log in to the RAM console.

    2. In the left navigation bar, select Identities > Users.

    3. On the Users page, click the target User Logon Name/Display Name to go to the user details page.

    4. On the user details page, click the Manage Permissions tab.

    5. On the Manage Permissions tab, select Individual, and click Grant Permission.

    6. In the Grant Permission panel, select the policy to grant to the user, and click OK.

      For Access Policy, select AliyunRAMReadOnlyAccess.

      For more information, see Grant permissions to a RAM user.

    SQL

    For the required SQL statements, see Permission model.

    Note

    Hologres also supports authorization by using a service-linked role. For more information, see Service-linked role authorization for Hologres (identity passthrough).

Cross-account authorization

If MaxCompute and Hologres are owned by different Alibaba Cloud accounts, grant permissions as follows:

  1. Create a RAM role

    Create a RAM role to obtain its ARN. The ARN is required for STS authentication when you create the external table. Select a trusted entity type based on your business requirements. In this example, the trusted entity type is an Alibaba Cloud account.

    Perform the following steps:

    1. Log in to the RAM console.

    2. In the left navigation bar, select Identities > Roles.

    3. On the Roles page, click Create Role.

  2. Modify the trust policy

    1. On the Roles page, click the target Role Name to go to the role details page.

    2. On the Trust Policy tab, click Edit Trust Policy, and on the Edit Trust Policy page, select the JSON Editor tab.

    3. Modify the trust policy as shown in the following examples.

      The trust policy configuration content depends on the selected trusted entity type. In this policy, <UID> represents the Alibaba Cloud account ID, which you can find on the User Information page.

      Trusted entity: Alibaba Cloud account

      {
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "RAM": [
 "acs:ram::<MaxCompute_owner_account_ID>:root"
 ]
 }
 },
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "<MaxCompute_owner_account_ID>@odps.aliyuncs.com"
 ]
 }
 }
 ],
 "Version": "1"
}

      Trusted entity: Identity provider

      {
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Condition": {
 "StringEquals": {
 "saml:recipient": "https://signin.aliyun.com/saml-role/sso"
 }
 },
 "Effect": "Allow",
 "Principal": {
 "Federated": [
 "acs:ram::<MaxCompute_owner_account_ID>:saml-provider/IDP"
 ]
 }
 },
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "<MaxCompute_owner_account_ID>@odps.aliyuncs.com"
 ]
 }
 }
 ],
 "Version": "1"
}

  3. Add the RAM role to a Hologres instance and grant permissions

    The RAM role requires developer permissions on the Hologres instance to access resources within its authorization scope.

    By default, RAM roles cannot view or manage instances in the Hologres management console. The Alibaba Cloud account owner must grant the necessary permissions before you proceed.

    Add the RAM role to a Hologres instance

    Hologres management console

    1. Log on to the Hologres management console. In the upper-left corner, select a region.

    2. In the navigation pane on the left, click Instances.

    3. On the Instances page, click the name of the target instance.

    4. On the instance details page, in the navigation pane on the left, click Account Management.

    5. On the Users page, click Add User.

    6. In the Add User dialog box, configure the settings and click OK.

      • For Role, select the RAM role that you created in Step 1.

      • For Member Role, select Normal User.

    7. On the Users page, in the navigation pane on the left, click DB authorization.

    8. On the DB authorization page, click Create Database.

    9. In the Create Database dialog box, configure the settings and click OK.

      For Policies, select SPM.

    10. On the DB authorization page, find the target database and click Authorize User in the Actions column.

    11. In the upper-right corner, click Create Authorization. In the Create Authorization dialog box, configure the User and User Group.

      • Users: Select the RAM user to configure.

      • For Groups, select Developer.

    RAM console

    Attach the AliyunODPSRolePolicy access policy to the new RAM role.

    1. Log in to the RAM console.

    2. In the left navigation bar, select Identities > Users.

    3. On the Users page, click the target User Logon Name/Display Name to go to the user details page.

    4. On the user details page, click the Manage Permissions tab.

    5. On the Manage Permissions tab, select Individual, and click Grant Permission.

    6. In the Grant Permission panel, select the policy to grant to the user, and click OK.

      For the access policy, select AliyunRAMReadOnlyAccess.

      For more information, see Grant permissions to a RAM user.

    SQL

    For the required SQL statements, see Permission model.

    Note

    Hologres also supports dual-signature authentication. For more information, see Hologres external tables.

DLF and OSS: RAM role authorization

When you build a lakehouse architecture with MaxCompute, DLF, and OSS, you must associate a RAM role that has the required data access permissions for DLF and OSS with an external data source. You can then create an external schema and use the permissions of the role to access DLF and OSS. To grant other users permissions on tables in the external schema, see Grant permissions on federated external tables in an external schema. The following authorization methods are supported:

  • One-click authorization (Recommended): Use this method if the Alibaba Cloud account that is used to create the MaxCompute project is the same as the account used to deploy DLF and OSS. Click Authorize DLF and OSS to complete the authorization in one click.

  • Custom authorization: Use this method for both same-account and cross-account scenarios. Perform the following steps.

    • If the RAM user for the MaxCompute project is the same as the account used to deploy DLF, you must set service to odps.aliyuncs.com when you add the trust policy.

    • If the RAM user for the MaxCompute project and the account used to deploy DLF are different, you must set the service parameter to <ID of the Alibaba Cloud account that owns the MaxCompute project>@odps.aliyuncs.com when you add a trust policy. You can obtain the ID of the Alibaba Cloud account that owns the MaxCompute project on the Account Overview page.

    1. Log on to the RAM console and create a RAM role for a trusted Alibaba Cloud account.

    2. In the RAM console, modify the trust policy of the RAM role. Use one of the following policies based on your scenario:

      Same account

      {
"Statement": [
{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "odps.aliyuncs.com"
 ]
 }
}
],
"Version": "1"
}

      Cross-account

      {
"Statement": [
{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": [
 "<MaxCompute_project_owner_account_ID>@odps.aliyuncs.com" 
 ]
 }
}
],
"Version": "1"
}

    3. In the RAM console, create a custom access policy for the new RAM role. For more information, see Create a custom access policy. Use the following content for the custom access policy:

      {
"Version": "1",
"Statement": [
{
 "Action": [
 "oss:ListBuckets",
 "oss:GetObject",
 "oss:ListObjects",
 "oss:PutObject",
 "oss:DeleteObject",
 "oss:AbortMultipartUpload",
 "oss:ListParts"
 ],
 "Resource": "*",
 "Effect": "Allow"
},
{
 "Action": [
 "dlf:CreateFunction",
"dlf:BatchGetPartitions",
"dlf:ListDatabases",
"dlf:CreateLock",
"dlf:UpdateFunction",
"dlf:BatchUpdateTables",
"dlf:DeleteTableVersion",
"dlf:UpdatePartitionColumnStatistics",
"dlf:ListPartitions",
"dlf:DeletePartitionColumnStatistics",
"dlf:BatchUpdatePartitions",
"dlf:GetPartition",
"dlf:BatchDeleteTableVersions",
"dlf:ListFunctions",
"dlf:DeleteTable",
"dlf:GetTableVersion",
"dlf:AbortLock",
"dlf:GetTable",
"dlf:BatchDeleteTables",
"dlf:RenameTable",
"dlf:RefreshLock",
"dlf:DeletePartition",
"dlf:UnLock",
"dlf:GetLock",
"dlf:GetDatabase",
"dlf:GetFunction",
"dlf:BatchCreatePartitions",
"dlf:ListPartitionNames",
"dlf:RenamePartition",
"dlf:CreateTable",
"dlf:BatchCreateTables",
"dlf:UpdateTableColumnStatistics",
"dlf:ListTableNames",
"dlf:UpdateDatabase",
"dlf:GetTableColumnStatistics",
"dlf:ListFunctionNames",
"dlf:ListPartitionsByFilter",
"dlf:GetPartitionColumnStatistics",
"dlf:CreatePartition",
"dlf:CreateDatabase",
"dlf:DeleteTableColumnStatistics",
"dlf:ListTableVersions",
"dlf:BatchDeletePartitions",
"dlf:ListCatalogs",
"dlf:UpdateTable",
"dlf:ListTables",
"dlf:DeleteDatabase",
"dlf:BatchGetTables",
"dlf:DeleteFunction"
 ],
 "Resource": "*",
 "Effect": "Allow"
}
]
}

    4. Attach the custom access policy to the new RAM role. For more information, see Manage the permissions of a RAM role.

Hologres: Service-linked role (identity passthrough)

When MaxCompute accesses data in Hologres through an external data source and an external project, you must use a service-linked role to grant the required data access permissions to Hologres. After you create the role, this mode uses identity passthrough for authorization. It passes the user's identity to Hologres for authentication to prevent unauthorized access. To use this feature, the user must be a member of the MaxCompute external project and have permissions on the Hologres data source that is mapped to the external project.

  1. Log in to the RAM console.

  2. In the left navigation bar, select Identities > Roles.

  3. On the Roles page, click Create Role.

  4. In the upper-right corner of the Create Role page, click Create Service Linked Role.

  5. On the Create Service Linked Role page, select Select Service: aliyunserviceroleformaxcomputeidentitymgmt, and click Create Service Linked Role.

    If a message indicates that the role already exists, the role has already been authorized. You can ignore the message.

Paimon and DLF: Service-linked role (identity passthrough)

When MaxCompute accesses data in DLF through an external data source and an external project, you must use a service-linked role to grant the required data access permissions to DLF. After you create the role, this mode uses identity passthrough for authorization. It passes the user's identity to DLF for authentication to prevent unauthorized access. To use this feature, the user must be a member of the MaxCompute external project and have permissions on the DLF data source that is mapped to the external project.

  1. Log in to the RAM console.

  2. In the left navigation bar, select Identities > Roles.

  3. On the Roles page, click Create Role.

  4. In the upper-right corner of the Create Role page, click Create Service Linked Role.

  5. On the Create Service Linked Role page, for Select Service, select AliyunServiceRoleForMaxComputeLakehouse, and click Create Service Linked Role.

    If a message indicates that the role already exists, the role has already been authorized. You can ignore the message.

Previous：Other security measuresNext：Secure configuration FAQ