RAM role-based authorization: Configure a data source

Procedure overview

Step 1: Create a role to be assumed

1. Create a custom role.

   In this example, a custom role is created with the trusted entity set to Alibaba Cloud Account and the role name set to BigDataOssRole. For more information about how to create a role, see Create a RAM role for a trusted Alibaba Cloud account.

2. Create a custom policy.

   In this example, a policy is created that allows read and write access only to data in a specified bucket. For more information about how to create a policy, see Create a custom policy. The policy content is as follows.

   {
       "Version": "1",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "oss:GetObject",
                   "oss:ListObjects",
                   "oss:GetObjectMetadata",
                   "oss:GetObjectMeta",
                   "oss:GetBucketAcl",
                   "oss:GetBucketInfo",
                   "oss:PutObject",
                   "oss:DeleteObject",
                   "oss:PutBucket"
               ],
               "Resource": [
                   "acs:oss:*:*:bucket_name_1",
                   "acs:oss:*:*:bucket_name_1/*"
               ]
           }
       ]
   }

3. Authorize the role with the specified policy.

   Modify the trust policy of the BigDataOSSRole role and grant the policy created in Step 2 to the BigDataOSSRole role. This allows users who are later authorized with the BigDataOSSRole role to read data from the two specified buckets.

   Important

   This step is required. Otherwise, the role cannot be used.

   Modify the role policy. For more information, see Edit the trust policy of a RAM role. The policy content is as follows.

   {
       "Statement": [
           {
               "Action": "sts:AssumeRole",
               "Effect": "Allow",
               "Principal": {
                   "Service": [
                       "di.dataworks.aliyuncs.com"
                   ]
               }
           }
       ],
       "Version": "1"
   }

Step 2: Authorize users who can use the role

Step 3: Create a data source

1. Use an Alibaba Cloud account or a RAM user that is granted the DataWorksFullAccess policy to create and configure an OSS data source.

   During configuration, set Access mode to RAM role authorization mode. Configure other parameters as described below. If you use a standard mode workspace, you can configure the Development or Production environment based on your business requirements.

   Note

   This topic uses an OSS data source as an example. In actual use, refer to the configuration page of the data source you need. For more information about how to configure an OSS data source, see Configure an OSS data source.

   Parameter	Description
   Data Source Name	The data source name must be a combination of letters, digits, and underscores (_), and cannot start with a digit or underscore (_).
   Description	A brief description of the data source. The description cannot exceed 80 characters.
   Endpoint	The OSS endpoint, in the format of http://oss.aliyuncs.com. The endpoint of an OSS service varies based on the region. You must enter different endpoints when you access different regions. Note The correct format for the endpoint is http://oss.aliyuncs.com. However, http://oss.aliyuncs.com with the bucket name prepended to "oss" and connected by a period, such as http://xxx.oss.aliyuncs.com, can pass the connectivity test but will cause errors during synchronization.
   Bucket	The OSS bucket, which is a storage space used as a container for storing objects. You can create one or more buckets, and each bucket can contain one or more files. You can search for files in the buckets specified here in a data synchronization task. Files in buckets that are not specified here cannot be searched.
   Access mode	Select RAM role authorization mode. This mode uses STS-based authorization to allow the cloud service account to assume the relevant role to access the data source, providing higher security.
   Select Role	Select a RAM role from the Select Role drop-down list.
   Region	Select a region from the Region drop-down list.

2. Test network connectivity.

   In the Connection Configuration section, click Test Connectivity next to the desired resource group.

   During data synchronization, a task can use only one type of resource group. You must test the connectivity for each resource group type to ensure that the Data Integration resource group used by the synchronization task can connect to the data source. Otherwise, the data synchronization task cannot run properly. To test multiple resource groups at the same time, select the desired resource groups and click Batch test connectivity. For more information, see Configure network connectivity for a data source.

3. After the connectivity test passes, click Complete.

Scenarios

Alibaba Cloud account A owns RAM role A-role, and Alibaba Cloud account B owns OSS bucket B-bucket, which stores the data. You need to use A-role in DataWorks in Alibaba Cloud account A to access B-bucket in Alibaba Cloud account B. Alibaba Cloud account B grants cross-account access by configuring a bucket policy for B-bucket. Alibaba Cloud account A manages the role, and Alibaba Cloud account B manages the bucket and its authorization policy.

Step 1: Configure the bucket policy in Alibaba Cloud account B

Use Alibaba Cloud account B or an identity that can manage the authorization policy of the target bucket to configure the bucket policy for B-bucket in the OSS console. For instructions, see bucket policy. Add the following statement to the bucket policy without removing any existing statements. The following complete policy example contains only this statement.

{
  "Version": "1",
  "Statement": [
    {
      "Sid": "cross-bucket-policy-demo2",
      "Effect": "Allow",
      "Action": [
        "oss:*"
      ],
      "Principal": [
        "arn:sts::<AccountA-ID>:assumed-role/<RoleName>/*"
      ],
      "Resource": [
        "acs:oss:*:<AccountB-ID>:<BucketName>",
        "acs:oss:*:<AccountB-ID>:<BucketName>/*"
      ]
    }
  ]
}

• Replace <AccountA-ID> with the ID of Alibaba Cloud account A, which owns the role, and <RoleName> with the actual name of A-role.

• Replace <AccountB-ID> with the ID of Alibaba Cloud account B, which owns the bucket, and <BucketName> with the actual name of B-bucket. The two Resource entries specify the bucket and its objects, respectively.

• To authorize a RAM role, set Principal to a temporary identity that starts with arn:sts, in the format arn:sts::<AccountID>:assumed-role/<RoleName>/<CustomSessionName>. Obtain the role ARN from the role information in the Resource Access Management (RAM) console in Alibaba Cloud account A, for example, acs:ram::<AccountA-ID>:role/<RoleName>. Extract the account ID and role name, and construct the temporary identity in the preceding format. Use the wildcard character * as the session name to match sessions of this role, not all roles. Do not directly use the acs:ram role ARN as Principal in this example.

Step 2: Configure the role trust policy and usage permissions in Alibaba Cloud account A

In the Resource Access Management (RAM) console in Alibaba Cloud account A, configure the following trust policy for A-role to allow the DataWorks data integration service to assume the role.

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "di.dataworks.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

If RAM users in Alibaba Cloud account A create the data source or run the synchronization task, follow "Step 2: Authorize users who can use the role" in "Same-account access to OSS" to grant the relevant users the ram:PassRole permission for A-role. The role resource in the access policy must use the ID of Alibaba Cloud account A and the actual name of A-role, not an identity in Alibaba Cloud account B. The role trust policy allows DataWorks to assume the role, whereas the bucket policy in Step 1 authorizes cross-account access to B-bucket. Neither policy replaces the other.

Step 3: Create and verify the OSS data source in Alibaba Cloud account A

1. In DataWorks in Alibaba Cloud account A, create an OSS data source by following "Step 3: Create a data source" in "Same-account access to OSS".

2. Set Access mode to RAM role authorization mode, and select A-role from Alibaba Cloud account A in Select Role.

3. Enter the name of B-bucket from Alibaba Cloud account B in Bucket. Configure Endpoint and Region for the region where B-bucket resides. The endpoint must not contain the bucket name; follow the format requirements in the preceding configuration instructions.

4. In Connection Configuration, run Test Connectivity for the resource group that the synchronization task uses. After the test passes, click Complete.

5. Create and run a synchronization task based on this data source, and verify that the task can access the required data in B-bucket in Alibaba Cloud account B. If a RAM user runs the task, ensure that the user has the role usage permission granted in Step 2.