RAM role-based authorization: Configure a data source

Workflow

Procedure

1. Create an assumable role.

   You need to create different custom roles for different data sources based on your security requirements. This topic uses the following scenario as an example to illustrate how to create an assumable role.

   Note

   Only an Alibaba Cloud account or a RAM user with the AliyunRAMFullAccess policy can perform this step.

   Consider an enterprise with 100 buckets that store all of its data, but the big data team only needs to use data from one specific bucket. If the predefined AliyunDataWorksAccessingOSSRole role were used, the big data team might access the other 99 buckets, which poses a management risk.

   To prevent this, the account administrator can create a custom role named BigDataOssRole for the big data team and restrict its use to relevant members of that team. This helps implement permission control between teams.

   1. Create a custom role.

      In this example, create a custom role named BigDataOssRole and set the trusted entity to Alibaba Cloud Account. For detailed steps, see Create a RAM role for a trusted Alibaba Cloud account.

   2. Create a custom permission policy.

      Create a policy that grants permissions to read data from and write data to only the specified bucket. For detailed steps, see Create a custom policy. The following code shows the policy document:

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "oss:GetObject",
                      "oss:ListObjects",
                      "oss:GetObjectMetadata",
                      "oss:GetObjectMeta",
                      "oss:GetBucketAcl",
                      "oss:GetBucketInfo",
                      "oss:PutObject",
                      "oss:DeleteObject",
                      "oss:PutBucket"
                  ],
                  "Resource": [
                      "acs:oss:*:*:bucket_name_1",
                      "acs:oss:*:*:bucket_name_1/*"
                  ]
              }
          ]
      }

   3. Grant the role the required permissions.

      Modify the trust policy of the BigDataOSSRole role and grant the BigDataOSSRole role the policy created in Step 2. This allows users who assume the BigDataOSSRole role to read data from the two specified buckets.

      Important

      You must perform this step. Otherwise, the role cannot be used.

      For more information about how to modify the trust policy of a role, see Modify the trust policy of a RAM role. The following code shows the policy document:

      {
          "Statement": [
              {
                  "Action": "sts:AssumeRole",
                  "Effect": "Allow",
                  "Principal": {
                      "Service": [
                          "di.dataworks.aliyuncs.com"
                      ]
                  }
              }
          ],
          "Version": "1"
      }

2. Grant users permission to use the role.

   After you create the roles to be assumed, you must grant a policy that contains the ram:PassRole permission to the relevant users. This allows them to use the role to create a data source and run synchronization tasks. You can also configure mappings between users and roles.

   • Policy template 1: You can create a policy based on the following template. This template allows the authorized user to use all roles related to DataWorks Data Integration. Grant this policy with caution.

     {
         "Version": "1",
         "Statement": [
             {
                 "Action": "ram:PassRole",
                 "Resource": "*",
                 "Effect": "Allow",
                 "Condition": {
                     "StringEquals": {
                         "acs:Service": "di.dataworks.aliyuncs.com"
                     }
                 }
             }
         ]
     }

   • Policy template 2: You can create a custom policy that contains the ram:PassRole permission and configure mappings between users and roles based on your security requirements.

     Note

     Only an Alibaba Cloud account or a RAM user with the AliyunRAMFullAccess policy can perform this step.

     Scenario example: As described in the preceding scenario, after the account administrator defines the BigDataOssRole role for the big data team, the administrator must specify which users can use this role. You can create a custom policy named BigDataOssRoleAllowUse and grant it to the relevant users.

     Create a policy named BigDataOssRoleAllowUse. For more information, see Create a custom policy. The following code shows the policy document:

     {
         "Version": "1",
         "Statement": [
             {
                 "Action": "ram:PassRole",
                 "Resource": "acs:ram::19122324****:role/BigDataOssRole",
                 "Effect": "Allow",
                 "Condition": {
                     "StringEquals": {
                         "acs:Service": [
                             "oss.aliyuncs.com",
                             "di.dataworks.aliyuncs.com"
                         ]
                     }
                 }
             }
         ]
     }

     Note

     Replace the UID (19122324****) in the policy with the UID of your Alibaba Cloud account.

     An administrator grants the BigDataOssRoleAllowUse policy to a RAM user who is allowed to use the BigDataOssRole role. A RAM user who is granted the BigDataOssRoleAllowUse policy can then assume the BigDataOssRole role to create data sources (using the BigDataOssRole as the access identity for the data source) and run synchronization tasks.

3. Create a data source.

   After the account administrator grants permissions to the data source creator, the creator can add the data source.

   1. An Alibaba Cloud account or a RAM user with the AliyunDataWorksFullAccess policy creates and configures an OSS data source.

      During configuration, select RAM role authorization mode for Access mode and configure the other parameters. If you use a workspace in standard mode, you can add the data source to the Development or Production.

      Note

      This topic uses an OSS data source as an example. The parameters may vary for other data source types. For more information about how to add an OSS data source, see Configure an OSS data source.

      Parameter	Description
      Data Source Name	It can contain letters, digits, and underscores (_) but cannot start with a digit or an underscore.
      Description	A brief description of the data source. The description can be up to 80 characters in length.
      Endpoint	The endpoint of the OSS service. The format is http://oss.aliyuncs.com. Endpoints vary by region. You must use the correct endpoint for the region you are accessing. Note The correct format for the Endpoint is http://oss.aliyuncs.com. However, using http://oss.aliyuncs.com with a bucket value prepended and separated by a dot is incorrect. For example, for an endpoint such as http://xxx.oss.aliyuncs.com, a connectivity test may pass, but the synchronization will fail.
      Bucket	The name of the OSS bucket. A bucket is a container that stores objects. You can create one or more buckets and add one or more files to each bucket. DataWorks can find objects only in the bucket that you specify here.
      Access mode	Select RAM role authorization mode. This mode uses Security Token Service (STS) to authorize the DataWorks service account to assume a role to access the data source, which provides higher security.
      Select Role	Select a RAM role from the Select Role drop-down list.
      Region	Select a region from the Region drop-down list.

   2. Test the network connectivity.

      In the Connection Configuration section, find the target resource group and click Test Connectivity in its row.

      Each synchronization task uses a single resource group. You must test the connectivity of the intended resource group to ensure it can connect to the data source; otherwise, the task will fail. To test multiple resource groups at the same time, select the resource groups and click Batch test connectivity. For more information, see Network connectivity solutions.

   3. After the connectivity test is successful, click Complete.

4. Create a synchronization task.

   After the data source is created, developers can create a synchronization task in DataStudio that uses this data source. For more information, see Configure a synchronization task.

5. Run the synchronization task.

   The task executor runs the data synchronization task in DataStudio or Operation Center.

   Note

   When running a task in DataStudio, ensure the task executor has the permissions granted in Step 2 (Grant users permission to use the role) to prevent task failures.