Set up a direct file upload service

更新时间:
复制 MD 格式

Accessing IMM from a browser is riskier than from a server, especially when handling long-term, fixed AccessKey pairs. To reduce the risk of key leaks, you can use Alibaba Cloud's Security Token Service (STS) to issue STS tokens, which are temporary credentials with a limited lifetime and the minimum necessary permissions. This approach prevents exposing your static AccessKey pair in the less secure browser environment.

Background

STS solves a core problem: how to securely authorize access to your resources without exposing your Alibaba Cloud account's AccessKey pair. Exposing an AccessKey pair creates a significant security risk, as anyone who obtains it can freely operate all resources, steal important information, and more. To handle large-scale concurrent data processing, the technical team at Enterprise A lets end-users upload files directly from their browsers to OSS and interact with IMM.

Security risks

To reduce the load on the enterprise's application servers and improve efficiency, the web application is designed to interact directly with IMM in the user's web browser, instead of routing all requests through the enterprise's servers.

Enterprise A initially plans to build the direct file upload service with the following solution:

image

Because a browser runs on the user side and is not under the direct control of Enterprise A, the enterprise faces the following challenges:

  • Credential leaks: Because a web browser is an untrusted environment controlled by the user, storing sensitive information like a RAM user's AccessKey pair in the browser creates a high risk of leaks.

  • Excessive permissions: When a RAM user is created as an identity for an application server, it often requires broad permissions to access other cloud services. If this RAM user's AccessKey pair is stored in the browser and gets compromised, these permissions could be abused, increasing security risks.

Solution

To address these risks, Enterprise A can add temporary authorization to its original design. This approach allows the enterprise to preserve the efficiency of direct data uploads and provides the following benefits:

  • Using time-limited STS tokens greatly reduces the security risk, even if a token is leaked. The credentials expire quickly, minimizing their potential for misuse.

  • Fine-grained permission control: STS lets you configure permissions based on the principle of least privilege, granting the web application only the necessary access. This approach limits the blast radius of a potential credential leak and mitigates the risk of over-privileged access.

Enterprise A ultimately adopts the following solution to build its IMM data processing service:

image

Implementation

This section uses a simple file upload scenario to guide you through deploying a browser-based data processing service for your web application by using OSS, IMM, and STS.

Prerequisites

  • Create an OSS bucket and an IMM project.

    Parameter

    Example

    region

    China (Hangzhou)

    Bucket name

    web-direct-upload

    Project name

    web-direct-project

    For more information, see Create a bucket and Create a project.

  • Create an ECS instance to act as your application server for generating STS tokens.

    Note

    In a production deployment, you can integrate the API call to the STS service into your own application server's API, which eliminates the need to create this ECS instance.

    Parameter

    Example

    Billing method

    pay-as-you-go

    region

    China (Hangzhou)

    public IP address

    Assign a public IPv4 address

    security group

    Open the HTTP (TCP:80) port

    For more information, see Create an instance by using the wizard.

  • Configure cross-origin resource sharing (CORS) for the created OSS bucket.

    Parameter

    Example

    Source

    http://<your_ecs_instance_public_ip>

    Allowed Methods

    PUT

    Allowed Headers

    *

    For more information, see Configure CORS.

Procedure

Step 1: Create a RAM user

First, create a RAM user for OpenAPI Call and obtain its AccessKey pair to use as the long-term identity credential for the application on the business server.

  1. Log on to the RAM console with your Alibaba Cloud account or an account administrator.

  2. In the left-side navigation pane, choose Identities > Users.

  3. Click Create User.

  4. Enter a Logon Name and a Display Name.

  5. For Access Mode, select Programmatic Access, and then click OK.

  6. Click Copy to save the AccessKey pair, which includes an AccessKey ID and an AccessKey Secret.

Step 2: Grant permission to call AssumeRole

After creating the RAM user, grant it permission to call the AssumeRole API operation of STS. This allows the RAM user to obtain an STS token by assuming a RAM role.

  1. In the left-side navigation pane, choose Identities > Users.

  2. On the Users page, find the target RAM user and click Add Permissions in the Actions column.

  3. On the Add Permissions page, select the AliyunSTSAssumeRoleAccess system policy.

    Note

    The AliyunSTSAssumeRoleAccess policy grants fixed permission to call the STS AssumeRole API operation. It is independent of the permissions required to obtain the STS token or to make IMM requests by using that token.

  4. Click OK.

Step 3: Create a RAM role

Create a RAM role for your Alibaba Cloud account and save its ARN. The RAM user you created will assume this role.

  1. In the left-side navigation pane, choose Identities > Roles.

  2. Click Create Role. For Select Trusted Entity, choose Alibaba Cloud Account, and then click Next.

  3. Enter a RAM Role Name and select Current Alibaba Cloud Account.

  4. Click OK. After the role is created, click Close.

  5. On the Roles page, enter the role name in the search box, for example, imm-web-upload.

  6. Click the Role Name, click Copy, and save the role's ARN.

Step 4: Create custom policies

Create a custom policy for OSS

Following the principle of least privilege, create a custom policy for the RAM role that restricts uploads to a specific OSS bucket.

  1. In the left-side navigation pane, choose Permissions > Policies.

  2. Click Create Policy.

  3. On the Create Policy page, click the JSON Editor tab. In the policy editor, replace <BucketName> with web-direct-upload.

Important

The following example is for reference only. You must configure a more fine-grained policy to prevent the risk of excessive permissions. For more information, see Common examples of custom policies.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "oss:PutObject",
      "Resource": "acs:oss:*:*:<BucketName>/uploads/*"
    }
  ]
}
  1. After configuring the policy, click Next to edit policy information.

  2. In the Basic Information section, enter a policy name, and then click OK.

Create a custom policy for IMM

Following the principle of least privilege, create another custom policy for the RAM role. The creation steps are the same as for the OSS policy. This policy restricts operations to a specific IMM project.

The following is a sample policy script:

Important

The following example is for reference only. You must configure a more fine-grained policy to prevent the risk of excessive permissions. For more information, see Grant permissions on IMM resources to a RAM user.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "imm:*",
            "Resource": "acs:imm:cn-hangzhou:1413397765616316:project/<ProjectName>"
        }
    ]
}

Step 5: Grant permissions to the RAM role

Attach the custom policies to the RAM role so that the role has the required permissions when it is assumed.

  1. In the left-side navigation pane, choose Identities > Roles.

  2. On the Roles page, find the target RAM role and click Add Permissions in the Actions column.

  3. On the Add Permissions page, click the Custom Policy tab and select the custom policies you created.

  4. Click OK.

Step 6: Obtain an STS token

In your web application, integrate the STS SDK on your application server to implement an endpoint for obtaining an STS token. When this endpoint (for example, /get_sts_token) is accessed via an HTTP GET request, it generates an STS token and returns it to the requester.

The following example shows how to use the Flask framework on an ECS instance to quickly build a web application and implement an endpoint to get an STS token:

  1. Connect to the ECS instance.

    For more information, see Create an instance by using the wizard.

  2. Install Python 3.

  3. Create a project folder and switch to the project directory.

    mkdir my_web_sample
    cd my_web_sample
  4. Install dependencies.

    pip3 install Flask
    pip3 install attr
    pip3 install yarl
    pip3 install async_timeout
    pip3 install idna_ssl
    pip3 install attrs
    pip3 install aiosignal
    pip3 install charset_normalizer
    pip3 install alibabacloud_tea_openapi
    pip3 install alibabacloud_sts20150401
    pip3 install alibabacloud_credentials
  5. Write the backend code.

    1. Create a main.py file.

    2. Add the following Python code to the file.

      import json
      from flask import Flask, render_template
      from alibabacloud_tea_openapi.models import Config
      from alibabacloud_sts20150401.client import Client as Sts20150401Client
      from alibabacloud_sts20150401 import models as sts_20150401_models
      from alibabacloud_credentials.client import Client as CredentialClient
      app = Flask(__name__)
      # Replace <YOUR_ROLE_ARN> with the ARN of the RAM role you obtained in Step 3.
      role_arn_for_oss_upload = '<YOUR_ROLE_ARN>'
      # Set this to the region of the STS service, for example, cn-hangzhou.
      region_id = 'cn-hangzhou'
      @app.route("/imm")
      def imm():
          return render_template('imm_example.html')
      @app.route('/get_sts_token', methods=['GET'])
      def get_sts_token():
          # If no parameters are specified when initializing CredentialClient, the default credential chain is used.
          # When running locally, you can specify the AccessKey pair via the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables.
          # When running on ECS, ECI, or a container service, you can specify the attached instance role via the ALIBABA_CLOUD_ECS_METADATA environment variable, and the SDK will automatically fetch the STS token.
          config = Config(region_id=region_id, credential=CredentialClient())
          sts_client = Sts20150401Client(config=config)
          assume_role_request = sts_20150401_models.AssumeRoleRequest(
              role_arn=role_arn_for_oss_upload,
              # Replace <YOUR_ROLE_SESSION_NAME> with a custom session name.
              role_session_name='<YOUR_ROLE_SESSION_NAME>'
          )
          response = sts_client.assume_role(assume_role_request)
          token = json.dumps(response.body.credentials.to_map())
          return token
      app.run(host="0.0.0.0", port=80) 
    3. Replace <YOUR_ROLE_ARN> in the code with the role ARN you obtained in Step 3.

    4. Replace <YOUR_ROLE_SESSION_NAME> in the code with a custom session name, such as role_session_test.

  6. Start the application using the AccessKey pair you obtained in Step 1.

    ALIBABA_CLOUD_ACCESS_KEY_ID=<YOUR_AK_ID> ALIBABA_CLOUD_ACCESS_KEY_SECRET=<YOUR_AK_SECRET> python3 main.py
  7. In your browser, go to http://<your_ecs_instance_public_ip>/get_sts_token.

    A successful response looks like this:

    {"AccessKeyId": "STS.NUDRy4pc4PrizHtBYxxx", "AccessKeySecret": "H3gq6ZP2fsapnuDmGcRBxoVfiakxxx", "Expiration": "2024-05-07T09:05:19Z", "SecurityToken": "CAISywJ1q6Ft5B2yfSjlr5bxGcOAnbwV57CCeG7FplkGX9VYnqDliTz2IHhMf3ltAu0ftPUxnxxx xxx xxx xxx xxx xxx"}

Step 7: Call the IMM service with an STS token

After configuring the endpoint on your application server to get an STS token, use a CDN to include the OSS SDK for JavaScript on the front-end page of your web application. This lets you listen for file uploads. When a user uploads a file, call the /get_sts_token endpoint to request an STS token from the application server, and then use the token to upload the file to OSS. When calling an IMM service from the web, you must encrypt the data using the HMAC algorithm. This example uses HmacSHA1 from the third-party library crypto.js. Then, use the STS token to call the DetectImageFaces API operation and retrieve the results.

The following example shows how to integrate the front-end code into the web application on your ECS instance:

  1. Press Ctrl + C to stop the application.

  2. Create the front-end project directory.

    mkdir templates
  3. Create the HTML template file.

    1. In the templates directory, create an imm_example.html file.

      vim templates/imm_example.html
    2. Add the following HTML code to the file. This page simulates a user selecting and uploading a file in a browser, then calls the IMM service to detect faces and face attributes in the image.

      <!DOCTYPE html>
      <html lang="en">
        <head>
          <meta charset="UTF-8" />
          <meta name="viewport" content="width=device-width, initial-scale=1.0" />
          <title>imm_example</title>
          <script src="https://gosspublic.alicdn.com/aliyun-oss-sdk-6.17.0.min.js"></script>
          <script src="https://unpkg.com/crypto-js@4.1.1/crypto-js.js"></script>
          <style>
            .app form div {
              display: flex;
              margin: 12px;
            }
            .app label {
              display: flex;
              width: 100px;
            }
            .app form input {
              width: 200px;
            }
          </style>
        </head>
        <body>
          <div class="app">
            <h2>1. Upload file to OSS:</h2>
            <form id="oss-upload-form">
              <div>
                <label for="file" class="form-label">Select file:</label>
                <input
                  type="file"
                  class="form-control"
                  id="file"
                  name="file"
                  required
                />
              </div>
              <button type="submit" class="btn btn-primary">Upload</button>
            </form>
            <div id="oss-result"></div>
          </div>
          <div class="app">
            <h2>2. Call IMM DetectImageFaces service:</h2>
            <form id="imm-form">
              <button type="submit">Request</button>
            </form>
          </div>
          <div>
            <h2>3. Result:</h2>
            <div id="result"></div>
          </div>
          <script>
            /**
             * Check if the temporary credentials have expired.
             **/
            function isCredentialsExpired(credentials) {
              if (!credentials) {
                return true;
              }
              const expireDate = new Date(credentials.Expiration);
              const now = new Date();
              // Consider credentials expired if they are valid for less than one minute.
              return expireDate.getTime() - now.getTime() <= 60000;
            }
            let credentials = null;
            let uploadResult = null;
            // ############# OSS-UPLOAD #################
            const ossForm = document.querySelector("#oss-upload-form");
            ossForm.addEventListener("submit", async (event) => {
              event.preventDefault();
              // Fetch new credentials only when the current ones are expired to reduce calls to the STS service.
              if (isCredentialsExpired(credentials)) {
                const response = await fetch("/get_sts_token", {
                  method: "GET",
                });
                if (!response.ok) {
                  // Handle HTTP error status codes.
                  throw new Error(
                    `Failed to get STS token: ${response.status} ${response.statusText}`
                  );
                }
                credentials = await response.json();
              }
              const client = new OSS({
                // Replace with your OSS bucket name.
                bucket: "web-direct-upload",
                // Replace with the region of your OSS bucket, for example, oss-cn-hangzhou.
                region: "oss-cn-hangzhou",
                accessKeyId: credentials.AccessKeyId,
                accessKeySecret: credentials.AccessKeySecret,
                stsToken: credentials.SecurityToken,
              });
              const fileInput = document.querySelector("#file");
              const file = fileInput.files[0];
              const result = await client.put("/" + file.name, file);
              if (result) {
                uploadResult = result;
                document.querySelector("#oss-result").textContent = "Upload successful";
              }
            });
            // ##################################################
            // ################# IMM ############################
            function getTimestamp() {
              const pad2 = (number) => {
                return String(number).padStart(2, "0");
              };
              let date = new Date();
              let YYYY = date.getUTCFullYear();
              let MM = pad2(date.getUTCMonth() + 1);
              let DD = pad2(date.getUTCDate());
              let HH = pad2(date.getUTCHours());
              let mm = pad2(date.getUTCMinutes());
              let ss = pad2(date.getUTCSeconds());
              return `${YYYY}-${MM}-${DD}T${HH}:${mm}:${ss}Z`;
            }
            function generateSecureNonce(length = 32) {
              const array = new Uint8Array(length);
              window.crypto.getRandomValues(array);
              return Array.from(array, (byte) => byte.toString(36))
                .join("")
                .substring(0, length);
            }
            function normalize(params) {
              const list = [];
              const flated = params;
              const keys = Object.keys(flated).sort();
              for (let i = 0; i < keys.length; i++) {
                const key = keys[i];
                const value = flated[key];
                list.push([encode(key), encode(value)]);
              }
              return list;
            }
            function canonicalize(normalized) {
              const fields = [];
              for (let i = 0; i < normalized.length; i++) {
                const [key, value] = normalized[i];
                fields.push(key + "=" + value);
              }
              return fields.join("&");
            }
            function encode(str) {
              const result = encodeURIComponent(str);
              return result
                .replace(/!/g, "%21")
                .replace(/'/g, "%27")
                .replace(/\(/g, "%28")
                .replace(/\)/g, "%29")
                .replace(/\*/g, "%2A");
            }
            async function sha1(data, key = null) {
              const keyData = CryptoJS.enc.Utf8.parse(key);
              // Use the HmacSHA1 method to calculate the HMAC.
              const hmac = CryptoJS.HmacSHA1(data, keyData);
              // Convert the HMAC (a CryptoJS object) to a base64 string.
              const base64Hmac = hmac.toString(CryptoJS.enc.Base64);
              return base64Hmac;
            }
            async function getRPCSignature(signedParams, method, secret) {
              const normalized = normalize(signedParams);
              const canonicalized = canonicalize(normalized);
              const stringToSign = `${method}&${encode("/")}&${encode(
                canonicalized
              )}`;
              const key = secret + "&";
              return await sha1(stringToSign, key);
            }
            async function getStsToken() {
              const response = await fetch("/get_sts_token", {
                method: "GET",
              });
              if (!response.ok) {
                // Handle HTTP error status codes.
                throw new Error(
                  `Failed to get STS token: ${response.status} ${response.statusText}`
                );
              }
              return await response.json();
            }
            // let credentials = null;
            async function getParams({ Action, ProjectName, SourceURI }) {
              if (isCredentialsExpired(credentials)) {
                credentials = await getStsToken();
              }
              const params = {
                AccessKeyId: credentials.AccessKeyId,
                Format: "JSON",
                Action,
                ProjectName,
                Timestamp: getTimestamp(),
                SecureTransport: true,
                SignatureMethod: "HMAC-SHA1",
                SignatureNonce: generateSecureNonce(),
                SignatureVersion: "1.0",
                SourceURI,
                Version: "2020-09-30",
                SecurityToken: credentials.SecurityToken,
              };
              params.Signature = await getRPCSignature(
                params,
                "POST",
                credentials.AccessKeySecret
              );
              return params;
            }
            async function detect(requestParams) {
              const params = await getParams(requestParams);
              const url = `${requestParams.EndPoint}?${new URLSearchParams(
                params
              ).toString()}`;
              const response = await fetch(url, {
                method: "POST",
                headers: {
                  "Content-Type": "application/json",
                },
              });
              if (!response.ok) {
                throw new Error(
                  `API request failed: ${response.status} ${response.statusText}`
                );
              }
              return await response.json();
            }
            const immForm = document.querySelector("#imm-form");
            immForm.addEventListener("submit", (e) => {
              e.preventDefault();
              detect({
                // Name of the IMM operation.
                Action: "DetectImageFaces",
                // Replace with your IMM project name.
                ProjectName: "web-direct-project",
                // Replace with the OSS URL of the file, for example: oss://web-direct-upload/test.jpg
                SourceURI: `oss://web-direct-upload/${uploadResult.name}`,
                // Replace with the endpoint of the IMM service, for example: https://imm.cn-beijing.aliyuncs.com
                EndPoint: "https://imm.cn-hangzhou.aliyuncs.com",
              })
                .then((res) => {
                  // console.log(res);
                  document.querySelector("#result").textContent = JSON.stringify(res);
                })
                .catch((err) => {
                  document.querySelector("#result").textContent = err;
                });
            });
          </script>
        </body>
      </html>
                            
  4. Start the application using the AccessKey pair you obtained in Step 1.

    ALIBABA_CLOUD_ACCESS_KEY_ID=<YOUR_AK_ID> ALIBABA_CLOUD_ACCESS_KEY_SECRET=<YOUR_AK_SECRET> python3 main.py
  5. In your browser, go to http://<your_ecs_instance_public_ip>/imm. On the page, select the image you want to analyze and upload it to simulate real user behavior in a browser.

    The page is divided into three sections. In the first section, upload a file to OSS. In the second section, click Request to call the IMM DetectImageFaces service. In the third section, view the face detection results in the Result area.

Verify and clean up

Verify the solution

After completing the steps, verify that the file was uploaded to OSS.

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets.

  3. On the Buckets page, click the target bucket.

  4. On the Objects page, view the successfully uploaded file.

After completing the steps, view the analysis results from IMM.

{
    "RequestId": "63661E75-3A41-5FBF-B023-867DA6A6AA81",
    "Faces": [
        {
            "Beard": "none",
            "MaskConfidence": 0.764,
            "Gender": "female",
            "Boundary": {
                "Left": 182,
                "Top": 175,
                "Height": 381,
                "Width": 304
            },
            "BeardConfidence": 0.987,
            "FigureId": "047d8d12-c3b6-4e22-9b9a-b91facc650fb",
            "Mouth": "open",
            "Emotion": "happiness",
            "Age": 45,
            "MouthConfidence": 0.999,
            "FigureType": "face",
            "GenderConfidence": 1,
            "HeadPose": {
                "Pitch": -16.206,
                "Roll": -5.124,
                "Yaw": 3.421
            },
            "Mask": "none",
            "EmotionConfidence": 0.984,
            "HatConfidence": 1,
            "GlassesConfidence": 0.976,
            "Sharpness": 1,
            "FigureClusterId": "figure-cluster-id-unavailable",
            "FaceQuality": 0.942,
            "Attractive": 0.044,
            "AgeSD": 7,
            "Glasses": "glasses",
            "FigureConfidence": 1,
            "Hat": "none"
        }
    ]
}

Clean up resources

In this tutorial, you created an ECS instance, an OSS bucket, an IMM project, a RAM user, and a RAM role. After you finish the tests, clean up these resources to avoid extra charges.