Configure a ZIP decompression trigger

更新时间:
复制 MD 格式

After you configure a decompression rule with a file prefix and destination directory, the system automatically decompresses matching ZIP files that you upload to your OSS bucket and saves the extracted files to that directory. This feature, built on the integration of FC and OSS, creates a native OSS trigger in FC for a code-free, event-driven workflow.

How it works

The ZIP decompression feature uses FC to decompress ZIP files. The process is shown in the following figure.

  1. A ZIP file that matches the decompression rule is uploaded to a directory matching the configured prefix.

  2. An FC function is automatically triggered.

  3. FC decompresses the ZIP file and stores the extracted files in the specified destination directory in OSS.

Use cases

  • Batch uploads

    When you need to upload a large number of small files, uploading them individually is time-consuming. You can use the ZIP decompression feature to package the files and upload them as a single archive.

  • Complete file uploads

    To ensure data integrity, you can group multiple files into a single resource. Uploading them separately can be unreliable. The ZIP decompression feature ensures all files are uploaded together, improving the success rate.

  • Uploads with a specific directory structure

    A website may reference many static resources with a complex directory structure. Creating each directory and uploading files one by one in OSS is inefficient. You can define the directory structure within a ZIP file on your local machine, upload the compressed file to OSS, and use the ZIP decompression feature to extract it.

  • Resource distribution

    When you need to distribute a large number of files to users or servers, you can package them into a ZIP file. Using the OSS ZIP decompression feature, you can extract the files to a specified directory, which reduces data transfer time and bandwidth consumption.

Billing

The ZIP decompression feature incurs costs for both OSS storage and FC function calls and resource usage. The details are as follows:

  • OSS fees

    You are charged a request fee for accessing resources and a storage fee for storing files in an OSS bucket.

  • FC fees

    You are charged for the number of function invocations and the vCPU, memory, and disk resources that invocations consume. These billable items are converted into CU-seconds for billing. For more information, see Billing overview.

Data transfer over an internal endpoint is free of charge.

Limitations

  • Storage class limitations

    You must restore objects in the Cold Archive or Deep Cold Archive storage class before they can be decompressed. For objects of the Archive storage class, if Archive Direct Read is not enabled for the bucket, the objects must be restored before they can be decompressed.

  • File and folder naming limitations

    Use UTF-8 or GB 2312 encoding for your file and folder names. Otherwise, the names of the decompressed files or folders may be garbled, or the decompression process may be interrupted.

  • Package limitations

    The size of a ZIP package cannot exceed 10 GB, and the size of a single file in the package cannot exceed 10 GB. The default maximum execution time for decompressing a single package is 2 hours. If decompression takes longer than 2 hours, you can adjust the function's execution timeout. For more information, see Decompression of a large file fails because the task takes more than two hours to complete.

Configure a ZIP decompression trigger

  1. Log on to the OSS console. In the left-side navigation pane, click Buckets.

  2. On the Buckets page, click the name of the target bucket.

  3. In the left-side navigation pane of the target bucket page, choose . Then, click ZIP package decompression. On the ZIP package decompression panel, configure the decompression rule.

    Parameter

    Description

    Example

    Service authorization

    Create a service-linked role for FC so it can access other cloud services. If you have not logged on to the FC console before, click Authorize to create the AliyunServiceRoleForFC service-linked role.

    Not applicable

    Function authorization

    Grant FC the permissions to read from and write to OSS.

    Click Authorize and follow the on-screen instructions to grant the permissions.

    Not applicable

    Trigger authorization

    Grant OSS the permissions to access FC.

    Click Authorize and follow the on-screen instructions to grant the permissions. If you have granted the permissions, this parameter is displayed as Trigger role.

    Not applicable

    Prefix

    Specify a file prefix for the trigger. If you upload a ZIP file with the specified prefix or upload a ZIP file to a directory with the specified prefix, the FC function is triggered. If you leave this parameter empty, the trigger matches all uploaded ZIP files.

    Important

    If you leave this parameter empty, circular triggers can occur. Specify a file prefix to prevent this issue. For more information, see How do I prevent circular triggers?

    zipfolder/

    Destination directory

    Specify the destination directory for the extracted files. If you leave this parameter empty, FC extracts the files to the root directory of the current bucket. For information about the file structure after decompression when different destination directories are configured, see the configuration examples in this topic.

    Warning

    To prevent impacts on the regular use of the OSS-HDFS service or to avoid risks of data contamination and data loss, do not set the destination directory to .dlsdata/ when you configure a ZIP decompression rule for a bucket for which the OSS-HDFS service is enabled.

    • The destination directory is destfolder, and Decompress directly to destination directory is selected.

      bucket
      ├─── zipfolder/
      │ ├─── a.zip
      │ └─── b.zip
      └─── destfolder/
      ├─── a.txt
      ├─── b.txt
      └─── ...
    • The destination directory is empty, and Keep compressed file name as a directory is selected.

      bucket/
      ├── zipfolder/
      │   ├── a.zip
      │   └── b.zip
      ├── a/
      │   ├── a.txt
      │   └── ...
      └── b/
          ├── b.txt
          └── ...
    • The destination directory is destfolder, and Keep compressed file name as a directory is selected.

      bucket/
      ├── zipfolder/
      │   ├── a.zip
      │   └── b.zip
      └── destfolder/
          ├── a/
          │   ├── a.txt
          │   └── ...
          └── b/
              ├── b.txt
              └── ...

After completing the configuration, you can modify the trigger's file prefix and related FC function settings. For more information, see Configure a native OSS trigger and Function instance specifications and auto scaling rules.

Related documents

FAQ

FC function trigger failure

ZIP decompression involves associated calls between OSS and FC. After you upload a ZIP file to OSS, OSS triggers an FC function to decompress the file. The call from OSS to trigger the FC function may fail. You can check the value of x-oss-event-status (Base64-encoded) in the response header to determine if the decompression triggered successfully. If the decoded value of x-oss-event-status is {"Result": "Ok"}, the trigger was successful. If the decoded value of x-oss-event-status is not {"Result": "Ok"}, the trigger failed. In this case, upload the ZIP file again. For more information, see Simple upload.

Note

The size of a single file in a ZIP package cannot exceed 10 GB. The maximum time allowed for decompressing a single package is 2 hours.

Decompression task timeout

The function execution timed out. You can adjust the function's execution timeout as needed. The maximum timeout period is 24 hours.

  1. In the OSS console, find the target bucket. On the Decompress ZIP Package page, click Modify in the row of the target trigger.

  2. On the displayed function details page, click the Configuration tab. In the left-side navigation pane, select Runtime and then click Modify.

  3. On the runtime editing panel, modify the Execution Timeout Period period and click Deploy.

Handling files larger than 10 GB

If a ZIP package exceeds 10 GB or a single file in the package exceeds 10 GB, you can use Apsara File Storage NAS as a temporary working directory for decompression. This solution may incur a small amount of NAS storage fees. For more information, see unzip-oss-nas.

Decompression of nested ZIP archives

No.

After you configure a ZIP decompression rule and upload a ZIP package to an OSS bucket, an FC function is triggered to decompress the package and transfer the extracted files to the specified directory in OSS. If the extracted files contain another ZIP package, the nested ZIP package cannot be decompressed. You can re-upload the nested ZIP package to the OSS bucket to trigger another decompression.

Decompression to another bucket

No.

The ZIP decompression feature supports only decompressing files into the destination directory that is specified when you configure the ZIP decompression rule. To decompress files into another bucket, you must customize the function code.

Viewing files during decompression

Yes.

When you use the ZIP decompression feature, FC decompresses the files and uploads them to OSS simultaneously. You can view the decompressed files in the destination directory at any time.

Determining decompression completion

You can check the file list of the target bucket to see if the files have been decompressed to the destination directory, or view the function invocation log in the FC console to check if the decompression is complete.

  1. On the Decompress ZIP Package page of the OSS console, click Modify in the row of the target trigger.

  2. On the displayed function details page, click the Logs tab.

    If the logging feature is not enabled for the function, click Enable with one click.

  3. Upload a ZIP package and check the Logs tab to see if the function was called successfully.

Decompression of .rar and .tar.gz files

No.

The ZIP decompression feature supports only .zip files. To decompress other types of compressed files, see decompress-oss.

Return message after decompression

No.

To view the decompression progress, go to the Invocation logs tab on the function details page, or configure an asynchronous destination for the decompression function. For more information, see Configure a destination for an asynchronous invocation.

Online compression support

No.

To package multiple objects from OSS and download them to your on-premises machine, you can use FC for packaging and downloading. For more information, see Package multiple files into a ZIP package for download.

Decompression of split ZIP archives

No.

The ZIP decompression feature does not merge the content of multiple decompressed packages and therefore does not support decompressing large files that are split into multiple ZIP archives.

Decompression button is disabled

Check if you have granted the required authorizations as prompted, such as creating the AliyunServiceRoleForFC service-linked role or granting FC permissions to upload files to OSS.