Upload and automatically decompress a ZIP package

更新时间:
复制 MD 格式

When you need to upload objects in batches, preserve a specific directory structure, upload a complete set of resources, or quickly distribute resources, you can configure a decompression rule. When you upload a ZIP file to a specified path in OSS, it triggers Function Compute to automatically decompress the file and save the extracted content to OSS.

Prerequisites

Function Compute is activated. You can activate the service on the Function Compute product page.

Use cases

  • Batch uploads: If you need to upload a large number of small objects, programmatic uploads can be time-consuming. You can use the ZIP decompression feature to package the objects and upload them as a single archive.

  • Upload complete resources: For resource integrity, you may need to group multiple objects into a single resource. Uploading objects separately can have a lower success rate. The ZIP decompression feature improves the upload reliability for these scenarios.

  • Upload with a specific directory structure: Websites often reference static resources with complex directory structures. Creating each directory and uploading objects one by one in OSS is inefficient. Instead, you can define the directory structure in a ZIP package locally, compress it, and upload it to OSS for automatic decompression.

  • Resource distribution: When you need to distribute a large number of objects to users or servers, you can package them into a ZIP archive. Using the OSS ZIP decompression feature, the archive is automatically extracted to a specified directory upon upload. This reduces transfer time and bandwidth consumption.

How it works

ZIP package decompression uses Function Compute to decompress ZIP files. The following diagram shows the decompression workflow.

image
  1. A ZIP file is uploaded to a directory that matches the prefix specified in the decompression rule.

  2. The upload event automatically triggers a function in Function Compute.

    When you configure the decompression rule, you must authorize the trigger. This allows OSS to use the trigger role, AliyunOSSEventNotificationRole, to invoke Function Compute. The authorization process automatically grants the necessary policy to the role.

  3. Function Compute decompresses the ZIP file and stores the extracted objects in the specified directory in OSS.

    When you configure the decompression rule, you must grant permissions to Function Compute. This creates a new role that Function Compute assumes to retrieve the ZIP package from OSS and write the decompressed objects to OSS. When the role is created, a policy that grants read and write permissions on the OSS bucket is attached to the role.

Billing

The ZIP package decompression feature is a value-added service and incurs fees for both OSS and Function Compute. Data is transferred between the bucket and Function Compute over an internal endpoint, so no data transfer fees are charged.

  • OSS: You are charged for API requests and storage. For more information, see OSS billing overview.

  • Function Compute: You are charged for vCPU, memory, and disk usage, billed based on execution duration. For more information, see FC billing overview.

    For example, a decompression task runs for 5 minutes (300 seconds) on a function with 2 vCPUs, 3 GB of memory, and a 10 GB temporary disk. The fees are calculated as follows:

    Item

    Unit

    Description

    Billing example

    vCPU usage

    vCPU-seconds

    Active vCPU usage = vCPU specification × Execution duration (seconds)

    If the function is configured with 2 vCPUs and runs for 300s, the vCPU usage for a single run is 2 × 300 = 600 vCPU-seconds.

    Memory usage

    GB-seconds

    Memory usage = Memory specification × Execution duration (seconds)

    If the function is configured with 3 GB of memory and runs for 300s, the memory usage for a single run is 3 × 300 = 900 GB-seconds.

    Disk usage

    GB-seconds

    Disk usage = Disk specification × Execution duration (seconds)

    Note: Disk usage up to 512 MB is free of charge.

    If the function is configured with a 10 GB temporary disk and runs for 300s, the billable disk usage is (10 - 512/1024) × 300 = 2,850 GB-seconds.

Limitations

  • Storage class: Objects of the Cold Archive or Deep Cold Archive storage class must be restored before they can be decompressed. Objects of the Archive storage class must also be restored if real-time access is not enabled for the bucket.

  • Object and directory naming: We recommend that you use UTF-8 or GB 2312 encoding for your object and directory names. Other encodings may cause garbled names or decompression interruptions.

  • ZIP package: The maximum size for a single object within a ZIP package is 1 GB. The default execution timeout is 2 hours. To extend this time, you can adjust the function's execution timeout. For more information, see Decompression of a large file fails because the task exceeds the 2-hour execution time limit.

Configure a ZIP package decompression rule

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets. On the Buckets page, find and click the desired bucket.

  3. In the left-side navigation pane, choose Data Processing > Decompress ZIP Package.

  4. Click Decompress ZIP Package. In the Decompress ZIP Package panel, configure the decompression rule.

    Parameters

    Parameter

    Required

    Description

    Service Authorization

    Yes

    Authorize Function Compute to read from and write to OSS, and to execute functions.

    Click Authorize and follow the on-screen instructions.

    Authorize Trigger

    Yes

    Authorize OSS to access Function Compute.

    Click Authorize and follow the on-screen instructions. After authorization, this field changes to Trigger Role.

    Prefix

    No

    Specifies an object prefix that triggers the function. The function is triggered when you upload a ZIP file to a directory matching the prefix. If this field is left empty, any uploaded ZIP file triggers the function.

    Important

    To prevent recursive executions, specify a prefix. For more information, see How do I prevent trigger loops?.

    Destination Directory

    No

    The directory where the extracted objects are stored. If you leave this empty, Function Compute extracts the objects to the root directory of the bucket.

    • To create a subdirectory named after the ZIP file, select Add the compressed file name to the path.

    • To extract objects directly into the destination directory without creating a subdirectory, select Decompress directly to the destination directory. See the examples below for details.

    Warning

    To prevent service disruption, data contamination, or data loss when configuring rules for ZIP package decompression on a bucket with OSS-HDFS enabled, do not set the Destination Directory to.dlsdata/.

    Examples

    Scenario

    Configuration

    Directory structure

    Decompress all ZIP packages from the zipfolder directory to the destfolder directory without preserving the ZIP package name as a subdirectory.

    • Set Prefix to zipfolder/.

    • Set Destination Directory to destfolder.

    • Select Decompress directly to the destination directory.

    bucket  ├─── zipfolder/   │    ├─── a.zip│    └─── b.zip└─── destfolder/     ├─── a.txt     ├─── b.txt     └─── ...

    Decompress all ZIP packages from the zipfolder directory to the root directory, preserving the ZIP package name as a subdirectory.

    Configure the following parameters:

    • Set Prefix to zipfolder/.

    • Leave Destination Directory empty.

    • Select Add the compressed file name to the path.

    bucket  ├─── zipfolder/   │    ├─── a.zip│    └─── b.zip├─── a/│    ├─── a.txt│    └─── ...└─── b/     ├─── b.txt     └─── ...

    Decompress all ZIP packages from the zipfolder directory to the destfolder directory, preserving the ZIP package name as a subdirectory.

    Configure the following parameters:

    • Set Prefix to zipfolder/.

    • Set Destination Directory to destfolder.

    • Select Add the compressed file name to the path.

    bucket  ├─── zipfolder/   │    ├─── a.zip│    └─── b.zip└─── destfolder/     ├─── a/     │    ├─── a.txt     │    └─── ...     └─── b/          ├─── b.txt          └─── ...
  5. Agree to the terms for activating Function Compute, and then click OK.

Modify a decompression rule

You can modify a ZIP package decompression rule as needed.

Modify the trigger prefix

  1. On the Decompress ZIP Package page in the OSS console, find the trigger that you want to modify and click Edit in the Actions column.

  2. On the Triggers tab of the function details page, find the trigger and click Modify in the Actions column.

  3. In the Edit Trigger dialog box, change the Object Prefix and keep the default settings for other parameters.

  4. Click OK.

Modify function configurations

  1. On the Configurations tab of the function details page in the Function Compute console, click Modify.

  2. Modify the function configurations, such as memory and execution timeout, as needed.

    For more information, see Manage functions.

Delete a trigger

Note

A deleted trigger cannot be restored or used to trigger functions. In-progress tasks will continue to run to completion.

  1. On the Triggers tab of the function details page, find the trigger that you want to delete and click Delete in the Actions column.

  2. In the dialog box that appears, click Delete.

Related documents

Package and download multiple objects as a ZIP package

FAQ

What do I do if an object upload fails to trigger a function?

Decompressing a ZIP file involves a coordinated call between OSS and Function Compute. After you upload a ZIP file to OSS, OSS triggers Function Compute to decompress the file. The trigger from OSS to Function Compute may fail. You can check the Base64-encoded value of the x-oss-event-status header in the response to determine whether the decompression was successfully triggered. If the decoded value of x-oss-event-status is {"Result": "Ok"}, the decompression was successfully triggered. If the decoded value of x-oss-event-status is not {"Result": "Ok"}, the trigger failed, and you should re-upload the ZIP file. For more information, see Simple Upload.

Note

The maximum size for a single object in a ZIP package is 1 GB. The maximum decompression time for a single package is 2 hours.

What do I do if a task to decompress a large file fails because it exceeds the execution timeout?

The default execution timeout for a function is 2 hours. After you configure a decompression rule, you can change this time to a maximum of 24 hours in the Function Compute console.

  1. On the Decompress ZIP Package page in the OSS console, find the trigger and click Edit in the Actions column.

  2. On the function details page that appears, click the Configurations tab.

  3. In the Environment Information section, click Modify.

  4. Modify the Execution Timeout Period and click OK.

What do I do if an object in the ZIP package is larger than 1 GB?

If an object in the ZIP package is larger than 1 GB, you can use Apsara File Storage NAS as a temporary directory for decompression. This solution may incur minor NAS storage fees. For more information, see unzip-oss-nas.

Does this feature support nested ZIP packages?

No.

After a decompression rule is configured, uploading a ZIP package to the bucket triggers a single decompression task. If the extracted files include another ZIP package, that inner package is not decompressed. To decompress the inner package, you must upload it to the bucket separately.

Can I decompress a ZIP package to another bucket?

No.

The ZIP package decompression feature only supports extracting objects to a destination directory within the same bucket. To decompress to another bucket, you must customize the function code.

Can I view the extracted objects in the destination directory while the decompression is in progress?

Yes.

Function Compute uploads objects to OSS as it decompresses them. You can view the extracted objects in the destination directory at any time during the process.

How do I determine whether the decompression is complete?

You can check the function's invocation log to determine whether the decompression is complete.

  1. On the Decompress ZIP Package page in the OSS console, find the trigger and click Edit in the Actions column.

  2. On the function details page that appears, click the Logs tab.

  3. Click Enable.

  4. Grant the AliyunLogFullAccess permission to the current RAM role. For more information, see Manage the permissions of a RAM role.

  5. Upload a ZIP file. Then, on the Function Logs tab, view the invocation log.

    The following log entry indicates that the function invocation has ended, which means the decompression is complete.

    image.png

Does this feature support the .rar and .tar.gz formats?

No.

The ZIP package decompression feature currently supports only the .zip format.

Does Function Compute send a notification to OSS after the decompression is complete?

No.

To check the decompression progress, view the Logs tab on the function details page or configure a destination for asynchronous invocations. For more information, see Configure destinations for asynchronous invocations.

Does OSS support online compression?

No.

If you need to package multiple objects from OSS and download them to a local device, you can use Function Compute. For more information, see Package and download multiple objects as a ZIP package.

Does this feature support split ZIP archives?

No.

The ZIP package decompression feature processes only single archives and does not merge content from multiple archives. Therefore, you cannot decompress large files that are split into multiple .zip archives.

Why is the ZIP Package Decompression button grayed out?

  • Make sure Function Compute is activated. You can activate the service on the Function Compute product page.

  • The ZIP package decompression feature is supported only in specific regions. For more information, see Limitations.