Process billions of objects using Batch Operations

Scope

The Batch Operations feature is fully available in the Singapore and Germany (Frankfurt) regions. To use this feature in the China (Hangzhou), China (Shanghai), China (Shenzhen), China (Guangzhou), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Chengdu), China (Hong Kong), or Indonesia (Jakarta) regions, contact Technical Support to apply for access.

Operation overview

1. Create a RAM role: Allow the OSS service to read the manifest file, execute batch operations, and generate a report.

1. Prepare an input manifest: Specify the objects to process using a bucket inventory file (manifest.json), a prefix, or a CSV manifest file (manifest.csv).

2. Create a job: Configure specific operations, such as tagging, setting permissions, or restoring objects, and then submit the job.

3. Manage and maintain the job: Monitor the job status, confirm pending jobs, view the execution report, and adjust the priority or cancel the job as needed.

Step 1: Create a RAM role

1. Go to the RAM console to create a role. Select Alibaba Cloud Service as the trusted entity type and Object Storage Service as the trusted service.

2. Create and attach a custom policy to the role. The policy content is as follows. Replace your-bucket with your actual bucket name.

   {
     "Version": "1",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "oss:GetBucketInfo",
           "oss:ListObjects",
           "oss:GetObject",
           "oss:GetObjectTagging",
           "oss:PutObjectTagging",
           "oss:DeleteObjectTagging",
           "oss:PutObjectAcl",
           "oss:RestoreObject",
           "oss:PutObject"
         ],
         "Resource": [
           "acs:oss:*:*:your-bucket",
           "acs:oss:*:*:your-bucket/*"
         ]
       }
     ]
   }

   Permission description

   • Required permissions

     • oss:GetBucketInfo: Retrieves bucket information.

   • Permissions related to the input manifest

     • oss:ListObjects: Required to generate an object list from a prefix.

     • oss:GetObject: Required to use a bucket inventory file or a CSV manifest file.

   • Permissions related to batch operations

     • oss:GetObjectTagging: Required to read existing tags for batch tag replacement.

     • oss:PutObjectTagging: Required for batch adding or replacing tags.

     • oss:DeleteObjectTagging: Required for batch tag deletion.

     • oss:PutObjectAcl: Required for batch permission modification.

     • oss:RestoreObject: Required for batch restoration of archived objects.

   • Permissions for report output

     • oss:PutObject: Required to upload the batch operation report.

3. Record the ARN of the role, for example, acs:ram::<your-account-uid>:role/BatchRole. You will use it in the following steps.

Step 2: Prepare an input manifest

Batch operations require you to specify which objects to process. You can choose one of the following three methods based on your scenario:

• Bucket inventory file: Suitable for large-scale, regularly repeated batch operations.

• Prefix: Processes all objects in a specific folder.

• CSV manifest file: A CSV file that lists the objects to be processed.

Bucket inventory file

Use a manifest file automatically generated by Bucket Inventory.

<Manifest>
    <Location>
        <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
        <Bucket>my-bucket</Bucket>
        <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
    </Location>
    <Spec>
        <Format>OSS_InventoryReport_CSV_20250611</Format>
    </Spec>
</Manifest>

Configuration description

• ETag: The ETag value of the manifest.json file.

• Format: OSS_InventoryReport_CSV_20250611

Prefix

Generates an object list based on a specified prefix.

<KeyPrefixManifestGenerator>
    <SourceBucket>my-bucket</SourceBucket>
    <Prefix>documents/2024/</Prefix>
</KeyPrefixManifestGenerator>

CSV manifest file

Lists specific objects using a CSV file. The CSV manifest file must be uploaded to OSS and located in the same region as the batch job.

<Manifest>
    <Location>
        <Bucket>my-bucket</Bucket>
        <Object>manifest.csv</Object>
        <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
    </Location>
    <Spec>
        <Format>OSS_BatchOperations_CSV_20250611</Format>
        <Fields>Bucket,Key</Fields>
    </Spec>
</Manifest>

Configuration description

• ETag: The ETag value of manifest.csv.

• Fields: The columns in the CSV file.

  • Versioning not enabled: Bucket,Key

  • Versioning enabled: Bucket,Key,VersionId

• Format: OSS_BatchOperations_CSV_20250611

CSV file format requirements

• File encoding: UTF-8.

• Field separator: Comma (,).

• Object Key encoding: URL-encoded (for example, documents%2Freport1.pdf).

• Comments: The CSV file cannot contain comment lines.

CSV file examples

• Versioning not enabled

  my-bucket,documents%2Freport1.pdf
  my-bucket,documents%2Freport2.pdf
  my-bucket,images%2Fphoto1.jpg

• Versioning enabled

  my-bucket,important%2Fdata.json,3/L4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo
  my-bucket,config%2Fsettings.xml,null

Step 3: Create a job

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectTagging>
            <TagSet>
                <Tag><Key>Environment</Key><Value>Production</Value></Tag>
                <Tag><Key>Team</Key><Value>DataOps</Value></Tag>
            </TagSet>
        </PutObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>put-tag-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch replace tags using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectTagging>
            <TagSet>
                <Tag><Key>Environment</Key><Value>Production</Value></Tag>
                <Tag><Key>Team</Key><Value>DataOps</Value></Tag>
            </TagSet>
        </PutObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>put-tag-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch replace tags for documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectTagging>
            <TagSet>
                <Tag><Key>Environment</Key><Value>Production</Value></Tag>
                <Tag><Key>Team</Key><Value>DataOps</Value></Tag>
            </TagSet>
        </PutObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>put-tag-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch replace tags for specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>ODRiZWQ5Njc5ZTE0NDE5NTlmNjJmZDlkMTk2ODI2NDU=</JobId>
</CreateJobResult>

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <AddObjectTagging>
            <TagSet>
                <Tag><Key>Department</Key><Value>Engineering</Value></Tag>
                <Tag><Key>Project</Key><Value>BatchOps</Value></Tag>
            </TagSet>
        </AddObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>add-tag-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch add tags using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <AddObjectTagging>
            <TagSet>
                <Tag><Key>Department</Key><Value>Engineering</Value></Tag>
                <Tag><Key>Project</Key><Value>BatchOps</Value></Tag>
            </TagSet>
        </AddObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>add-tag-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch add tags to documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <AddObjectTagging>
            <TagSet>
                <Tag><Key>Department</Key><Value>Engineering</Value></Tag>
                <Tag><Key>Project</Key><Value>BatchOps</Value></Tag>
            </TagSet>
        </AddObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>add-tag-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch add tags to specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>Nzg5ZjE3ZTU5OGMxNDZjZDk0YjJiODMzZDZiZjA0ZmU=</JobId>
</CreateJobResult>

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <DeleteObjectTagging />
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>delete-tag-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch delete tags using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <DeleteObjectTagging />
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>delete-tag-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch delete tags for documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <DeleteObjectTagging />
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>delete-tag-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch delete tags for specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>NTVlYmQ4ZmQyOTM0NDJmY2E2NmNjYTUwNjU1ZGRkMGM=</JobId>
</CreateJobResult>

Batch modify permissions

Batch permission modification supports four access control list (ACL) types: private (only the owner can read and write), public-read (anyone can read, but only the owner can write), public-read-write (anyone can read and write), and default (inherits the permissions of the bucket). Permission changes take effect immediately and directly affect object access. Choose the appropriate permission type carefully based on your business needs.

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectAcl>
            <ObjectAcl>private</ObjectAcl>
        </PutObjectAcl>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>acl-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch change ACL using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectAcl>
            <ObjectAcl>private</ObjectAcl>
        </PutObjectAcl>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>acl-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch change ACL to private for documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectAcl>
            <ObjectAcl>private</ObjectAcl>
        </PutObjectAcl>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>acl-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch change ACL to private for specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>YWNsR29iMTIzNDU2Nzg5MA==</JobId>
</CreateJobResult>

Batch data restore

The batch restore operation supports objects in the Archive, ColdArchive, and DeepColdArchive storage classes. Restoring Archive objects typically completes within one hour, and the restored objects remain accessible for 1 to 7 days. The ColdArchive storage class supports Standard mode (2 to 5 hours) and Bulk mode (5 to 12 hours). The DeepColdArchive storage class only supports Standard mode, which takes up to 48 hours to complete. You can set the number of days that the restored object remains accessible to a value from 1 to 365.

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <RestoreObject>
            <Days>7</Days>
            <Tier>Standard</Tier>
        </RestoreObject>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>restore-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch restore using Inventory manifest for 7 days</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <RestoreObject>
            <Days>7</Days>
            <Tier>Standard</Tier>
        </RestoreObject>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>restore-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>archive/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch restore archive objects for 7 days</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <RestoreObject>
            <Days>7</Days>
            <Tier>Standard</Tier>
        </RestoreObject>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>restore-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch restore specified archive objects for 7 days</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>cmVzdG9yZUpvYjEyMzQ1Njc4OTA=</JobId>
</CreateJobResult>

Step 4: Job management and O&M

View job status

After creating a job, you can monitor and confirm its status by querying its details. The job details provide a complete overview of the job's status, including its execution progress and configuration parameters.

ossutil api invoke-operation --op-name describe-job --method GET --parameters batchJob --parameters batchJobId=<JobId> --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<DescribeJobResult>
  <Job>
    <ConfirmationRequired>true</ConfirmationRequired>
    <CreationTime>1761723644</CreationTime>
    <FailureReasons>
      <JobFailure />
    </FailureReasons>
    <JobId>M2E0ODMyNmJjYmYzNGY0ZThiYjMyMmI1ZDE1YWE0OGE=</JobId>
    <Operation>
      <PutObjectTagging>
        <TagSet>
          <Tag>
            <Key>Environment</Key>
            <Value>Production</Value>
          </Tag>
          <Tag>
            <Key>Team</Key>
            <Value>DataOps</Value>
          </Tag>
        </TagSet>
      </PutObjectTagging>
    </Operation>
    <Report>
      <Enabled>true</Enabled>
      <Bucket>my-bucket</Bucket>
      <Prefix>batch-reports/</Prefix>
      <ReportScope>AllTasks</ReportScope>
    </Report>
    <Manifest>
      <Location>
        <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
        <Bucket>my-bucket</Bucket>
        <Object>manifest.csv</Object>
      </Location>
      <Spec>
        <Format>OSS_BatchOperations_CSV_20250611</Format>
        <Fields>Bucket,Key</Fields>
      </Spec>
    </Manifest>
    <Description>Test task for Suspended status</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
    <ProgressSummary>
      <NumberOfTasksFailed>0</NumberOfTasksFailed>
      <NumberOfTasksSucceeded>0</NumberOfTasksSucceeded>
      <TotalNumberOfTasks>3</TotalNumberOfTasks>
      <Timers>
        <ElapsedTimeInActiveSeconds>-1</ElapsedTimeInActiveSeconds>
      </Timers>
    </ProgressSummary>
    <Status>Suspended</Status>
    <TerminationDate>-1</TerminationDate>
  </Job>
</DescribeJobResult>

Confirm a pending job

You can change the job status from Suspended to Ready to place it in the execution queue. If a job is not confirmed within 14 days, its status automatically changes to Failed.

ossutil api invoke-operation --op-name update-job-status --method POST --parameters batchJobStatus --parameters batchJobId=<JobId> --parameters requestedJobStatus=Ready --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<UpdateJobStatusResult>
  <JobId>NGFhNmFmZDRlNTcxNDE2OGFhY2FjYWQyNDcxYjc3YzM=</JobId>
  <Status>Ready</Status>
</UpdateJobStatusResult>

List jobs in batches

You can view all jobs and use the batch query feature to filter the list by status and creation time range. You can also limit the number of results and use paged queries to manage many jobs. By default, results are sorted by creation time in descending order, with the most recent jobs appearing first.

ossutil api invoke-operation --op-name list-jobs --method GET --parameters batchJob --parameters maxKeys=100 --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<ListJobsResult>
  <Jobs>
    <Job>
      <JobId>OTE5ZGU1NWMxNjkyNGFkMjhkYTNmOGEzNTU2YmJiOWE=</JobId>
      <Status>Complete</Status>
      <CreationTime>1761666548</CreationTime>
    </Job>
  </Jobs>
</ListJobsResult>

Adjust job priority

The priority value ranges from 0 to 2147483647, where a higher value indicates a higher priority. Job priority does not guarantee a strict execution order. Instead, OSS attempts to allocate more resources to higher-priority jobs.

ossutil api invoke-operation --op-name update-job-priority --method POST --parameters batchJobPriority --parameters batchJobId=<JobId> --parameters priority=1000 --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<UpdateJobPriorityResult>
  <JobId>OTE5ZGU1NWMxNjkyNGFkMjhkYTNmOGEzNTU2YmJiOWE=</JobId>
  <Priority>1000</Priority>
</UpdateJobPriorityResult>

Cancel a job

You can terminate a job that is running or waiting to run. Completed operations on objects cannot be rolled back.

ossutil api invoke-operation --op-name update-job-status --method POST --parameters batchJob --parameters batchJobId=<JobId> --parameters requestedJobStatus=Cancelled --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<UpdateJobStatusResult>
  <JobId>OTE5ZGU1NWMxNjkyNGFkMjhkYTNmOGEzNTU2YmJiOWE=</JobId>
  <Status>Cancelled</Status>
</UpdateJobStatusResult>

View a job report

After a job is complete, the system generates a detailed execution report in the specified path. The report contains two key files: a manifest named manifest.json and a detailed report in CSV format. These files provide the processing status, HTTP response code, and failure reason for each object, which you can use for troubleshooting and result validation.

<Bucket>/<Prefix>/<JobId>/results/

<Report>
    <Bucket>my-bucket</Bucket>
    <Prefix>batch-reports/</Prefix>
    <ReportScope>AllTasks</ReportScope>
</Report>

{
  "Format": "Report_CSV_20250901",
  "ReportCreationDate": "2025-10-31T07:18:56Z",
  "Results": [
    {
      "TaskExecutionStatus": "succeeded",
      "Bucket": "test-bucket",
      "MD5Checksum": "\"424356315f4a82c1b6a4c190dbed15d9\"",
      "Key": "reports/prefix-job-0/job-MTRiZGI1NDFiMTI2NDFiNDg0YWQyYTM2YmY1NzNjNDM=/results/report-0.csv"
    }
  ],
  "ReportSchema": "Bucket,Key,VersionId,HTTPStatus,Error"
}

{
  "Format": "Report_CSV_20250901",
  "ReportCreationDate": "2025-10-31T07:37:56Z",
  "Results": [
    {
      "TaskExecutionStatus": "failed",
      "Bucket": "test-bucket",
      "MD5Checksum": "\"85749f22b5b62b161daf00b4adcf6da7\"",
      "Key": "reports/prefix-job-0/job-M2E4MDkwNGJmNmQzNDgwYmE4ODI1YjdjNDgyNjk5NmM=/results/report-0.csv"
    }
  ],
  "ReportSchema": "Bucket,Key,VersionId,HTTPStatus,Error"
}

test-bucket,batchOperation%2F1761895099%2Fobject-1.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-9.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-4.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-8.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-3.txt,,200,

test-bucket,batchOperation%2F1761896253%2Fobject-0.txt,,403,"Error returned by Service. {""EC"":""0003-00000001"",""Error Code"":""AccessDenied"",""Http Status Code"":403,""Message"":""You have no right to access this object."",""Request Endpoint"":""PUT http://test-bucket.oss-cn-hangzhou-internal.aliyuncs.com/batchOperation/1761896253/object-0.txt?tagging"",""Request Id"":""690467520D9F2E3232004FFD"",""Timestamp"":""2025-10-31T07:37:54Z""}"
test-bucket,batchOperation%2F1761896253%2Fobject-7.txt,,403,"Error returned by Service. {""EC"":""0003-00000001"",""Error Code"":""AccessDenied"",""Http Status Code"":403,""Message"":""You have no right to access this object."",""Request Endpoint"":""PUT http://test-bucket.oss-cn-hangzhou-internal.aliyuncs.com/batchOperation/1761896253/object-7.txt?tagging"",""Request Id"":""69046752A016113937EB7029"",""Timestamp"":""2025-10-31T07:37:54Z""}"

# Download the report manifest file
ossutil cp oss://my-bucket/batch-reports/<JobId>/results/manifest.json ./

# Download the detailed CSV report
ossutil cp oss://my-bucket/batch-reports/<JobId>/results/report-0.csv ./

Going live

Best practices

Job planning strategies

• Scale control: Limit the number of objects in a single job to fewer than 100 million to ensure stable execution. Split large-scale data processing into multiple jobs to mitigate single points of failure.

• Priority management: Set reasonable job priorities to prevent important jobs from being blocked. Estimate execution times to schedule maintenance windows appropriately.

• Notes on concurrent operations: When multiple jobs operate on the same batch of objects concurrently, the execution order is not guaranteed. Avoid performing concurrent operations on the same object.

Performance optimization measures

• Manifest file optimization: Organize the manifest file structure to avoid creating excessively large single files, which can slow down parsing.

• QPS quota utilization: Take full advantage of the independent QPS quota to ensure that business API calls are not affected. Batch operations do not consume your QPS quota.

• Concurrency control: Limit the number of concurrently running jobs to three or fewer to avoid resource competition.

• Hot spot avoidance: If hot spots, such as a lexicographically ordered prefix, exist in the operation range, the execution speed of the batch operation can be affected. In severe cases, this may cause some file operations to fail.

Error handling mechanisms

• Detailed reports: Enable detailed reports (ReportScope=AllTasks) to obtain complete execution results for troubleshooting.

• Retry mechanism: In rare cases, if a batch operation encounters a problem during execution, it automatically retries several times. The final API request charges are based on the total number of API calls made.

• Failure threshold: If the failure rate for tasks in a job exceeds 20%, the entire job fails immediately and stops running.

• Status confirmation anomalies: In rare cases, some operations may succeed but be recorded as failed in the execution report. Double-check any items recorded as failed.

Key points for result confirmation

• Execution integrity verification: A job can have one of the following statuses: Running, Complete, Failed, Paused, or Cancelled. Some objects may have been processed successfully, while others may have failed. Check the number of successful and failed tasks in the report to ensure the results meet your expectations.

• Suggestions for handling failures:

  • A Complete status indicates only that the job has finished, not that all objects were processed successfully. You must check the execution report for details.

  • Analyze the cause of failed tasks and retry them. For jobs with partial failures, you can create a retry job based on the failure report.

  • For jobs with a Cancelled, Paused, or Failed status, some objects may have already been processed successfully. You must verify the results.

Notes on manifest files

Execution order and data consistency

• Execution order: Batch operations are performed on the items in the manifest list, but the execution order is not guaranteed.

• File updates: If an object is updated before the action is performed, the job operates on the updated file. Do not modify files while a batch job is running.

• Limitations of the prefix method: If you use a prefix to provide the input manifest, the batch operation cannot detect files that are added or deleted during its execution. Do not add or delete files under the prefix while a batch job is running.

Manifest file format requirements

• URL encoding: Object keys in the CSV manifest file must be URL-encoded. Otherwise, parsing may fail.

• Handling duplicate keys: If the manifest file contains duplicate keys, the batch operation is performed multiple times, but the order of operations is not guaranteed.

Resource scheduling mechanism

Priority scheduling

Job priorities are supported, with values ranging from 0 to 2147483647.

Concurrency control

A single account can run a maximum of six concurrent jobs. Additional jobs are placed in a waiting queue.

Job status transitions

Quotas and limits

Billing description

OSS Batch Operations is a pay-as-you-go service. You are charged CNY 1 for every 1 million objects processed, with no additional fees for job creation, management, or data transfer.

Billing examples:

• Batch tagging 5 million objects: 5 million / 1 million × CNY 1 = CNY 5

• Large-scale data processing of 5 billion objects: 5 billion / 1 million × CNY 1 = CNY 5,000

Cost optimization suggestions: Consolidate related batch operations into a single job to reduce management overhead. Use precise prefixes or manifest files to avoid processing unnecessary objects. Schedule non-urgent batch operations during off-peak business hours.

FAQ

Why is my job stuck in the Preparing state for a long time?

Possible reasons include the following:

• The manifest file is too large and takes a long time to parse.

• The prefix scan covers many objects.

• The system load is high and the processing queue is busy.

To resolve this, you can split large jobs into smaller ones or use a pre-generated manifest file.

What should I do if some objects fail to process?

Follow these steps:

1. Download the job execution report and analyze the cause of the failure.

2. Create a new retry job based on the list of failed objects.

3. Verify that the permission configuration is correct and that the IAM role has full permissions.

How do I cancel a running job?

You can use the following command:

POST /?batchJobStatus&batchJobId=<JobId>&requestedJobStatus=Cancelled&statusUpdateReason=User cancelled HTTP/1.1
Host: oss-cn-hangzhou.aliyuncs.com
Authorization: <Signature information>

Why do I get a permission denied error?

Check the following permission configurations:

1. Ensure the RAM user has the oss:CreateBatchJob permission.

2. Verify that the IAM role has the required permissions for the operation.

3. Verify the permission settings of the destination bucket for cross-bucket operations.

IAM Role Configuration Requirements?

The IAM role must include the following:

• Read and write permissions for OSS resources.

• Permissions to call Batch Operations APIs.

• Write permissions for the report storage bucket.

Example trust policy:

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": ["oss.aliyuncs.com"]
      }
    }
  ]
}

How can I improve the execution speed of batch operations?

Optimization recommendations:

1. Structure your manifest files to avoid creating single files that are too large.

2. Avoid running large jobs during peak business hours.

3. Assign job priorities to ensure important jobs are executed first.

4. Avoid including too many objects in a single job to prevent long runtimes. We recommend a limit of 100 million objects per job.

5. Avoid splitting a large job into too many smaller jobs, such as splitting 1 million objects into 1,000 jobs. This increases the time spent on task scheduling.

Why is the job taking longer than expected to run?

Possible reasons include the following:

• Objects are unevenly distributed, which causes some shards to take longer to process.

• The source or destination objects have hot spots, which can slow down processing.

• Too many jobs are running concurrently, which leads to resource competition.

We recommend adjusting the job scheduling strategy or reducing the number of concurrent jobs.