DocsICP FilingConsole
Documentation
Home Page Image Search User Guide Console Guide Batch operations

batch operation

Updated at:
Copy as MD
Product
My Favorites

Image Search supports batch operations to import images from Object Storage Service (OSS) or delete images that are already in the search index. The batch operation feature is stable, fast, and easy to use, making it ideal for managing large volumes of offline image data. This topic describes how to add and delete images in bulk.

Note

For more product details or technical support, click Online Consulting or contact us through our DingTalk group: 35035130.

Procedure

  1. Activate OSS.

  2. Import images into OSS.

  3. Create an increment.meta file.

  4. Authorize Image Search to access OSS.

  5. Create a batch task.

Activate OSS

If you have not activated the Object Storage Service (OSS), see Activate OSS. After you activate the service, you must create a bucket. For more information, see Create a bucket in the console.

Note

  • OSS is billed separately. For more information, see billing overview.

  • Ensure that your OSS service and bucket are in the same region as your Image Search instance.

Import images into OSS

To import images, see Upload files.

Note

You can use a custom storage path for your images. You only need to specify the image path when you create a batch task.

Create an increment.meta file

In the OSS path where your images are stored, create an increment.meta file. This file stores information related to the batch task.

Note

  1. The images referenced in the increment.meta file must be in the same bucket as the increment.meta file. Otherwise, the batch operation fails.

  2. The filename increment.meta is required and cannot be changed. Image Search uses this specific name to read batch content.

  3. Image Search supports batch operations on images in different paths within the same bucket. For example, assume the bucket is named imagesearch, the increment.meta file is in the root directory of this bucket, the path is girlCloth, and the image name is girl_cloth8.jpg. In the meta file, PicName must be set to "PicName":"girlCloth/girl_cloth8.jpg". Note: Do not add a forward slash (/) before girlCloth or after girl_cloth8.jpg. This logic applies to multi-level directories as well.

  4. You must include the file extension in the image names in OSS. For example, for an image named girl_cloth10.jpg, you must specify "PicName":"girl_cloth10.jpg", not "PicName":"girl_cloth10". Otherwise, the processing fails.

  5. Each line in the increment.meta file must represent a single image operation. Do not place multiple operations on the same line.

The following example shows a sample file in JSON format:

{"OperationType": "ADD","ProductId": "1000","PicName": "girl_cloth1.jpg","CategoryId": 0,"IntAttr": 0,"StrAttr": "value1","CustomContent": "k1:v1,k2:v2,k3:v3","Region": "20,40,60,80"}
{"OperationType": "ADD","ProductId": "1000","PicName": "girl_cloth2.jpg","CategoryId": 0,"IntAttr": 0,"StrAttr": "value2","CustomContent": "k1:v1,k2:v2,k3:v3","Region": "20,40,60,80"}
{"OperationType": "ADD","ProductId": "1001","PicName": "girl_cloth3.jpg","CategoryId": 1,"CustomContent": "k1:v1,k2:v2,k3:v3"}
{"OperationType": "ADD","ProductId": "1002","PicName": "girl_cloth4.jpg","CategoryId": 0,"CustomContent": "k1:v1,k2:v2,k3:v3","Crop": false}
{"OperationType": "ADD","ProductId": "1003","PicName": "girl_cloth7.jpg","CustomContent": "https://www.aliyun.com/imagesearch/girl_cloth7.jpg"}
{"OperationType": "ADD","ProductId": "1003","PicName": "girl_cloth6.jpg","CustomContent": "k1:v1,k2:v2,k3:v3"}
{"OperationType": "ADD","ProductId": "1006","PicName": "girlCloth/girl_cloth10.jpg","CustomContent": "k1:v1,k2:v2,k3:v3"}
{"OperationType": "DELETE","ProductId": "1004","PicName": "fengyi.jpg"}
{"OperationType": "DELETE","ProductId": "1005"}

The following example shows a sample file in CSV format:image

The following table describes the fields.

Parameter

Type

Required

Description

OperationType

String

Yes

The operation type.

  • ADD: Add an image.

  • DELETE: Delete an image.

ProductId

String

Yes

A unique identifier for the image. It can be up to 256 characters long.

PicName

String

Yes

The name of the image in the OSS bucket. It can be up to 256 characters long.

Important

  1. Each line can contain only one image name. If a single ProductId corresponds to multiple PicName values, you must specify each one on a separate line.

  2. You must include the file extension in the image name. For example, for the girl_cloth.jpg image, specify the name as girl_cloth.jpg, not girl_cloth. Otherwise, the processing fails.

  3. If an image has the same combination of ProductId and PicName as an existing image, the new image overwrites the previous one, regardless of whether the image content is different.

  4. The images to be added must meet the following requirements:

    • The image size cannot exceed 4 MB.

    • Supported formats: PNG, JPG, JPEG, BMP, GIF, WEBP, TIFF, and PPM.

    • The transfer timeout must not exceed 5 seconds.

    • For product image search, generic image search, furniture and household supply image search, and industrial hardware image search, the image width and height must be between 100 and 4,096 pixels.

    • For brand image search, the image width and height must be between 200 and 4,096 pixels.

    • For cloth image search, the image width and height must be between 448 and 4,096 pixels.

    • The image cannot contain rotation information.

CategoryId

Integer

No

For more information, see Category reference.

  • For product search, if you set a category, the specified category is used. The category must be within the range defined in the category reference. Otherwise, the image fails to be indexed.

  • If you do not set a category, the system predicts one. The predicted category can be found in the response.

  • For generic search, fabric, trademark, copyrighted image search, industrial hardware, and furniture and home goods image search, the system sets the category to 88888888, regardless of whether you specify one.

CustomContent

String

Yes

User-defined content. It can be up to 4,096 characters long.

Note

The search results include this value. You can use this field to store an image URL or other additional information to link the image to your business data.

IntAttr

Integer

No

An integer attribute used for conditional filtering.

Note

Use integer attributes in a filter condition to filter search results. The int_attr field supports up to 9 digits. This digit limit applies both when you enter the value in the console and when you call it in the AddImage operation. For more information, see Filter condition.

StrAttr

String

No

A string attribute up to 128 characters long, used for conditional filtering.

Note

  • Use string attributes in a filter condition to filter search results. For more information, see Filter condition.

  • Special characters such as \, ¥, $, &, and % are not supported.

Crop

Boolean

No

Specifies whether to enable subject identification. The default value is true.

  • true: The system performs subject identification and searches based on the identified subject.

  • false: The system searches the entire image without performing subject identification.

Note

For Fabric Image Search and Copyrighted Image Search, this parameter is ignored. The search is always based on the entire image.

Region

String

No

The subject area of the image. The format is x1,x2,y1,y2, where (x1, y1) are the coordinates of the upper-left corner and (x2, y2) are the coordinates of the lower-right corner.

Note

  • If you set this parameter, the Crop parameter is ignored, and this setting takes precedence.

  • For Fabric Image Search and Copyrighted Image Search, this parameter is ignored. The search is always based on the entire image.

Image Search authorization

Image Search performs batch operations by using OSS. You must complete a one-click authorization the first time you use this feature. If you have already granted authorization, you can skip this step.

  1. Log in to the Image Search console.

  2. Select the target service type and instance.

  3. Click the Batch Operation tab.

    批量操作

  4. Click Create Batch Task, and in the dialog box that appears, click Confirm Authorization.

  5. On the Cloud Resource Access Authorization page, click Confirm Authorization.

Create a batch task

  1. Log in to the Image Search console.

  2. Select the target service type and instance.

  3. Click the Batch Operation tab.

    image

  4. Click Create Batch Task.

    p278660.png

  5. Set the parameters for the batch task.

    • Region

      The region where the instance resides.

    • Bucket name

      The name of your OSS bucket.

      Important

      Ensure that the OSS service and the created bucket are in the same region as your Image Search instance.

    • Data path

      The storage path for the images to be uploaded and the increment.meta file.

      批量上传.png

    • Meta file

      • Self-written (recommended): A file that lists the images to be processed. The file must be named increment.meta. The system can check if the file exists. For more information, see Create an increment.meta file.

      • Automatic generation: This option automatically scans the specified data directory for all eligible image files based on the entered bucket name and data path. Use this option with caution. After the file is generated, you must download the meta file from the task list and upload it to the correct OSS path to start the batch task.image

        Note

        • The generated file contains only the required fields: OperationType (set to ADD), ProductId, and PicName. The values for PicName and ProductId are the same.

        • If you need to add other fields, download the generated file from the task list and modify it. For more information, see Create an increment.meta file.

    • Completion callback

      Enter the callback URL. It must start with HTTP or HTTPS.

      {
  "finishTime": "2021-05-19 17:50:00",
  "instanceId": "imagesearch-cn-xxxx",
  "instanceName": "instanceName",
  "message": "success",
  "processResultUrl": "https://image-search-task-info.oss-cn-shanghai.aliyuncs.com/yyyyyyyyyy",
  "status": "NORMAL",
  "taskId": 111
}

      Field

      Description

      finishTime

      The time the task completed, accurate to the second.

      instanceName

      The name of the Image Search instance.

      instanceId

      The ID of the Image Search instance.

      status

      The task execution result. NORMAL indicates success. FAIL indicates failure.

      taskId

      The task ID.

      message

      A description of the result. success indicates that the task is successful.

      processResultUrl

      The URL to download the error results.

Cancel a batch task

  1. To cancel a task that is in the Processing state, click Cancel.image

  2. Read the content in the pop-up window and click OK to stop the batch task.image

  3. After you cancel the batch task, any processed data is retained, and the task status changes to Stop. Click download results to view the execution details.image

Next steps

On the instance details page, you can view all batch tasks. For tasks in the Processing state, the task progress is displayed. After a batch task succeeds, you can view its corresponding message:image

  • You can search for an instance by Bucket Name or Data Path.

  • If you receive the prompt Some photos failed to process and have been separated for you, it means that some images in the incremental task failed to process. Click Download Results to download the corresponding file and view the reasons for the failure.

Previous：Image queryNext：Delete images