How to detect image labels-Object Storage Service(OSS)-阿里云帮助中心

Image label detection identifies label information such as scenes, objects, and events in an image. Use it to tag your images automatically.

Use cases

Scenario	Description
Content recognition	Detect items, scenes, and other information in captured or uploaded images for object recognition or educational applications.
Smart album	Classify images automatically based on content to organize photo albums and galleries without manual effort.
Scene analysis	Detect objects and scenes in images, then apply content labels to reduce manual annotation costs.
Content operations	Retrieve image labels for content recommendation on social media, news, and e-commerce platforms.

Use cases

Scenario	Description
Content recognition	Detect items, scenes, and other information in captured or uploaded images for object recognition or educational applications.
Smart album	Classify images automatically based on content to organize photo albums and galleries without manual effort.
Scene analysis	Detect objects and scenes in images, then apply content labels to reduce manual annotation costs.
Content operations	Retrieve image labels for content recommendation on social media, news, and e-commerce platforms.

Use cases

Scenario	Description
Content recognition	Detect items, scenes, and other information in captured or uploaded images for object recognition or educational applications.
Smart album	Classify images automatically based on content to organize photo albums and galleries without manual effort.
Scene analysis	Detect objects and scenes in images, then apply content labels to reduce manual annotation costs.
Content operations	Retrieve image labels for content recommendation on social media, news, and e-commerce platforms.

Use cases

Scenario	Description
Content recognition	Detect items, scenes, and other information in captured or uploaded images for object recognition or educational applications.
Smart album	Classify images automatically based on content to organize photo albums and galleries without manual effort.
Scene analysis	Detect objects and scenes in images, then apply content labels to reduce manual annotation costs.
Content operations	Retrieve image labels for content recommendation on social media, news, and e-commerce platforms.

Precautions

Image label detection supports only images in JPG, PNG, or JPEG format.
The following limits apply to image size:
- The image size cannot exceed 20 MB.
- The image height or width cannot exceed 30,000 pixels.
- The total number of pixels in the image cannot exceed 250 million.
Image label detection supports only synchronous processing, by using the x-oss-process method.
Anonymous access will be denied.

Anonymous access will be denied.

How to use

Prerequisites

Activate Intelligent Media Management (IMM).
In OSS, create a bucket and upload the files that you want to process to the bucket.
Create and attach an IMM project. You can attach it in the OSS console or by calling an API. The IMM project must be in the same region as the bucket.
Grant the user the required permissions.

Detect image labels

The following examples show how to detect labels using common SDKs. Adapt the code from these examples for other SDKs.

Python

Use Python SDK 2.18.4 or later.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables.
# Set OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET before running this code.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Set the endpoint for the region where the bucket is located.
endpoint = 'https://oss-cn-hangzhou.aliyuncs.com'
region = 'cn-hangzhou'
bucket = oss2.Bucket(auth, endpoint, 'examplebucket', region=region)

# Specify the object key. Include the full path if the image is not in the root directory,
# for example, exampledir/example.jpg.
key = 'example.jpg'

process = 'image/labels'

try:
    result = bucket.get_object(key, process=process)
    image_labels = result.read().decode('utf-8')
    print("Image labels:")
    print(image_labels)
except oss2.exceptions.OssError as e:
    print("Error:", e)

Java

Use Java SDK 3.17.4 or later.

import com.aliyun.oss.ClientBuilderConfiguration;
import com.aliyun.oss.OSS;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.common.auth.CredentialsProviderFactory;
import com.aliyun.oss.common.auth.EnvironmentVariableCredentialsProvider;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.model.OSSObject;
import com.aliyun.oss.model.GetObjectRequest;
import com.aliyuncs.exceptions.ClientException;

import java.io.ByteArrayOutputStream;
import java.io.IOException;

public class Demo {
    public static void main(String[] args) throws ClientException, ClientException {
        // Set the endpoint for the region where the bucket is located.
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        String region = "cn-hangzhou";
        // Obtain access credentials from environment variables.
        // Set OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET before running this code.
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        String bucketName = "examplebucket";
        // Specify the object key. Include the full path if the image is not in the root directory,
        // for example, exampledir/example.jpg.
        String key = "example.jpg";

        ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
        clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);
        OSS ossClient = OSSClientBuilder.create()
                .endpoint(endpoint)
                .credentialsProvider(credentialsProvider)
                .clientConfiguration(clientBuilderConfiguration)
                .region(region)
                .build();

        try {
            GetObjectRequest getObjectRequest = new GetObjectRequest(bucketName, key);
            getObjectRequest.setProcess("image/labels");

            OSSObject ossObject = ossClient.getObject(getObjectRequest);

            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            byte[] buffer = new byte[1024];
            int bytesRead;
            while ((bytesRead = ossObject.getObjectContent().read(buffer)) != -1) {
                baos.write(buffer, 0, bytesRead);
            }
            String imageLabels = baos.toString("UTF-8");
            System.out.println("Image labels:");
            System.out.println(imageLabels);
        } catch (IOException e) {
            System.out.println("Error: " + e.getMessage());
        } finally {
            ossClient.shutdown();
        }
    }
}

Go

Use Go SDK 3.0.2 or later.

package main

import (
	"fmt"
	"io"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Obtain access credentials from environment variables.
	// Set OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET before running this code.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Set the endpoint for the region where the bucket is located.
	client, err := oss.New("https://oss-cn-hangzhou.aliyuncs.com", "", "", oss.SetCredentialsProvider(&provider), oss.AuthVersion(oss.AuthV4), oss.Region("cn-hangzhou"))
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	bucket, err := client.Bucket("examplebucket")
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Specify the object key. Include the full path if the image is not in the root directory,
	// for example, exampledir/example.jpg.
	body, err := bucket.GetObject("example.jpg", oss.Process("image/labels"))
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	defer body.Close()

	data, err := io.ReadAll(body)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	fmt.Println("Image labels:", string(data))
}

PHP

Use PHP SDK 2.7.0 or later.

<?php
if (is_file(__DIR__ . '/../autoload.php')) {
    require_once __DIR__ . '/../autoload.php';
}
if (is_file(__DIR__ . '/../vendor/autoload.php')) {
    require_once __DIR__ . '/../vendor/autoload.php';
}
use OSS\Credentials\EnvironmentVariableCredentialsProvider;
use OSS\OssClient;

try {
    // Obtain access credentials from environment variables.
    // Set OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET before running this code.
    $provider = new EnvironmentVariableCredentialsProvider();
    // Set the endpoint for the region where the bucket is located.
    $endpoint = 'https://oss-cn-hangzhou.aliyuncs.com';
    $bucket = 'examplebucket';
    // Specify the object key. Include the full path if the image is not in the root directory,
    // for example, exampledir/example.jpg.
    $key = 'example.jpg';

    $config = array(
        "provider" => $provider,
        "endpoint" => $endpoint,
        "signatureVersion" => OssClient::OSS_SIGNATURE_VERSION_V4,
        "region" => "cn-hangzhou"
    );
    $ossClient = new OssClient($config);
    $options[$ossClient::OSS_PROCESS] = "image/labels";
    $result = $ossClient->getObject($bucket, $key, $options);
    var_dump($result);
} catch (OssException $e) {
    printf($e->getMessage() . "\n");
    return;
}

Get labels by using the default threshold

Threshold setting

The thr parameter is not specified, so the default threshold of 0.7 applies.

Processing example

GET /example.jpg?x-oss-process=image/labels HTTP/1.1
Host: image-demo.oss-cn-hangzhou.aliyuncs.com
Date: Fri, 21 Jul 2023 08:30:25 GMT
Authorization: SignatureValue

Sample response

HTTP/1.1 200 OK
Server: AliyunOSS
Date: Fri, 21 Jul 2023 08:30:26 GMT
Content-Type: application/json;charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
x-oss-request-id: 64BA42225DFDD13437ECD00E
Last-Modified: Mon, 10 Jul 2023 13:07:30 GMT
x-oss-object-type: Normal
x-oss-hash-crc64ecma: 13420962247653419692
x-oss-storage-class: Standard
x-oss-ec: 0048-00000104
Content-Disposition: attachment
x-oss-force-download: true
x-oss-server-time: 489
Content-Encoding: gzip

{
    "Labels": [
        {
            "CentricScore": 0.823,
            "LabelConfidence": 1.0,
            "LabelLevel": 2,
            "LabelName": "Outerwear",
            "Language": "zh-Hans",
            "ParentLabelName": "Clothing"
        },
        {
            "CentricScore": 0.721,
            "LabelConfidence": 0.735,
            "LabelLevel": 2,
            "LabelName": "Apparel",
            "Language": "zh-Hans",
            "ParentLabelName": "Clothing"
        }
        ...
    ],
    "RequestId": "0EC0B6EC-EB16-5EF4-812B-EF3A60C7D20D"
}

Get labels by using a specified threshold

Threshold setting

The thr parameter is set to 0.85.

Processing example

GET /example.jpg?x-oss-process=image/labels,thr_0.85 HTTP/1.1
Host: image-demo.oss-cn-hangzhou.aliyuncs.com
Date: Fri, 21 Jul 2023 08:44:58 GMT
Authorization: SignatureValue

Sample response

HTTP/1.1 200 OK
Server: AliyunOSS
Date: Fri, 21 Jul 2023 08:45:00 GMT
Content-Type: application/json;charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
x-oss-request-id: 64BA458C7FFDC2383651DF09
Last-Modified: Mon, 10 Jul 2023 13:07:30 GMT
x-oss-object-type: Normal
x-oss-hash-crc64ecma: 13420962247653419692
x-oss-storage-class: Standard
x-oss-ec: 0048-00000104
Content-Disposition: attachment
x-oss-force-download: true
x-oss-server-time: 421
Content-Encoding: gzip

{
    "RequestId": "B7BDAFD5-C0AF-5042-A749-88BF6E4F2712",
    "Labels": [
        {
            "CentricScore": 0.797,
            "Language": "zh-Hans",
            "LabelConfidence": 0.927,
            "LabelName": "Apparel",
            "LabelLevel": 2,
            "ParentLabelName": "Clothing"
        }
        ...
    ]
}

Parameters

Action: image/labels

Request parameters

Parameter	Type	Required	Description	Example
thr	float	No	Labels whose `LabelConfidence` value is lower than the specified threshold are not displayed. Valid values: [0, 1]. Default value: 0.7.	0.5

Raise thr to return only labels with high confidence. Lower thr to return more labels.

Response parameters

For more information about the response parameters, see DetectImageLabels - Detect labels in an image.

Billing

Image label detection calls IMM. Therefore, it generates billable items for both OSS and IMM:

OSS: For pricing information, see Object Storage Service pricing.

API	Billable item	Description
GetObject	GET requests	You are charged based on the number of successful requests.
GetObject	Outbound traffic over the internet	If you call the GetObject operation by using a public endpoint (for example, the China (Hangzhou) endpoint oss-cn-hangzhou.aliyuncs.com) or an acceleration endpoint (for example, oss-accelerate.aliyuncs.com), you are charged for outbound traffic over the internet based on the data volume.
GetObject	Volume of retrieved Infrequent Access (IA) data	If the retrieved data is Infrequent Access (IA) data, you are charged for data retrieval based on the volume of retrieved data.
GetObject	Volume of data retrieved by using real-time access of Archive objects	If you read an Archive object from a bucket for which real-time access of Archive objects is enabled, you are charged for data retrieval by using real-time access of Archive objects based on the volume of retrieved data.
GetObject	Transfer acceleration	If transfer acceleration is enabled and you use an acceleration endpoint to access your bucket, you are charged for transfer acceleration based on the data volume.
HeadObject	GET requests	You are charged based on the number of successful requests.

IMM: For pricing information, see IMM billable items.

Important

Starting at 11:00 (UTC+8) on July 28, 2025, the price of the IMM image label detection service remains unchanged, but the billable item is renamed from ImageClassification to ImageLabel. For more information, see Announcement on IMM billing adjustment.

API	Billable item	Description
DetectImageLabels	ImageLabel	You are charged based on the number of successful requests.

Use cases

Use cases

Use cases

Use cases

Precautions

How to use

Prerequisites

Detect image labels

Python

Java

Go

PHP

Get labels by using the default threshold

Threshold setting

Processing example

Sample response

Get labels by using a specified threshold

Threshold setting

Processing example

Sample response

Parameters

Request parameters

Response parameters

Related API operations

Billing