Image label detection

更新时间:
复制 MD 格式

Identifies scenes, objects, and events in images and returns structured labels. Supports thousands of labels across more than 30 categories in a hierarchical taxonomy.

Use cases

Scenario Description
Content recognition Detect items, scenes, and other information in captured or uploaded images for object recognition or educational applications.
Smart album Classify images automatically based on content to organize photo albums and galleries without manual effort.
Scene analysis Detect objects and scenes in images, then apply content labels to reduce manual annotation costs.
Content operations Retrieve image labels for content recommendation on social media, news, and e-commerce platforms.

Limits

Constraint Limit
Supported image formats PNG, JPG, JPEG
Maximum image file size 20 MB
Maximum image height or width 30,000 pixels
Maximum total pixels 250 million

Prerequisites

You need:

Note

Call CreateProject to create a project programmatically, or ListProjects to view existing projects in a region.

Detect image labels

Call DetectImageLabels to detect image labels.

Request parameters

Parameter Type Description
ProjectName String The name of the IMM project.
SourceURI String The OSS URI of the image, in the format oss://bucket-name/object-key.
Threshold Float The minimum confidence score for returned labels. Labels below this threshold are filtered out. Example: 0.7.

Request example

{
    "ProjectName": "test-project",
    "SourceURI": "oss://test-bucket/test-object.jpg",
    "Threshold": 0.7
}

Response fields

Each label in the Labels array contains:

Field Type Description
LabelName String The name of the detected label, such as Dog, Building, or Camping.
LabelConfidence Float The confidence score of the label, ranging from 0 to 1. Higher values indicate greater certainty.
CentricScore Float A score indicating how prominent or central the labeled object is within the image. Higher values mean the object is more visually prominent.
LabelLevel Integer The hierarchy depth of the label. 1 = top-level category, 2 = mid-level category, 3 = fine-grained label.
ParentLabelName String The name of the parent label in the taxonomy. Empty for top-level labels.
Language String The language of the label taxonomy, such as zh-Hans.

The response also includes a RequestId that identifies each request.

Response example

{
  "RequestId": "91C92EBA-5E51-50C8-B51B-8C3BDC66EB86",
  "Labels": [
    {
      "CentricScore": 0.797,
      "Language": "zh-Hans",
      "LabelConfidence": 1,
      "LabelName": "Clothing",
      "LabelLevel": 2,
      "ParentLabelName": "Clothes"
    },
    {
      "CentricScore": 0.695,
      "Language": "zh-Hans",
      "LabelConfidence": 1,
      "LabelName": "Carnivore",
      "LabelLevel": 2,
      "ParentLabelName": "Wild Animal"
    },
    {
      "CentricScore": 0.723,
      "Language": "zh-Hans",
      "LabelConfidence": 0.987,
      "LabelName": "Tent",
      "LabelLevel": 2,
      "ParentLabelName": "Other Scenes"
    },
    {
      "CentricScore": 0.695,
      "Language": "zh-Hans",
      "LabelConfidence": 0.949,
      "LabelName": "Pet",
      "LabelLevel": 1,
      "ParentLabelName": ""
    },
    {
      "CentricScore": 0.695,
      "Language": "zh-Hans",
      "LabelConfidence": 0.939,
      "LabelName": "Dog",
      "LabelLevel": 3,
      "ParentLabelName": "Pet Dog"
    },
    {
      "CentricScore": 0.803,
      "Language": "zh-Hans",
      "LabelConfidence": 0.924,
      "LabelName": "Person",
      "LabelLevel": 2,
      "ParentLabelName": "Face"
    },
    {
      "CentricScore": 0.689,
      "Language": "zh-Hans",
      "LabelConfidence": 0.885,
      "LabelName": "Camping",
      "LabelLevel": 2,
      "ParentLabelName": "Entertainment"
    }
  ]
}
Note

This response is abbreviated. A full response returns all labels meeting the Threshold. Top-level labels include Clothes, Wild Animal, Pet, Face, and Entertainment, with finer-grained labels like Dog, Tent, Camping, and Person nested within.

Sample code (Python)

Example using the IMM Python SDK:

# -*- coding: utf-8 -*-
import sys
import os
from typing import List

from alibabacloud_imm20200930.client import Client as imm20200930Client
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_imm20200930 import models as imm_20200930_models
from alibabacloud_tea_util import models as util_models
from alibabacloud_tea_util.client import Client as UtilClient


class Sample:
    def __init__(self):
        pass

    @staticmethod
    def create_client(
        access_key_id: str,
        access_key_secret: str,
    ) -> imm20200930Client:
        """
        Initialize the client with an AccessKey ID and AccessKey secret.
        @param access_key_id:
        @param access_key_secret:
        @return: Client
        @throws Exception
        """
        config = open_api_models.Config(
            access_key_id=access_key_id,
            access_key_secret=access_key_secret
        )
        # Specify the endpoint of IMM.
        config.endpoint = f'imm.cn-beijing.aliyuncs.com'
        return imm20200930Client(config)

    @staticmethod
    def main(
        args: List[str],
    ) -> None:
        # Obtain credentials from environment variables.
        # Do not hardcode AccessKey credentials in your code.
        imm_access_key_id = os.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
        imm_access_key_secret = os.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
        # Initialize the client.
        client = Sample.create_client(imm_access_key_id, imm_access_key_secret)
        detect_image_labels_request = imm_20200930_models.DetectImageLabelsRequest(
            project_name='test-project',
            source_uri='oss://test-bucket/test-object.jpg',
            threshold=0.7
        )
        runtime = util_models.RuntimeOptions()
        try:
            # Print the return value of the API operation.
            client.detect_image_labels_with_options(detect_image_labels_request, runtime)
        except Exception as error:
            # Print the error message if the request fails.
            UtilClient.assert_as_string(error.message)

    @staticmethod
    async def main_async(
        args: List[str],
    ) -> None:
        # Obtain credentials from environment variables.
        # Do not hardcode AccessKey credentials in your code.
        imm_access_key_id = os.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
        imm_access_key_secret = os.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
        # Initialize the client.
        client = Sample.create_client(imm_access_key_id, imm_access_key_secret)
        detect_image_labels_request = imm_20200930_models.DetectImageLabelsRequest(
            project_name='test-project',
            source_uri='oss://test-bucket/test-object.jpg',
            threshold=0.7
        )
        runtime = util_models.RuntimeOptions()
        try:
            # Print the return value of the API operation.
            await client.detect_image_labels_with_options_async(detect_image_labels_request, runtime)
        except Exception as error:
            # Print the error message if the request fails.
            UtilClient.assert_as_string(error.message)


if __name__ == '__main__':
    Sample.main(sys.argv[1:])
Important

Use a RAM user instead of your Alibaba Cloud account to call API operations. Store AccessKey credentials in environment variables (ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET) rather than hardcoding them in your source code.

Billing

Image label detection incurs charges for both OSS and IMM.

OSS billing

API Billable item Description
GetObject GET requests Billed based on the number of successful requests.
GetObject Infrequent Access (IA) data retrieval If retrieving IA data, charged based on the volume of retrieved data.
GetObject Archive real-time access data retrieval If reading an Archive object from a bucket with real-time access enabled, charged based on the size of retrieved data.
GetObject Transfer acceleration If transfer acceleration is enabled and an acceleration endpoint is used, charged based on the data size.
HeadObject GET requests Billed based on the number of successful requests.

OSS Pricing.

IMM billing

API Billable item Description
DetectImageLabels ImageLabel Billed based on the number of successful requests.
Important

Starting at 11:00 on July 28, 2025 (UTC+8), the billable item for IMM image label detection changed from ImageClassification to ImageLabel. Notice on IMM billing adjustment.

IMM billable items.

FAQ

Does image label detection support extracting text, dates, or locations from images as labels?

No. Image label detection identifies visual content (objects, scenes, events) but does not perform OCR or extract metadata.

To extract text from images, use image semantic search, then parse results for dates or organization names. To retrieve photo locations, read GPS data from the image EXIF metadata through an image information API operation.

Note

For images with violent, pornographic, or other sensitive content, automatic detection may lack accuracy. Manual review may be required.