IPC content understanding

更新时间:
复制 MD 格式

IPC content understanding uses AI algorithms to automatically identify objects, scenes, text, and icons in media content. This analysis enables subsequent media processing, such as search. This document explains how to use the API to implement IPC content understanding.

Prerequisites

  1. You have purchased the service by creating an IPC order.

  2. You have set up the environment. For details, see OpenAPI portal tool.

Basic usage

Create an IPC algorithm template

The following example shows how to call the CreateCustomTemplate operation to create a custom template:

# -*- coding: utf-8 -*-
from alibabacloud_ice20201109.client import Client as ICE20201109Client
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_ice20201109 import models as ice20201109_models
from alibabacloud_tea_util import models as util_models

# For production environments, we recommend using a more secure method for authentication that does not involve access keys (AKs). For more information about how to configure credentials, see https://help.aliyun.com/document_detail/378659.html.
credential = CredentialClient()
config = open_api_models.Config(
    credential=credential
)
# For more information about endpoints, see https://api.aliyun.com/product/ICE
config.endpoint = 'ice.cn-shanghai.aliyuncs.com'
client = ICE20201109Client(config)

create_custom_template_request = ice20201109_models.CreateCustomTemplateRequest(
    name='ipc_template_001',
    template_config='{"AnalyseTypes":"vlEventDesc","Industry":"common","Scene":"search","TemplateName":"ipc_template_001"}',
    type=11
)
runtime = util_models.RuntimeOptions()
response = client.create_custom_template_with_options(create_custom_template_request, runtime)
print(response)

Request parameters:

Parameter

Type

Description

Example

Name

string

The template name.

ipc_template_001

Type

integer

The template type. For IPC jobs, set this parameter to 11, which indicates an AI smart tag template.

11

TemplateConfig

string

The template configuration in JSON format. For IPC jobs, you must use the format shown in the example to create an IPC algorithm template.

{\"AnalyseTypes\":\"vlEventDesc\",\"TemplateName\":\"test\",\"Industry\":\"common\",\"Scene\":\"search\"}

Response parameters:

Parameter

Type

Description

Example

RequestId

string

The request ID.

******11-DB8D-4A9A-875B-275798******

CustomTemplate

object

The template information.

CustomTemplate.TemplateId

string

The template ID.

****20b48fb04483915d4f2cd8ac****

CustomTemplate.TemplateName

string

The template name.

test-template

CustomTemplate.Type

integer

The template type ID.

11

CustomTemplate.TypeName

string

The template type name.

AISmarttagTemplate

CustomTemplate.AppId

string

CustomTemplate.Version

string

The template version.

1

CustomTemplate.Status

string

The template status.

Normal

CustomTemplate.IsDefault

boolean

Indicates whether this is the default template.

true

CustomTemplate.TemplateConfig

string

The template configuration.

{\"AnalyseTypes\":\"vlEventDesc\",\"TemplateName\":\"test\",\"Industry\":\"common\",\"Scene\":\"search\"}

CustomTemplate.CreateTime

string

When the template was created.

CustomTemplate.ModifiedTime

string

When the template was last modified.

Response example:

{
  "CustomTemplate": {
    "CreateTime": "2025-12-22T03:11:02Z",
    "IsDefault": false,
    "ModifiedTime": "2025-12-22T03:11:02Z",
    "Status": "Normal",
    "TemplateConfig": "{\"AnalyseTypes\":\"vlEventDesc\"}",
    "TemplateId": "****a0da83434f0a8d57ffb0415c****",
    "TemplateName": "ipc_template_001",
    "Type": 11,
    "TypeName": "AISmarttagTemplate"
  },
  "RequestId": "******22-1193-5A70-8D4C-E0533C******"
}

Submit a smart tag job

The following example shows how to call the SubmitSmarttagJob operation to submit a smart tag job:

# -*- coding: utf-8 -*-
from alibabacloud_ice20201109.client import Client as ICE20201109Client
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_ice20201109 import models as ice20201109_models
from alibabacloud_tea_util import models as util_models
import json

# For production environments, we recommend using a more secure method for authentication that does not involve access keys (AKs). For more information about how to configure credentials, see https://help.aliyun.com/document_detail/378659.html.
credential = CredentialClient()
config = open_api_models.Config(
    credential=credential
)
# For more information about endpoints, see https://api.aliyun.com/product/ICE
config.endpoint = 'ice.cn-shanghai.aliyuncs.com'
client = ICE20201109Client(config)

submit_smarttag_job_request_input = ice20201109_models.SubmitSmarttagJobRequestInput(
    type='Multi',
    media=json.dumps(
        {
            "type": "URL",
            "inputs": [
                "https://xxx.png",
                "https://xxx.png"
            ]
        }
    )
)
submit_smarttag_job_request = ice20201109_models.SubmitSmarttagJobRequest(
    title='ipc-test-001',
    input=submit_smarttag_job_request_input,
    template_id='xxx',
    # params='{"vlEventDescParams":{"deviceId":"camera_001"}}',
)
runtime = util_models.RuntimeOptions()
response = client.submit_smarttag_job_with_options(submit_smarttag_job_request, runtime)
print(response)

Request parameters:

Parameter

Type

Description

Example

Title

string

The job title.

xxx

Input

object

The input object.

Input.Type

string

Media type: For an IPC task, you must enter Multi.

Multi

Input.Media

string

The input media. When Type is set to Multi, this parameter must be a JSON string. Use one of the following formats: {"type": "URL", "inputs": ["xxx.png", "xxx.png"]} or {"type": "URL", "inputs": ["xxx.mp4"],"extra":{"mediaType":"video", "fps":2,"customPrompt":"Custom prompt text"}}.

The type field supports only URL. The inputs field supports a list of image URLs or a single video URL. The video duration must be between 2 and 40 seconds.

Parameters in the extra field:

  • mediaType: The default value is image. For videos, you must explicitly set this parameter to video.

  • fps: The sampling rate in frames per second. This parameter is valid only for video inputs. Default: 2.

  • customPrompt: A custom prompt of up to 2,000 bytes. If this parameter is not set, the default prompt is used.

  • targetLanguage: The target language for translating the brief field. See the table below for valid values. Default: Simplified Chinese.

{"type": "URL", "inputs": ["xxx.png", "xxx.png"]}

{"type": "URL", "inputs": ["xxx.m3u8"],"extra":{"mediaType":"video","fps":2,"targetLanguage": "it"}}

TemplateId

string

The ID of the IPC algorithm template, which is returned by the CreateCustomTemplate operation.

3d5***ca7

Params

string

The extended configuration parameters in JSON format. The vlEventDescParams field is used to configure extended parameters for IPC content understanding.

{"vlEventDescParams":{"deviceId":"xxx"}}

Description of the vlEventDescParams field:

Parameter

Type

Required

Description

Example

deviceId

string

No

The unique identifier for the job's source device. It can be up to 32 characters long and is limited to letters, digits, underscores (_), and hyphens (-).

camera_001

Valid values for targetLanguage:

Parameter

Language

en

English

ja

Japanese

id

Indonesian

es

Spanish

pt

Portuguese

ar

Arabic

fr

French

tr

Turkish

de

German

ko

Korean

th

Thai

vi

Vietnamese

ru

Russian

ms

Malay

zh-tw

Traditional Chinese

km

Khmer

pl

Polish

nl

Dutch

cs

Czech

fi

Finnish

ro

Romanian

sv

Swedish

fa

Persian

it

Italian

Response parameters:

Parameter

Type

Description

Example

RequestId

string

The request ID.

******11-DB8D-4A9A-875B-275798******

JobId

string

The ID of the smart tag job. Save this ID for subsequent API calls.

****d80e4e4044975745c14b****

Response example:

{
    "RequestId": "******11-DB8D-4A9A-875B-275798******",
    "JobId": "****d80e4e4044975745c14b****"
}

Query smart tag job results

The following example shows how to call the QuerySmarttagJob operation to query job results:

# -*- coding: utf-8 -*-
from alibabacloud_ice20201109.client import Client as ICE20201109Client
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_ice20201109 import models as ice20201109_models
from alibabacloud_tea_util import models as util_models

# For production environments, we recommend using a more secure method for authentication that does not involve access keys (AKs). For more information about how to configure credentials, see https://help.aliyun.com/document_detail/378659.html.
credential = CredentialClient()
config = open_api_models.Config(
    credential=credential
)
# For more information about endpoints, see https://api.aliyun.com/product/ICE
config.endpoint = 'ice.cn-shanghai.aliyuncs.com'
client = ICE20201109Client(config)

query_smarttag_job_request = ice20201109_models.QuerySmarttagJobRequest(
    job_id='xxx'
)
runtime = util_models.RuntimeOptions()
response = client.query_smarttag_job_with_options(query_smarttag_job_request, runtime)
print(response)

Request parameters:

Parameter

Type

Description

Example

JobId

string

The ID of the smart tag job to query.

88c6ca184c0e47098a5b665e2****

Response parameters:

Parameter

Type

Description

Example

RequestId

string

The request ID.

******11-DB8D-4A9A-875B-275798******

JobStatus

string

The job status. Valid values:

  • Success: The job completed successfully.

  • Fail: The job failed.

  • Processing: The job is being processed.

  • Submitted: The job is submitted and awaiting processing.

Success

UserData

string

The passthrough information for MNS callbacks.

Results

object

The analysis result object.

Results.Result

array<object>

An array of analysis result objects.

Type

string

The type of analysis result.

Data

string

The detailed analysis data as a JSON string. See the table below for a description of its fields.

Data fields:

Parameter

Description

brief

A brief description of the video.

caption

A detailed description of the video content.

title

The title.

event

Event information.

object

Object information.

action

Action information.

label

Label information.

Response example:

{
  "JobStatus": "Success",
  "RequestId": "41B38CF1-C1EC-538E-81FB-791355065538",
  "Results": {
    "Result": [
      {
        "Data": "\"{\\\"brief\\\":\\\"An adult male is sitting on a chair eating, while two dogs in a nearby cage are moving around.\\\",\\\"caption\\\":\\\"In a brick-paved courtyard, an adult male (short hair, wearing a beige robe) is sitting on a chair and eating. Next to him is a table with food and a water bottle. Two medium-sized, grayish-white dogs are moving around in a cage.\\\",\\\"action\\\":[\\\"eating\\\",\\\"moving\\\"],\\\"label\\\":{\\\"package\\\":false,\\\"human\\\":true,\\\"pet\\\":true,\\\"vehicle\\\":false},\\\"title\\\":\\\"Man eats in courtyard, dogs move in cage\\\",\\\"event\\\":[\\\"adult male eating\\\",\\\"dog moving\\\"],\\\"object\\\":[\\\"adult male\\\",\\\"dog\\\",\\\"table\\\",\\\"chair\\\"]}\"",
        "Type": "EventDesc"
      }
    ]
  },
  "UserData": "{\"fromICE\":true,\"pk\":1346693276669150}"
}

Advanced usage

Advanced features let you identify specific pedestrians and pets.

Create a custom recognition library

Call the CreateCustomRecognitionLib operation to create a custom recognition library. The following code provides an example:

# -*- coding: utf-8 -*-
from alibabacloud_ice20201109.client import Client as ICE20201109Client
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_ice20201109 import models as ice20201109_models
from alibabacloud_tea_util import models as util_models

# For production environments, we recommend that you use a more secure method that does not involve access keys (AKs) for authentication. For more information about how to configure credentials, see https://help.aliyun.com/document_detail/378659.html.
credential = CredentialClient()
config = open_api_models.Config(
    credential=credential
)
# For more information about endpoints, see https://api.aliyun.com/product/ICE.
config.endpoint = 'ice.cn-shanghai.aliyuncs.com'
client = ICE20201109Client(config)

create_recognition_lib_request = ice20201109_models.CreateRecognitionLibRequest(
    algorithm='pedestrian',
    lib_name='ipc_peds_lib_001'
)
runtime = util_models.RuntimeOptions()
response = client.create_recognition_lib_with_options(create_recognition_lib_request, runtime)
print(response)

Request parameters:

Parameter

Type

Description

Example

Algorithm

string

The recognition algorithm. Valid values:

  • pedestrian: Custom pedestrian.

  • pet: Custom pet.

pedestrian

LibName

string

The name of the recognition library. The name can be up to 64 bytes in length.

ipc_peds_lib_001

Response parameters:

Parameter

Type

Description

Example

RequestId

string

The request ID.

******11-DB8D-4A9A-875B-275798******

LibId

string

The ID of the created recognition library.

*************24b47865c6**************

Response example:

{
    "LibId": "*************24b47865c6**************",
    "RequestId": "******11-DB8D-4A9A-875B-275798******"
}

Create a custom recognition entity

Call the CreateRecognitionEntity operation to create an entity in the library. The following code provides an example:

# -*- coding: utf-8 -*-
from alibabacloud_ice20201109.client import Client as ICE20201109Client
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_ice20201109 import models as ice20201109_models
from alibabacloud_tea_util import models as util_models

# For production environments, we recommend that you use a more secure method that does not involve access keys (AKs) for authentication. For more information about how to configure credentials, see https://help.aliyun.com/document_detail/378659.html.
credential = CredentialClient()
config = open_api_models.Config(
    credential=credential
)
# For more information about endpoints, see https://api.aliyun.com/product/ICE.
config.endpoint = 'ice.cn-shanghai.aliyuncs.com'
client = ICE20201109Client(config)

create_recognition_entity_request = ice20201109_models.CreateRecognitionEntityRequest(
    algorithm='pedestrian',
    lib_id='xxx',
    entity_name='Xiaoshuai'
)
runtime = util_models.RuntimeOptions()
response = client.create_recognition_entity_with_options(create_recognition_entity_request, runtime)
print(response)

Request parameters:

Parameter

Type

Description

Example

Algorithm

string

The recognition algorithm. Valid values:

  • pedestrian: Custom pedestrian.

  • pet: Custom pet.

pedestrian

LibId

string

The ID of the recognition library.

*************24b47865c6**************

EntityName

string

The name of the custom recognition entity. The name can be up to 64 bytes in length.

Xiaoshuai

Response parameters:

Parameter

Type

Description

Example

RequestId

string

The request ID.

******11-DB8D-4A9A-875B-275798******

EntityId

string

The ID of the created entity.

**************544cb84754************

Response example:

{
  "EntityId": "**************544cb84754************",
  "RequestId": "******11-DB8D-4A9A-875B-275798******"
}

Create a custom recognition sample

Call the CreateRecognitionSample operation to create a sample for an entity. The following code provides an example:

# -*- coding: utf-8 -*-

from alibabacloud_ice20201109.client import Client as ICE20201109Client
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_ice20201109 import models as ice20201109_models
from alibabacloud_tea_util import models as util_models

# For production environments, we recommend that you use a more secure method that does not involve access keys (AKs) for authentication. For more information about how to configure credentials, see https://help.aliyun.com/document_detail/378659.html.
credential = CredentialClient()
config = open_api_models.Config(
    credential=credential
)
# For more information about endpoints, see https://api.aliyun.com/product/ICE.
config.endpoint = 'ice.cn-shanghai.aliyuncs.com'
client = ICE20201109Client(config)

create_recognition_sample_request = ice20201109_models.CreateRecognitionSampleRequest(
    algorithm='pedestrian',
    lib_id='xxx',
    entity_id='xxx',
    image_url='https://xxx.png'
)
runtime = util_models.RuntimeOptions()
response = client.create_recognition_sample_with_options(create_recognition_sample_request, runtime)
print(response)

Request parameters:

Parameter

Type

Description

Example

Algorithm

string

The recognition algorithm. Valid values:

  • pedestrian: Custom pedestrian.

  • pet: Custom pet.

pedestrian

LibId

string

The ID of the recognition library.

*************24b47865c6**************

EntityId

string

The entity ID.

**************544cb84754************

ImageUrl

string

The URL of the sample image.

https://xxx.png

Response parameters:

Parameter

Type

Description

Example

RequestId

string

The request ID.

******11-DB8D-4A9A-875B-275798******

SampleId

string

The ID of the created sample.

**************4d2ba728e2f**************

Response example:

{
    "RequestId": "******11-DB8D-4A9A-875B-275798******",
    "SampleId": "**************4d2ba728e2f**************"
}

Create an IPC algorithm template

Call the CreateCustomTemplate operation to create a template. The process is the same as in the basic usage scenario, but you must also configure a pedestrian library or pet library. The following code provides an example:

# -*- coding: utf-8 -*-
from alibabacloud_ice20201109.client import Client as ICE20201109Client
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_ice20201109 import models as ice20201109_models
from alibabacloud_tea_util import models as util_models

# For production environments, we recommend that you use a more secure method that does not involve access keys (AKs) for authentication. For more information about how to configure credentials, see https://help.aliyun.com/document_detail/378659.html.
credential = CredentialClient()
config = open_api_models.Config(
    credential=credential
)
# For more information about endpoints, see https://api.aliyun.com/product/ICE.
config.endpoint = 'ice.cn-shanghai.aliyuncs.com'
client = ICE20201109Client(config)

create_custom_template_request = ice20201109_models.CreateCustomTemplateRequest(
    name='ipc_template_001_with_group',
    template_config='{"AnalyseTypes":"vlEventDesc","Industry":"common","Scene":"search","TemplateName":"ipc_template_001_with_group","PedestrianGroupIds":"xxx"}',
    type=11
)
runtime = util_models.RuntimeOptions()
response = client.create_custom_template_with_options(create_custom_template_request, runtime)
print(response)

Request parameters:

Parameter

Type

Description

Example

Name

string

The template name.

xxx

Type

integer

The template type. For IPC jobs, you must set this parameter to 11, which specifies an intelligent tag template.

11

TemplateConfig

string

The template configuration in JSON format. For IPC jobs, you must specify the ID of a pedestrian or pet library. The value must follow the format in the example.

  • AnalyseTypes: The analysis type. Set this to vlEventDesc for event analysis.

  • TemplateName: The custom template name.

  • Industry: The industry. The default value is common.

  • Scene: The scene. The default value is search.

  • PedestrianGroupIds: The ID of the pedestrian library.

  • PetGroupIds: The ID of the pet library.

{"AnalyseTypes":"vlEventDesc","Industry":"common","Scene":"search","TemplateName":"ipc_template_001_with_group","PedestrianGroupIds":"xxx","PetGroupIds":"xxx"}

Submit an IPC intelligent tag job

Call the SubmitIntelligentTagJob operation to submit a job. The process is the same as in the basic usage scenario.

Query IPC job results

Call the QueryIntelligentTagJob operation to query the job results. The process is the same as in the basic usage scenario. As the following example shows, the output replaces the generic term pedestrian with the custom name Xiaoshuai:

{
  "JobStatus": "Success",
  "RequestId": "******F9-82A8-5BC4-B63E-63D756******",
  "Results": {
    "Result": [
      {
        "Data": "\"{\\\"brief\\\":\\\"Xiaoshuai is sitting on a chair eating, with two dogs in a nearby cage.\\\",\\\"caption\\\":\\\"In a brick-paved courtyard, an adult male (Xiaoshuai) is sitting on a chair and eating. Next to him is a table with food and a water bottle. Two medium-sized, grayish-white dogs are in a cage.\\\",\\\"action\\\":[\\\"eating\\\",\\\"other events\\\"],\\\"label\\\":{\\\"package\\\":false,\\\"human\\\":true,\\\"pet\\\":true,\\\"vehicle\\\":false},\\\"title\\\":\\\"Xiaoshuai eats in the courtyard while dogs are in the cage\\\",\\\"event\\\":[\\\"adult male eating\\\",\\\"dog other events\\\"],\\\"object\\\":[\\\"adult male\\\",\\\"dog\\\",\\\"table\\\",\\\"chair\\\"]}\"",
        "Type": "EventDesc"
      }
    ]
  },
  "UserData": "{\"fromICE\":true,\"pk\":1346693276669150}"
}