Standardize tagging for DLC, DSW, and EAS

更新时间:
复制 MD 格式

Tag rules let workspace administrators enforce consistent tagging across DLC, DSW, and EAS. Administrators define required tag keys, predefined values, descriptions, and the applicable product scope. When a rule is active, ML developers must tag DLC jobs, DSW instances, and EAS services at creation time — tags are validated automatically. Consistent tags support cost allocation, job distribution analysis, and resource consumption reporting.

Define tag rules

Note

Only workspace owners and administrators can edit tag rules.

  1. Go to the workspace details page, and in the upper-right corner, click Configure Workspace > General Configurations.

  2. Scroll down to the Label Configuration section, and click Modify Configuration to enter edit mode.

    image.png

  3. Add tag rules as needed, then click Save. Configured rules take effect for the selected product scope. On the corresponding sub-product pages, these tags are required by default. If a tag is missing or non-compliant, the submission fails. ML developers can also add custom tags on top of the required ones, as long as the tag key and tag value requirements are met.

    • Tag Key requirements: Can't start with aliyun or acs:, can't contain http:// or https://, and must be no longer than 128 characters. Only letters, digits, hyphens (-), Chinese characters, and periods (.) are allowed; underscores (_) are not.

    • Value requirements: Can't contain http:// or https://, and must be no longer than 256 characters. Separate multiple enumerated values with commas.

    • Value Constraints: When enabled, users must select from the predefined values and can't enter custom values. Enable this option to enforce consistent tagging across the team.

    • Applicable Services: Select which sub-products (DLC, DSW, or EAS) this rule applies to. At least one must be selected.

Apply tags

Apply tags according to the workspace tag rules when submitting a DLC job or creating a DSW or EAS instance. The following steps use DLC job submission as an example:

  1. Go to the DLC console and click Create Task.

  2. On the job configuration page, locate the Tag section, and fill in the key-value pairs according to the workspace tag rules.

    You can also add custom tags during job creation, such as the self-define-key tag in the screenshot.

    image.png

  3. After filling in all required tags, submit the job.

Important

Tag rules are validated at submission. If a required tag is missing or the value doesn't match the rule, the submission fails with an error message.

Filter jobs by tag

Use tag-based filtering on the DLC job list to break down job distribution and resource usage by scenario, model type, or other criteria.

  1. Go to the DLC console job list page.

  2. In the search bar, select Tag as the filter condition, enter the tag key and value, and click search. The list shows only jobs that match the criteria.

    image

    The list updates to show only jobs matching the specified tags:

    image

Appendix

API reference

To configure tag rules in bulk or integrate with automation pipelines, use the following code examples. Replace the region, workspace ID, and AccessKey ID and Secret values before running the code.

Configure tag rules in bulk

The UpdateConfigs API supports bulk configuration of preset tag rules and is suited for applying the same tag configuration across multiple workspaces.

Sample request body structure:

{
  "Configs": [
    {
      "ConfigKey": "predefinedTags",
      "ConfigValue": "[{\"Type\":\"Tag\",\"Key\":\"scenario\",\"Value\":\"{\\\"Products\\\":\\\"DLC,DSW,EAS\\\",\\\"Values\\\":\\\"train,evluate,inference\\\",\\\"ValueFlexible\\\":false,\\\"Description\\\":\\\"select scenario\\\"}\"}]",
      "Labels": [
        {
          "Key": "system.categoryName",
          "Value": "CommonTagConfig"
        }
      ]
    }
  ]
}
            

Python sample code: configure tag rules in bulk

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
PAI Workspace - Batch create/update predefined tag rules (CommonTagConfig)
API: PUT /api/v1/workspaces/{WorkspaceId}/configs
Before use, fill in ACCESS_KEY_ID, ACCESS_KEY_SECRET, REGION_ID, and WORKSPACE_ID,
and define tag rules in the TAGS list.
"""

import json, sys, hashlib, hmac, datetime, requests

ACCESS_KEY_ID     = "your-AccessKeyId"
ACCESS_KEY_SECRET = "your-AccessKeySecret"
REGION_ID         = "cn-hangzhou"
WORKSPACE_ID      = "your-workspace-ID"

ENDPOINT = f"aiworkspace.{REGION_ID}.aliyuncs.com"
API_PATH = f"/api/v1/workspaces/{WORKSPACE_ID}/configs"

# Tag definition: key-tag name, products-applicable products, values-allowed values, flexible-whether custom values are allowed, desc-description
TAGS = [
    {"key": "Scene",      "products": "DLC,DSW,EAS", "values": "Training,Evaluation,Inference",         "flexible": False, "desc": "Select the current use scenario"},
    {"key": "ModelType",  "products": "DLC,DSW",     "values": "NLP-Text,CV-Image,Multimodal",          "flexible": False, "desc": "Select the model type"},
    {"key": "ExpDirection","products": "DLC",         "values": "LR-Schedule,DataAugmentation,Distillation", "flexible": True, "desc": "Customize the experiment direction"},
]

def build_config_value(tags):
    arr = []
    for t in tags:
        inner = {"Products": t["products"], "Values": t["values"],
                 "ValueFlexible": t.get("flexible", False), "Description": t.get("desc", "")}
        arr.append({"Type": "Tag", "Key": t["key"], "Value": json.dumps(inner, ensure_ascii=False)})
    return json.dumps(arr, ensure_ascii=False)

def sign_request(method, path, headers, body):
    now = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
    headers.update({"x-acs-date": now, "x-acs-version": "2021-02-04",
                    "x-acs-action": "UpdateConfigs",
                    "x-acs-signature-nonce": hashlib.md5((now + body).encode()).hexdigest()})
    signed_keys = sorted(k for k in headers if k.startswith("x-acs-") or k in ("host", "content-type"))
    canonical_headers = "".join(f"{k}:{headers[k]}\n" for k in signed_keys)
    hashed_body = hashlib.sha256(body.encode("utf-8")).hexdigest()
    canonical_request = f"{method}\n{path}\n\n{canonical_headers}\n{';'.join(signed_keys)}\n{hashed_body}"
    string_to_sign = f"ACS3-HMAC-SHA256\n{hashlib.sha256(canonical_request.encode()).hexdigest()}"
    signature = hmac.new(ACCESS_KEY_SECRET.encode(), string_to_sign.encode(), hashlib.sha256).hexdigest()
    headers["Authorization"] = (f"ACS3-HMAC-SHA256 Credential={ACCESS_KEY_ID},"
                                 f"SignedHeaders={';'.join(signed_keys)},Signature={signature}")
    return headers

def main():
    config_value = build_config_value(TAGS)
    request_body = json.dumps({"Configs": [{"ConfigKey": "predefinedTags", "ConfigValue": config_value,
                                            "Labels": [{"Key": "system.categoryName", "Value": "CommonTagConfig"}]}]},
                              ensure_ascii=False)
    url = f"https://{ENDPOINT}{API_PATH}"
    headers = sign_request("PUT", API_PATH, {"host": ENDPOINT, "content-type": "application/json"}, request_body)
    resp = requests.put(url, headers=headers, data=request_body.encode("utf-8"), timeout=30)
    print(f"HTTP {resp.status_code}: {resp.text}")
    if resp.status_code != 200:
        sys.exit(1)

if __name__ == "__main__":
    main()

Specify tags when submitting a DLC job

When submitting a DLC job via the CreateJob API, specify tags in the Settings.Tags field. All required tags must be filled in. Values must be selected from the predefined options, except for tags where ValueFlexible is true, which accept custom input.

Python sample code: specify tags when submitting a DLC job

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
PAI-DLC Submit training job (with predefined tags)
API: POST /api/v1/jobs (API version 2020-12-03)
Tags are placed in Settings.Tags as a dictionary {"tag-name": "tag-value"}.
"""

import json, sys, hashlib, hmac, datetime, time, requests

ACCESS_KEY_ID     = "your-AccessKeyId"
ACCESS_KEY_SECRET = "your-AccessKeySecret"
REGION_ID         = "cn-hangzhou"
WORKSPACE_ID      = "your-workspace-ID"
RESOURCE_ID       = ""  # Resource quota ID; leave empty to use public resources

DLC_ENDPOINT = f"pai-dlc.{REGION_ID}.aliyuncs.com"
DLC_API_PATH = "/api/v1/jobs"

# Fill in according to the workspace predefined tags; all predefined tags must be provided
TAGS = {
    "Scene":        "Training",
    "ModelType":    "NLP-Text",
    "ExpDirection": "Fine-grained-DataProcessing",
}

def sign_request(method, path, headers, body, action="CreateJob"):
    now = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
    headers.update({"x-acs-date": now, "x-acs-version": "2020-12-03", "x-acs-action": action,
                    "x-acs-signature-nonce": hashlib.md5((now + body).encode()).hexdigest()})
    signed_keys = sorted(k for k in headers if k.startswith("x-acs-") or k in ("host", "content-type"))
    canonical_headers = "".join(f"{k}:{headers[k]}\n" for k in signed_keys)
    hashed_body = hashlib.sha256(body.encode("utf-8")).hexdigest()
    canonical_request = f"{method}\n{path}\n\n{canonical_headers}\n{';'.join(signed_keys)}\n{hashed_body}"
    string_to_sign = f"ACS3-HMAC-SHA256\n{hashlib.sha256(canonical_request.encode()).hexdigest()}"
    signature = hmac.new(ACCESS_KEY_SECRET.encode(), string_to_sign.encode(), hashlib.sha256).hexdigest()
    headers["Authorization"] = (f"ACS3-HMAC-SHA256 Credential={ACCESS_KEY_ID},"
                                 f"SignedHeaders={';'.join(signed_keys)},Signature={signature}")
    return headers

def main():
    job = {
        "DisplayName": f"my-training-job-{int(time.time())}",
        "JobType": "PyTorchJob",
        "WorkspaceId": WORKSPACE_ID,
        "UserCommand": "echo hello && sleep 5",
        "JobSpecs": [{"Type": "Worker",
                      "Image": "registry.cn-hangzhou.aliyuncs.com/pai-dlc/pytorch-training:1.12-gpu-py39-cu116-ubuntu20.04",
                      "PodCount": 1, "EcsSpec": "ecs.g6.large"}],
        "Settings": {"Tags": TAGS},  # Tags are placed in Settings.Tags
    }
    if RESOURCE_ID:
        job["ResourceId"] = RESOURCE_ID
    request_body = json.dumps(job, ensure_ascii=False)
    url = f"https://{DLC_ENDPOINT}{DLC_API_PATH}"
    headers = sign_request("POST", DLC_API_PATH, {"host": DLC_ENDPOINT, "content-type": "application/json"}, request_body)
    resp = requests.post(url, headers=headers, data=request_body.encode("utf-8"), timeout=30)
    print(f"HTTP {resp.status_code}: {resp.text}")
    if resp.status_code != 200:
        sys.exit(1)

if __name__ == "__main__":
    main()

Filter DLC jobs by tag

The ListJobs API supports tag-based filtering via the Tags query parameter (JSON string format). Multiple tags use AND logic — all specified tags must match for a job to appear in the results.

Python sample code: filter DLC jobs by tag

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import json
import sys
import hashlib
import hmac
import datetime
import requests
from urllib.parse import urlencode

ACCESS_KEY_ID     = "your-AccessKeyId"
ACCESS_KEY_SECRET = "your-AccessKeySecret"
REGION_ID         = "cn-hangzhou"
WORKSPACE_ID      = "your-workspace-ID"

ENDPOINT = f"pai-dlc.{REGION_ID}.aliyuncs.com"
API_PATH = "/api/v1/jobs"

def build_query_params(workspace_id, filter_tags, page_size=50, page_number=1):
    query_params = {
        "WorkspaceId": workspace_id,
        "PageSize": str(page_size),
        "PageNumber": str(page_number),
        "SortBy": "GmtCreateTime",
        "Order": "desc",
    }

    if filter_tags:
        query_params["Tags"] = json.dumps(
            filter_tags, ensure_ascii=False, separators=(",", ":")
        )

    return query_params

def sign_request(method, path, query_string, headers, body=""):
    now = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
    headers.update({
        "x-acs-date": now,
        "x-acs-version": "2020-12-03",
        "x-acs-action": "ListJobs",
        "x-acs-signature-nonce": hashlib.md5((now + query_string + body).encode()).hexdigest(),
    })

    normalized_headers = {k.lower(): v for k, v in headers.items()}
    signed_keys = sorted(
        k for k in normalized_headers
        if k.startswith("x-acs-") or k in ("host", "content-type")
    )

    canonical_headers = "".join(f"{k}:{normalized_headers[k]}\n" for k in signed_keys)
    hashed_body = hashlib.sha256(body.encode("utf-8")).hexdigest()

    canonical_request = (
        f"{method}\n"
        f"{path}\n"
        f"{query_string}\n"
        f"{canonical_headers}\n"
        f"{';'.join(signed_keys)}\n"
        f"{hashed_body}"
    )

    string_to_sign = f"ACS3-HMAC-SHA256\n{hashlib.sha256(canonical_request.encode()).hexdigest()}"
    signature = hmac.new(
        ACCESS_KEY_SECRET.encode(),
        string_to_sign.encode(),
        hashlib.sha256
    ).hexdigest()

    headers["Authorization"] = (
        f"ACS3-HMAC-SHA256 Credential={ACCESS_KEY_ID},"
        f"SignedHeaders={';'.join(signed_keys)},"
        f"Signature={signature}"
    )
    return headers

def list_jobs_by_tags(filter_tags, page_size=50, page_number=1):
    query_params = build_query_params(WORKSPACE_ID, filter_tags, page_size, page_number)
    query_string = urlencode(sorted(query_params.items()))
    url = f"https://{ENDPOINT}{API_PATH}?{query_string}"

    headers = {
        "host": ENDPOINT,
        "content-type": "application/json",
    }
    headers = sign_request("GET", API_PATH, query_string, headers, body="")

    resp = requests.get(url, headers=headers, timeout=30)
    print(f"HTTP {resp.status_code}")
    print(resp.text)

    if resp.status_code != 200:
        sys.exit(1)

    return resp.json()

def main():
    filter_tags = {"Scene": "Training"}
    list_jobs_by_tags(filter_tags)

if __name__ == "__main__":
    main()

View tag change history

ActionTrail records all changes to tag rules, including the operator, timestamp, and what was changed.

Log in to the ActionTrail console, go to Advanced query, set the service name to PAI and the event name to UpdateConfigs, then filter for records where ConfigKey is predefinedTags to view the tag rule change history.

image