Automatic Document Tagging Using Large Language Models

Updated at:

Automatically extract and write back document tags using the Knowledge Q&A service, enabling an end-to-end workflow—from document ingestion to tag-based retrieval.

NoteThis practice leverages the retrieval and generation capabilities of the Knowledge Q&A service. A large language model (LLM) automatically extracts tags from all document chunks and writes them back to the document—establishing an end-to-end pipeline from ingestion → automatic tagging → tag-based retrieval. The entire process is completed in the console and is fully reproducible.

Solution Overview

Background

Manual document tagging suffers from three key issues:

  • Inability to keep pace with ingestion speed: During bulk document ingestion, manual tagging lags far behind ingestion throughput.
  • Latency between ingestion and usability: Documents remain unavailable for tag-based search and distribution until manually tagged.
  • Inconsistent tag standards: Human annotators interpret content differently, leading to inconsistent tags across similar documents—and degrading retrieval quality.

Automated tag extraction by LLMs—guided by document content—enables tagging to occur concurrently with ingestion, while consistent prompting ensures uniform tag semantics.

Applicable Scenarios

ScenarioDescriptionValue of Automatic Tagging
Enterprise Document ManagementTag contracts, project documentation, and technical manuals with attributes such as department, project, topic, and classification levelImproves search efficiency and supports permission control and compliance traceability
Knowledge Asset ConsolidationTag technical documents, research reports, and industry whitepapers with domain, tech stack, and use casesStructures scattered materials, enabling precise tag-based Q&A and recommendation
Marketing Asset ManagementTag product manuals, marketing plans, and customer case studies with product line, industry, and scenarioEnables rapid asset discovery and supports tag-based filtering and reuse
Audio and Video Content ManagementTag training videos, meeting recordings, courses, and promotional videos with topic, type, speaker, and key contentClassifies media without watching it end to end, supporting tag-based retrieval and reuse

Technical Principle

The Knowledge Q&A service is fundamentally a RAG (Retrieval-Augmented Generation) system: it retrieves relevant chunks first, then uses an LLM to generate answers based on those chunks. By shifting the generation objective from answering questions to extracting tags, we achieve automatic tagging: assign a unique temporary tag to the target document, filter retrieval exclusively to that document’s chunks via tag filtering, prompt the LLM to output structured tags, and finally write them back.

Three critical implementation points:

  • Single-document retrieval: Bailian’s tag filtering operates on the tags field—not document IDs. Assigning a unique tag (e.g., auto_tag_temp) to the target document ensures only its chunks are retrieved.
  • Maximize coverage: Disable the ranking model and set recall count to maximum so as many chunks as possible enter the LLM context.
  • Structured output: Use prompt engineering to constrain LLM output to parseable, directly writable tag arrays.

WarningThe maximum number of chunks retrievable per Knowledge Q&A request is capped at 20. If a document has ≤20 chunks, full coverage is achieved in one call. For documents exceeding 20 chunks, either split the document or perform batched tagging. Sample documents used in this demo typically contain fewer than 20 chunks.

Step-by-Step Implementation

This practice demonstrates using the Bailian Technical Documentation Sample (92 technical documents about the Bailian platform), taking "First Call to Tongyi Qwen API" as the example document. You may substitute your own documents.

The following tags will be written back automatically—no pre-creation required:

TagDescriptionExample Values
productProduct the document belongs toproduct_Bailian
topicCore subject of the documenttopic_API_call
difficultyEntry-level / Intermediate / Expertdifficulty_Beginner
keywordKey terms (multi-value)keyword_Environment_Variable, keyword_curl

WarningBailian tags are simple string arrays—no schema or data types. Supported characters: Chinese, English letters (case-sensitive), digits, underscores (_), and hyphens (-). Colons (:), spaces, and other special characters are not allowed. Each tag must be ≤64 characters; each document supports up to 100 tags. Therefore, semantic tags follow the field_name_value format (e.g., product_Bailian). Multi-value tags (e.g., keywords) are represented as separate strings.

1. Create a Knowledge Base and Import Documents

Navigate to Data Ingestion → Knowledge Management, click Create, and select Premium Edition (required for higher recall limits). Follow the three-step wizard:

  1. Basic Info: Enter knowledge base name and description. Select Document Search as knowledge base type and Basic Document Q&A as use case.
  2. Select Data: Configure step-by-step: choose dataset (default: File Dataset), select Upload Files as data source, configure categories, and upload the sample ZIP file in the File Upload section.
  3. Index Settings: Keep defaults and complete creation.

After creation, go to the knowledge base’s Document List, and wait until document status changes to Parsing Complete (system auto-performs parsing, chunking, and vectorization).

NoteThe Add Tag option at the bottom of the Select Data step applies tags batch-wise to all uploaded files—and cannot isolate individual documents. Unique per-document tags (e.g., auto_tag_temp) are added separately in the next step. Tags are free-text; no prior business tag definition is needed.

See Create a Knowledge Base and Document Management for details.

2. Assign a Unique Temporary Tag to the Target Document

Since tag filtering is our mechanism for single-document retrieval, first assign a unique temporary tag (e.g., auto_tag_temp) to the document you wish to tag:

  • Console: Go to Data Ingestion → Knowledge Management, click View Details for the knowledge base created above, locate the target document in the Document List, click More → Tags, enter auto_tag_temp in the tag management modal, press Enter, then confirm.
  • API: Call batchUpdateFileTag to write ["auto_tag_temp"] into the document’s tags array.

3. Configure the Knowledge Q&A Service

  1. Navigate to Knowledge Services → Knowledge Q&A, click Create, and bind the knowledge base from Step 1.
  2. Select a generation model. Tag extraction demands strong comprehension and instruction-following capability—use a high-performance model (e.g., qwen3.7-plus).
  3. Click Expand Knowledge Base Configuration, and set retrieval parameters as follows:
ParameterSettingReason
Tag FilteringInput or select the unique tag auto_tag_tempEnsures only chunks from this document are retrieved
Initial Vector Retrieval TopKIncrease (≥ number of document chunks; default 50)Maximizes inclusion of this document’s chunks in candidate pool
Initial Keyword Retrieval TopKIncrease (≥ number of document chunks; default 50)Covers keyword-based retrieval path
Ranking ModelSelect Do Not Use ModelPrevents reranking from discarding relevant chunks
Max Recall CountSet to maximum 20Maximizes number of chunks passed to final LLM

WarningCore goal: maximize LLM’s exposure to the full document. Always set ranking model to Do Not Use Model, and max recall count to 20. Due to this hard limit, documents with >20 chunks require splitting or batched tagging.

4. Configure the Tagging Prompt

Paste the tagging prompt template into the Prompt input box on the configuration page. The service automatically concatenates retrieved chunks into context—no placeholder syntax needed. The prompt must explicitly instruct the model to output a parseable, writeable field_value tag array:

# Task
You are a document content understanding assistant. The provided reference material consists of multiple chunks from the *same document*, selected to cover its content as comprehensively as possible. Treat these chunks as sections of a single, unified document—and extract tags accordingly.

# Behavioral Requirements
1. Synthesize information across *all* chunks—do not rely on isolated segments alone.
2. All tag values must derive *exclusively* from the reference material—no external knowledge or subjective inference.
3. Omit any dimension whose value is not explicitly stated or clearly implied in the content.
4. Do not repeat text verbatim or explain reasoning—output *only* the tag array.

# Tag Dimensions
- product: Product the document belongs to (e.g., Bailian, Tongyi Qwen)
- topic: Primary subject (e.g., API_call, model_inference, data_import)
- difficulty: Beginner / Intermediate / Expert
- keyword: 1–5 key terms from the document

# Output Format
- Express each tag as `field_name_value`; multi-value dimensions (e.g., keyword) yield multiple tags.
- Only allow Chinese characters, letters, digits, underscores (`_`), and hyphens (`-`). No colons, spaces, or other symbols.
- Output *exactly one JSON string array*, with no Markdown, comments, or extraneous text. Example:
["product_Bailian", "topic_API_call", "difficulty_Beginner", "keyword_Environment_Variable", "keyword_curl"]

5. Execute Tagging via Console

In the Debug panel on the right side of the configuration page, enter a tagging instruction (e.g., Please extract tags for this document) and send. Verify that the LLM’s returned tag array matches expectations.

For batch tagging multiple documents: after confirming correctness, click Publish to obtain the service ID (agent_id). Then invoke the Knowledge Q&A API to submit tagging requests (responses are SSE-streamed; see interface docs for parsing). Finally, proceed to Step 6 to write tags back. For batch workflows, see Notes for ordering guidance.

6. Write Tags Back & Enable Tag-Based Retrieval

Once the LLM outputs the tag array, call batchUpdateFileTag to persist tags (using OVERWRITE mode replaces the temporary auto_tag_temp):

curl -X POST 'https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v1/connector/dash/batchUpdateFileTag' \
  -H 'Authorization: Bearer $API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "fileInfos": [
      {
        "fileId": "file-xxx",
        "tags": ["product_Bailian", "topic_API_call", "difficulty_Beginner", "keyword_Environment_Variable", "keyword_curl"]
      }
    ],
    "updateMode": "OVERWRITE"
  }'

Retrieve fileId via listFile, or manually write tags via More → Tags in the console’s document list. After writing, enable precise retrieval by selecting corresponding tags under Knowledge Base Configuration → Tag Filtering, in either Knowledge Search or Q&A services.

Tagging Audio and Video Files

Audio and video files can be tagged automatically through the same mechanism: assign a unique tag to the target file, isolate its chunks via tag filtering, have the LLM output tags from those chunks, and write the result back. The difference is that media requires an Audio & Video Search knowledge base, whose chunks consist of speech transcripts and visual descriptions. Knowledge base creation, retrieval parameters, and the prompt therefore need adjusting. Only the differences from the steps above are described below.

1. Create an Audio & Video Search Knowledge Base and Import Files

When creating the knowledge base, set Knowledge Base Type to Audio & Video Search. This type has no Use Case option; the chunking method in index settings supports Smart Chunking only, and the embedding model is fixed to qwen3-vl-embedding.

For this type, Data Source offers only Select Files and Select Category—both import from a dataset. The Upload Files option available to Document Search is absent, so local media must enter the dataset first:

  1. In the file list under Select Files, click Add Data and choose Local Upload as the import method (OSS, DingTalk, Lark, Yuque, and SharePoint are also supported).
  2. Keep Parsing Method as Default Settings. Media files are automatically parsed with audio and video parsing (speech transcription plus frame extraction, annotated with timestamps).
  3. In the Configure Data Tags field of the same dialog, enter the unique tag (for example, auto_tag_video) and press Enter.
  4. Confirm, wait until the file status becomes Parsed, then select the file to import it into the knowledge base.

For supported media formats and size limits, see Knowledge Base Specifications. For the scope of audio and video parsing, see Data Ingestion.

NoteStep 3 already assigns the unique tag, so Assign a Unique Temporary Tag can be skipped. For files imported without tags, add one via More → Tags in the knowledge base Document List—the same entry point used for documents.

2. Configure Retrieval Parameters

An Audio & Video Search knowledge base exposes the same retrieval parameters as Document Search in the Knowledge Q&A service. Configure them as follows:

ParameterSettingRationale
Tag FilteringEnter or select the unique tag (for example, auto_tag_video)Retrieves only this file's chunks, enabling single-file tagging
Initial Vector Retrieval TopKIncrease to at least the file's chunk count (default 50)Maximizes the chunks entering the candidate set
Initial Keyword Retrieval TopKIncrease to at least the file's chunk count (default 50)Covers the keyword retrieval path
Ranking ModelSelect Do Not Use ModelPrevents reranking from discarding chunks by relevance. The alternative for media is qwen3-vl-rerank
Similarity ThresholdLower to 0.01The default is 0.20; similarity between a tagging instruction and media chunks is generally low, so the default may discard chunks belonging to the file
Max Recall CountRaise to the ceiling of 20The default is 5 and must be raised manually

3. Adapt the Tagging Prompt

Media chunks consist of a time range, a visual summary, and narration text. The prompt must therefore state that the chunks are consecutive time segments of one file, and the tag dimensions should be replaced with media-appropriate ones:

# Task
You are an audio and video content understanding assistant. The reference material provided comes from multiple chunks of the same media file. Each chunk contains the speech transcript and visual description for one time segment, ordered chronologically. Treat these chunks as different time segments of one complete media file, understand them as a whole, and extract tags for the file.

# Requirements
1. Consider all chunks together. Do not rely on information from a single time segment.
2. Every tag value must come from the reference material itself. Do not introduce outside knowledge or speculation.
3. If a dimension is not clearly evidenced in the content, omit that dimension.
4. Do not restate the transcript or explain your analysis. Output only the tag array.

# Tag Dimensions
- topic: Subject of the media file (for example, product introduction, change management, stability governance)
- type: screen recording / course / meeting / interview / promotional video
- language: Chinese / English
- keyword: 1 to 5 content keywords

# Output Format
- Express each tag as field_name_value. For multi-value dimensions such as keyword, output multiple tags.
- Tags may contain only Chinese characters, letters, digits, underscores, and hyphens. Do not use colons, spaces, or other characters.
- Output exactly one JSON string array with no Markdown, comments, or extra text, for example:
["topic_product_introduction", "type_promotional_video", "language_Chinese", "keyword_multimodal"]

After sending the tagging instruction in the Debug panel, the model returns a tag array in the same form as document tagging, for example:

["topic_Alibaba_Cloud_AI_advances", "type_promotional_video", "language_Chinese", "keyword_Alibaba_Cloud", "keyword_LLM", "keyword_multimodal", "keyword_agent"]

Writing tags back and retrieving by tag work exactly as in Write Tags Back & Enable Tag-Based Retrieval: batchUpdateFileTag operates on fileId regardless of file type, and tag filtering is available on both Document Search and Audio & Video Search knowledge bases.

Duration Determines Single-Pass Coverage

The max recall ceiling of 20 constrains media more tightly than documents. Media is chunked by content, so the chunk count grows with both duration and the rate of visual change, and chunk density varies noticeably across content types.

File TypeDurationChunks
Narrated explainer video3 min 50 s12
Screen recording1 min 02 s9

A screen recording with frequent visual changes yields considerably more chunks per unit of time than a narrated explainer. Before tagging, check the actual chunk count under Document List → View Chunks: at 20 or fewer, one pass covers the whole file; above 20, split the file by time range before upload or tag in batches, then merge the per-segment results.

Notes

Tags Are String Arrays

Bailian tags are untyped string arrays. Allowed characters: Chinese, English letters (case-sensitive), digits, underscores (_), hyphens (-). Colons (:), spaces, and other symbols are forbidden. Max length per tag: 64 chars; max tags per document: 100. Semantic meaning is encoded via field_name_value (e.g., product_Bailian). Multi-value fields (e.g., keywords) become separate strings. Tags are free-text—no pre-registration required.

Single-Document Retrieval Relies on Unique Tags

Bailian’s retrieval and Q&A do not support document-ID filtering—only tags-based filtering. Thus, single-document tagging requires assigning a unique temporary tag (e.g., auto_tag_temp) per document. Tags added during the Select Data step apply in bulk to all uploaded files—and cannot isolate individual documents.

Constrained by Max Recall Limit

The Knowledge Q&A service caps retrievable chunks per request at 20. Documents with >20 chunks cannot be fully processed in one call—split them or use batched tagging. Chunk counts for media files grow with duration and the rate of visual change, making this ceiling more restrictive—see Duration Determines Single-Pass Coverage.

Tag Filtering Is a Static, Service-Level Configuration

Tag filtering in Q&A services is fixed upon publishing the configuration—it does not vary per request. For batch tagging multiple documents, choose one of two approaches:

ApproachHow ToUse Case
Shared Temporary TagAssign same temp tag (e.g., auto_tag_temp) to all docs; tag sequentially; overwrite with business tags afterwardSmall batches, serial execution
Dedicated ServicesCreate one Q&A service per document (or group), each configured with its own tag filterLarge-scale parallel tagging

Additional Limits on Audio & Video Knowledge Bases

Audio & Video Search knowledge bases do not support adding chunks (deletion only), nor automatic update and synchronization—see the Knowledge Base API Guide. Segmented tagging of media therefore requires splitting the file by time range before import; chunks cannot be split after ingestion.

An Alternative: Extracting Metadata at Index Time

The Index Settings step offers a Metadata Extraction switch. When enabled, you can define a metadata template of up to 5 fields. Each field's value method can be constant, variable, LLM, regular expression, or keyword search, and you can control whether the field participates in retrieval and in model responses. With LLM selected, field values are extracted when the file is ingested, removing the need for the Q&A service and write-back steps described here. That approach has a fixed field count and writes to metadata rather than tags, which suits stable structured schemas. The approach in this topic allows dimensions and prompts to change at any time and writes results directly into tags for tag filtering.

Frequently Asked Questions

Why does tagging only cover part of the document? Confirm ranking model is disabled and both TopK and max recall are maximized. For documents with >20 chunks, split or batch.

Why does the model output extra text—not just the tag array? Strengthen prompt constraints requiring only the JSON array. Before parsing, extract the first [...] substring from the response.

Why does tag write-back fail? Check for unsupported characters (colons, spaces) and ensure no tag exceeds 64 characters.