AI Search Open Platform splits documents into smaller chunks through the SDK. Use this to prepare text content for downstream embedding and retrieval in Retrieval-Augmented Generation (RAG) workflows — each chunk becomes an independently searchable unit fed into your vector store or search index.
The total size of retrieved chunks must fit within the context window of the large language model (LLM) you are using. A chunk size of around 300 characters is a reasonable starting point; adjust based on your embedding model's token limit and retrieval precision requirements.
Prerequisites
Before you begin, ensure that you have:
Activated AI Search Open Platform. See Activate AI Search Open Platform.
An API key for authentication. See Manage API key.
How it works
Submit a document to the chunking service with a content type and a chunking strategy. The service splits the content and returns a list of chunks. The workspace and service ID in the request identify the deployment you are targeting.
Chunk a document
The following example splits a plain text document using a maximum chunk size of 300 characters.
from alibabacloud_tea_openapi.models import Config
from alibabacloud_searchplat20240529.client import Client
from alibabacloud_searchplat20240529.models import GetDocumentSplitRequest
if __name__ == '__main__':
config = Config(
bearer_token='<your-api-key>',
# Remove the http:// or https:// prefix from the endpoint URL.
endpoint='<your-api-endpoint>',
protocol='http')
client = Client(config=config)
request = GetDocumentSplitRequest().from_map({
"document": {
"content": "Your document text goes here.",
# Supported value: text
"content_type": "text"
},
"strategy": {
# Maximum number of characters per chunk
"max_chunk_size": 300,
# Set to True to keep sentence boundaries intact
"need_sentence": False
}
})
# Replace "default" with your workspace name and "ops-document-split-001" with your service ID.
response = client.get_document_split("default", "ops-document-split-001", request)
print(response)Replace the following placeholders before running the code:
| Placeholder | Description | Example |
|---|---|---|
<your-api-key> | Your API key from the AI Search Open Platform console | ey... |
<your-api-endpoint> | The API endpoint, without the http:// or https:// prefix | searchplat.cn-hangzhou.aliyuncs.com |
default | Your workspace name | my-workspace |
ops-document-split-001 | Your service ID | ops-document-split-001 |
Request parameters
| Field | Type | Description |
|---|---|---|
document.content | String | The text to chunk. |
document.content_type | String | Content format. Use text for plain text. |
strategy.max_chunk_size | Integer | Maximum character count per chunk. Start around 300 and increase if chunks are too small for your embedding model, or decrease if retrieval precision is low. |
strategy.need_sentence | Boolean | When true, the service avoids splitting mid-sentence, preserving sentence boundaries at the cost of slightly uneven chunk sizes. Set to false for strict size control. |
For the full parameter reference, see Document chunking API.
Limitations
The request body cannot exceed 8 MB. If your document exceeds this limit, split it into smaller segments before calling the API.
What's next
Document chunking API — Full parameter reference for the chunking API.
Manage API key — Create and rotate API keys.