Architecture and technology selection

System architecture

Knowledge Storage Service uses a fully managed serverless architecture. You interact through APIs while the platform manages all underlying resources. The data flow:

1. Call the AddDocuments API and specify an OSS file path, or upload a local file through the SDK.

2. The system reads the file from OSS, then parses and chunks the document.

3. An embedding model generates vectors for each chunk and writes them to the Chunk table in Tablestore.

4. Vector indexes and full-text indexes are built automatically.

5. When you call the Retrieve API, the system runs a hybrid search and returns the matching chunks.

Applicability

Recommended scenarios:

• RAG retrieval for LLM applications

• Hybrid vector + full-text search

• Multi-tenant data isolation

• Document volumes from hundreds to hundreds of millions

• Strict data security: all data stays in your own account

• Pay-as-you-go with zero operations overhead

Scenarios that require evaluation:

• Near-real-time indexing. Ingestion is asynchronous, so there is latency between upload and searchability.

• Custom chunking. The system handles chunking automatically.

Not recommended:

• Pure structured data queries. Use Tablestore wide column tables or relational databases instead.

• Real-time streaming data indexing. Use Tablestore search indexes directly instead.

Data model

A knowledge base organizes data into four entity types:

The system creates these data tables in Tablestore:

• Document table: {knowledge_base_name}_{knowledge_base_id}

• Chunk table: {knowledge_base_name}_{knowledge_base_id}_chunk

• Index table: {knowledge_base_name}_{knowledge_base_id}_index

Data sources

Knowledge Storage Service uses Alibaba Cloud Object Storage Service (OSS) as its data source. Regardless of how you upload documents, all raw data is stored in your own OSS bucket.

The knowledge base reads documents from OSS for parsing and chunking, and writes processing results back to OSS.

Import documents in three ways:

• Upload local files: Specify a local file or directory. The SDK uploads files to OSS and adds them to the knowledge base. Directories are traversed recursively.

• Add OSS files: Specify the OSS path to add existing files without re-uploading.

• Batch import from an OSS directory: Specify an OSS directory path for the system to recursively scan all files. Use inclusionFilters and exclusionFilters to filter files by name patterns.

Each import method and its detailed usage are described in Document management.

Embedding configuration

The embedding configuration determines how chunks are vectorized. Set it when creating a knowledge base.

If you omit embeddingConfiguration, the system defaults to Model Studio text-embedding-v4 with 1,024 dimensions, which suits most use cases.

Model selection guidelines:

• For general Chinese and English content, text-embedding-v4 (1,024 dimensions) balances semantic accuracy and performance.

• If you run a proprietary or third-party embedding service, use custom mode for consistency.

• Higher dimensions capture richer semantics but increase storage and compute costs. 1,024 is recommended for most scenarios.

Retrieval strategies

The retrieval configuration (retrievalConfiguration) resolves in this order:

Subspace multi-tenancy

A Subspace is an isolation partition within a knowledge base. Use Subspaces to isolate data by user, department, or tenant. Enable with "subspace": true at creation.

Key rules:

• With Subspaces enabled, all document operations (add, query, delete, list) and retrieval calls must include the subspace field, or the API returns an error.

• The Retrieve API queries up to 32 Subspaces at a time.

• Subspace names: 128 characters maximum.

• The Subspace setting is immutable after creation.

For example, a department administrator creates one knowledge base with Subspaces enabled and assigns each employee a Subspace (e.g., employee_zhangsan, employee_lisi). Documents are isolated per employee, while the administrator manages configuration centrally. To search across employees, pass multiple Subspace names in the Retrieve call.