Document Q&A using IDP + RAG
Solution Description
This solution combines intelligent document processing (IDP) with retrieval-augmented generation (RAG). It integrates pre-trained large language models (LLMs), such as Qwen, with enterprise document data to generate enhanced responses backed by a custom knowledge base. This approach effectively mitigates LLM hallucination, improves response timeliness, and strengthens data security.
Environment Preparation
Integrate IDP with LlamaIndex using one of the following methods:
-
Use Alibaba Cloud Model Studio DashScopeParser. Enable the Docmind service to obtain an AccessKey ID and AccessKey secret. Alternatively, enable the DashScopeParse service through DashScope to obtain an API key.
-
Use the Document Mind parsing service. Obtain an AccessKey ID and AccessKey secret.
Before using the environment, install the LlamaIndex dependencies and the DashScopeParser and IDP-SDK Python packages.
pip install llama-index
pip install llama-index-readers-dashscope
pip install llama-index-llms-dashscope
pip install llama-index-vector-stores-dashvector
pip install https://doc-mind-pro.oss-cn-hangzhou.aliyuncs.com/doc_json_sdk-1.0.0-py3-none-any.whl
Solution Implementation Example
Document Q&A using DashScopeParser + RAG
Parse documents using DashScopeParse. For more information, see intelligently parse and chunk documents. Activate the service, obtain an API key, and set the environment variable as follows:
export DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEY
Call the service and process documents. Build and store vectors. This example uses Qwen as the LLM to process and answer questions about document content. It uses Qwen_max in DashScope to summarize document content. Enable the DashScope Qwen service.
import logging
import sys
import io
from llama_index.readers.dashscope.base import DashScopeParse
from llama_index.readers.dashscope.utils import ResultType
from doc_json_sdk.loader.document_model_loader import DocumentModelLoader
from doc_json_sdk.render.document_model_render import DocumentModelRender
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.vector_stores.dashvector import DashVectorStore
import dashvector
import os
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
# Document list
file_list = [
# your files (accept doc, docx, pdf)
]
parse = DashScopeParse(result_type=ResultType.DASHCOPE_DOCMIND)
documents = parse.load_data(file_path=file_list)
# Process JSON content into Markdown content
for document in documents:
loader = DocumentModelLoader()
docmind_document = loader.load(doc_json_fp=io.StringIO(document["text"]))
render = DocumentModelRender(document_model=docmind_document)
docmind_markdown_result = render.render_markdown_result()
document.update("text", docmind_markdown_result)
api_key = os.environ["DASHVECTOR_API_KEY"]
client = dashvector.Client(api_key=api_key)
client.create("llama-demo", dimension=1536)
dashvector_collection = client.get("quickstart")
vector_store = DashVectorStore(dashvector_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
documents, storage_context=storage_context
)
from llama_index.llms.dashscope import DashScope, DashScopeGenerationModels
dashscope_llm = DashScope(model_name=DashScopeGenerationModels.QWEN_MAX)
query_engine = index.as_query_engine(llm=dashscope_llm)
response = query_engine.query("Report highlights")
Document Q&A using Docmind + RAG
Use Document Mind for intelligent parsing. For more information, see the new user guide. Activate the service, obtain an AccessKey ID and AccessKey secret, and set the environment variables.
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=YOUR_ALIBABA_CLOUD_ACCESS_KEY_SECRET
export ALIBABA_CLOUD_ACCESS_KEY_ID=YOUR_ALIBABA_CLOUD_ACCESS_KEY_ID
Call the service and process documents. Build and store vectors. This example uses Qwen as the LLM to process and answer questions about document content. It uses Qwen_max in DashScope to summarize document content. Enable the DashScope Qwen service.
from doc_json_sdk.loader.document_model_loader import DocumentModelLoader
from doc_json_sdk.handler.document_handler import DocumentExtractHandler, DocumentDigitalExtractHandler
from doc_json_sdk.render.document_model_render import DocumentModelRender
from llama_index.core.schema import Document
import os
import dashvector
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.vector_stores.dashvector import DashVectorStore
# your files (accept doc, docx, pdf)
file_path = ""
# Loader file
loader = DocumentModelLoader(handler=DocumentExtractHandler())
docmind_document = loader.load(file_path=file_path)
# Render as Markdown
render = DocumentModelRender(document_model=docmind_document)
docmind_markdown_result = render.render_markdown_result()
document = Document(text=docmind_markdown_result, metadata={})
documents = [document]
api_key = os.environ["DASHVECTOR_API_KEY"]
client = dashvector.Client(api_key=api_key)
client.create("llama-demo", dimension=1536)
dashvector_collection = client.get("quickstart")
vector_store = DashVectorStore(dashvector_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
documents, storage_context=storage_context
)
from llama_index.llms.dashscope import DashScope, DashScopeGenerationModels
dashscope_llm = DashScope(model_name=DashScopeGenerationModels.QWEN_MAX)
query_engine = index.as_query_engine(llm=dashscope_llm)
response = query_engine.query("Report highlights")