Summarize and extract information from documents using IDP and an LLM
Combine the output from Intelligent Document Processing (IDP) with a Large Language Model (LLM) and user prompts to extract, summarize, or format information from documents. This topic explains how to summarize documents using IDP and an LLM.
Solution description
You can combine the output from IDP with an LLM and user prompts to extract, summarize, or format information from documents. This topic describes how to use this process to summarize documents.
Prerequisites
You can call the Document Mind service to obtain an AccessKey ID and an AccessKey secret.
Install the IDP-SDK and OpenAI Python packages.
pip install llama-index
pip install llama-index-readers-dashscope
pip install llama-index-llms-dashscope
pip install https://doc-mind-pro.oss-cn-hangzhou.aliyuncs.com/doc_json_sdk-1.0.2-py3-none-any.whl Example implementation
This example calls the Alibaba Cloud Document Mind service with the Qwen model to extract and summarize information.
dashscopeParse, Document Mind parsing, and Document Mind parsing (LLM version) are each billed separately.
Summarize documents using Document Mind and Qwen
Activate the Document Mind service by following the Quick Start guide. Then, obtain an AccessKey ID and an AccessKey secret, and set the environment variables.
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=YOUR_ALIBABA_CLOUD_ACCESS_KEY_SECRET
export ALIBABA_CLOUD_ACCESS_KEY_ID=YOUR_ALIBABA_CLOUD_ACCESS_KEY_ID You can activate the Qwen model through DashScope to obtain an API key, and then call the service to process the document.
import logging
import sys
import io
import os
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
from doc_json_sdk.loader.document_model_loader import DocumentModelLoader
from doc_json_sdk.handler.document_handler import DocumentExtractHandler, DocumentDigitalExtractHandler, DocumentParserWithCallbackHandler
from doc_json_sdk.render.document_model_render import DocumentModelRender
from llama_index.core.schema import Document
file_path = "/local_file" # your file
# handler: For more parameters, see the IDP-SDK Quick Start.
# DocumentExtractHandler: Intelligent document parsing.
# DocumentDigitalExtractHandler: Digital document parsing.
loader = DocumentModelLoader(handler=DocumentExtractHandler())
docmind_document = loader.load(file_path=file_path,
reveal_markdown=True, # Renders tables and image links in Markdown. Set this to False if you do not process images.
formula_enhancement=False, # Formula enhancement.
use_url_response_body=True)
# or file url
# docmind_document = loader.load(file_url="http://xxxx")
render = DocumentModelRender(document_model=docmind_document)
docmind_markdown_result = render.render_markdown_result()
# View the Markdown result.
print(docmind_markdown_result)
# use qwen_max as llm
from llama_index.llms.dashscope import DashScope, DashScopeGenerationModels
dashscope_llm = DashScope(model_name=DashScopeGenerationModels.QWEN_MAX,api_key="your-dashscope-sk")
# Summarize
response = dashscope_llm.complete("Summarize the document content and present the result in Markdown format \n: "+docmind_markdown_result)
print("Summary:----------------------\n")
print(response.text)
# Extract
response = dashscope_llm.complete("Extract the document content and present it as key-value pairs \n: "+docmind_markdown_result)
print("Extraction:----------------------\n")
print(response.text)