DocsICP FilingConsole
官方文档
首页

Long document information extraction

更新时间：
复制 MD 格式
产品详情
我的收藏

Introduction

  • Long document information extraction is a self-learning model task based on deep learning. It allows you to define custom extraction fields and provides a guided visual interface for data annotation and model training. This process enables high-precision information extraction from unstructured documents with various layouts.

  • With high-quality images and annotations from over 100 training samples, a fine-tuned model can achieve an accuracy of over 85%.

Benefits

  • High precision: Powered by Alibaba Cloud's robust pre-trained models, a fine-tuned mixed-layout model can achieve an accuracy of over 85%.

  • Few-shot learning: You only need to annotate a small amount of data to optimize and iterate on your model, which has strong generalization capabilities.

  • Low barrier to entry: The service is available out-of-the-box, with a user-friendly interface for rule configuration.

  • High efficiency: The intelligent pre-annotation feature and support for collaborative annotation significantly reduce the required time.

Use cases

  • High-performance model: Ideal for documents with simple styles and formats, such as those containing only titles and paragraphs. It supports PDF and image formats and is suitable for use cases like certificates, official documents, files, letters, and announcements.

  • Mixed-layout model: Designed for documents with rich styles and formats, including titles, paragraphs, tables, and forms. It supports PDF and image formats and is suitable for use cases like contracts, bidding documents, insurance policies, and engineering forms.

  • This feature is also suitable for long documents where continuous model optimization is required and a substantial amount of data (more than 20 samples) is available for training to improve performance.

Related links

Procedure

Watch this video for a walkthrough of the long document information extraction process:

The following diagram shows the workflow for creating a long document information extraction model. You need more than 20 images for training to create a model.

image

Step 1: Prepare data

In Data Center > Datasets, you can upload and manage the data required for your model task. Click Add Dataset, enter a name for the dataset, and upload your business data.

Note

For a custom long document information extraction model, you need at least 50 training documents to achieve good recognition and extraction results.

Tips:

  1. Data preparation requirements

    1. You can upload images, documents, or compressed packages to a dataset.

      1. Documents: Must be PDF files no larger than 20 MB. We recommend that each PDF file has no more than 5 pages.

      2. Images: Must be in JPG, JPEG, or PNG format and no larger than 10 MB.

      3. Compressed packages: Must be in ZIP format and no larger than 20 MB.

    2. For a single image, the longest side cannot exceed 8,192 pixels, and the shortest side must be at least 15 pixels. If the longest side exceeds 1,024 pixels, the aspect ratio must not exceed 50:1.

    3. Prepare at least 50 documents of the same type for model training and evaluation.

  2. Tips for better recognition results

    1. High-quality data with clear text leads to better recognition and extraction accuracy.

    2. For best results, ensure that the size of individual characters is between 10 and 50 pixels.

    3. Use data from real-world business scenarios that cover all relevant types and layouts.

Step 2: Annotate data

Data annotation involves three main steps: task creation, annotation, and quality check.

Go to Data Center > Annotation Tasks and click Create Annotation Task. Enter a Task Name and select a dataset or upload data directly under Upload Data. Then, proceed to Field Settings.

For example, add two fields, Party A's Name and Party B's Name, and select general as the field type for both.

Pre-annotation: If you enable OCR pre-annotation, the text within any box you draw during annotation will be automatically recognized, improving your efficiency.

Field library: The field library lets you reuse predefined fields for your annotation tasks.

Field Name: This is the external name for the recognition field, which corresponds to the name in the API. Field names must be globally unique.

Field Type: This defines the field's properties. Choosing the right field type can improve end-to-end recognition performance. You can choose from common field types or add your own custom types. If no post-processing is needed, select the general field type.

Tips:

  1. Uploading data for an annotation task

    1. You can upload data from your local machine or select it from a dataset. If you have already uploaded your data to a dataset, click **Add Dataset** above the list and choose the dataset you want to annotate.

  2. Configuring key-value pair extraction

    1. Enter all the field names you want to annotate in the content box. Selecting the appropriate field type can improve extraction accuracy.

    2. You can reuse fields from an existing field library. We recommend using the same field library for a single model to ensure consistent field settings.

Annotation

In Data Center > Annotation Tasks, select a created task and click Annotate to go to the Data Annotation interface. In the annotation tool, use the Bounding Box Selection tool to draw a bounding box around a field you want to recognize, select the corresponding field name, and carefully check the automatically recognized text. After annotating all fields in all images, click Submit Task.

Important

The quality of your annotated data (both the text and its location) directly affects the model's training performance and evaluation metrics.

You can skip images that are incorrect or cannot be annotated.

Tips:

  1. Long document information extraction currently supports using a bounding box to select fields for recognition. When drawing a bounding box, make sure its corners are as close to the text as possible.

  2. Ensure your data is clear, well-formatted, and free of obstructions. Also, make sure it covers all the layout types you expect to see.

  3. OCR pre-annotation supports printed text but not handwritten text. Always double-check the automatically recognized text to ensure annotation accuracy.

  4. Annotating text that spans multiple lines

    1. Draw a separate box for each part of the text on different lines and assign them the correct field name.

    2. Press and hold the Shift key and click the borders of the annotation boxes you just created.

    3. Release the Shift key. The boxes will be automatically merged. Select the correct field name for the combined data.

    4. Click **OK**. A "combination" appears in the right-side panel; you do not need to delete the original, smaller boxes.

Quality check

To start a quality check, go to Data Center > Annotation Tasks and select a completed annotation task. The quality checker reviews all annotations, modifying or rejecting them as needed, before submitting the task.

The top of the quality check interface displays statistics such as progress, error count, and accuracy. The left side is the document preview area. The Bounding Box Selection Fields panel on the right lists all annotated fields and their recognized results, allowing you to review, edit, or delete each one. The Annotation History area at the bottom includes a Mark as Error button.

Tips

  1. Performing a quality check

    1. The quality checker can add, delete, or modify annotations directly using the annotation tool.

    2. When the quality checker clicks **Submit**, the quality check is complete for all images in the task.

  2. Handling low-quality annotations

    1. Select the incorrect field and edit it directly.

    2. Click **Reject** to send the task back to the annotator for correction.

Step 3: Train and evaluate the model

Go to the Model Center and click Create Model. On the model creation screen, select the annotated results for your training and test sets, and fill in the basic information. After the model is created, training starts automatically.

Pre-trained models: Long document information extraction currently offers two model types: high-performance model and mixed-layout model.

High-performance model: Ideal for documents with simple styles and formats, such as those containing only titles and paragraphs. It supports PDF and image formats.

Mixed-layout model: Designed for documents with complex styles and formats, such as those including titles, paragraphs, tables, and forms. It supports PDF and image formats.

Training set: The data source used to train the model. You can only select datasets that have been annotated and have passed a quality check. A dataset selected as a test set cannot also be a training set. We recommend using at least 20 valid documents for model training.

Test set: The data source used to evaluate the model. You can only select datasets that have been annotated and have passed a quality check. A dataset selected as a training set cannot also be a test set.

Automatically use 1/10 of the training set as a test set: When this option is enabled, you do not need to select a test set because the system automatically reserves 10% of the training set for testing. The system ignores any manually uploaded test set if this option is enabled.

On the Model Training configuration page, the document preview area is on the left, and the field configuration area is on the right. Here, you can set the field type for each field, such as general.

Model training costs and estimated duration vary based on the task type and data quantity. The exact figures are displayed on the interface.

Training Duration: The time required for training depends on multiple factors, including data volume, annotation quality, and machine resources. For example, training on 60,000 characters using a V100 GPU takes about 1 minute.

Tips:

  1. You must verify that the fields in your training set and test set are consistent.

  2. During model training configuration, you can select the appropriate field type for each field or remove fields based on your business needs to improve training accuracy.

  3. Larger and more accurately annotated datasets generally result in better model training and evaluation performance.

  4. Configure appropriate field types to improve end-to-end accuracy.

After model training is complete, you can view the training duration, start and end times, and progress. The Service Parameter Configuration area includes parameters such as expected inference hardware, inference time (ms), maximum training duration (h), and computing specifications.

Algorithm evaluation metrics: Metrics are provided for two dimensions: overall metrics and field-specific metrics.

Overall metrics

  • Precision: In the test set, precision measures the proportion of predicted fields that perfectly match the ground truth annotation in both content and location.image

  • Recall: In the test set, recall is the ratio of correctly predicted fields to the total number of actual fields that should have been predicted.image

  • F1-score: The harmonic mean of precision and recall, providing a single score that balances both metrics.image

Field-specific metrics

  • Precision: For a specific field, this is the ratio of correctly predicted instances to the total number of predicted instances of that field.image

  • Recall: For a specific field, this is the ratio of correctly predicted instances to the total number of actual instances of that field.image

  • F1-score: The harmonic mean of precision and recall for a specific field.image

Tuning recommendations:

Fine-tuning the model

  1. Ensure that your task type is within the scope of the feature and that your data quality is as high as possible.

  2. During data annotation, make sure to annotate every field completely. The annotation boxes should fit the text as closely as possible. You can also adjust field types or advanced options to improve accuracy.

  3. The more data you have, the better the model will perform. A dataset of 200+ documents typically yields good results.

  4. Ensure that the data types and layout distributions in your training set match your real-world business scenarios. The model will only perform well on data types and layouts it has been trained on.

Step 4: Deploy the model

After the model is trained, go to Model Center > Model Details and click the Deploy button at the bottom of the page to start model deployment. After a brief deployment period, you can visually test the model in the online trial or call the service via the API.

Note

After you publish and deploy the model service, see the API overview for information about using the service. You can also go to the online trial interface to test the model's performance.

Important

OCR Self-learning has been commercially available since August 23, 2023. Model training is billed by duration, and model inference is billed per call. For more information, see OCR Self-learning Billing.

Tool: Field library

Field library: A tool in the Field Settings step that lets you pre-create and reuse fields across multiple annotation tasks. This reduces repetitive work and minimizes errors.

Note

Modifying a field library does not affect any annotation tasks or models that have already been created.

Previous：Table Information ExtractionNext：Toolbox