DocsICP FilingConsole
官方文档
首页

Long Document Information Extraction

更新时间：
复制 MD 格式
产品详情
我的收藏

Introduction

  • Long Document Information Extraction is a self-learning information extraction task based on deep learning. It lets you define custom fields for extraction. The platform provides visual guides to help you annotate data and train models. This process enables high-precision extraction from unstructured documents with various layouts.

  • For high-quality images, fine-tuning a model with over 100 annotated training samples achieves an accuracy rate of over 85%.

Benefits

  • High precision: Using Alibaba Cloud's powerful pre-trained models, fine-tuned models for multiple layouts can achieve an accuracy rate of over 85%.

  • Few-shot learning: You can optimize and iterate models by annotating only a small amount of data. The models also have good generalization capabilities.

  • Low barrier to entry: You can use the feature out-of-the-box without writing any code and configure rules independently through a user-friendly and controllable interface.

  • High efficiency: The platform provides intelligent pre-annotation capabilities, which shortens the time required for collaborative annotation by multiple users.

Scenarios

  • High-performance model: Suitable for documents with simple styles or formats, such as documents that contain only titles and paragraphs. Supported document formats include PDF and images. This model is applicable to scenarios such as certificates, official papers, files, letters, and announcements.

  • Hybrid-layout model: Suitable for documents with rich styles or formats, such as documents that include titles, paragraphs, tables, and forms. Supported document formats include PDF and images. This model is applicable to scenarios such as contracts, bids, insurance policies, and engineering forms.

  • Long documents that require continuous optimization and have many data samples (more than 20) available for model training to improve performance.

Related links

Operation guide

For a video tutorial on how to use Long Document Information Extraction, see the following video:

The following figure shows the workflow for creating a Long Document Information Extraction task. You need more than 20 images for training to complete model creation.

image

Step 1: Prepare data

In the **Data Center** > **Datasets** section, you can upload and manage the data required for your model task. Click **Add Dataset** to open the upload interface, edit the dataset name, and upload your business data.

Note

To achieve better recognition and extraction results, a custom Long Document Information Extraction model requires at least 50 training images.

Tips:

  1. What are the specific requirements for data preparation?

    1. You can upload images, documents, or compressed packages to a dataset.

      1. Documents must be in PDF format and no larger than 20 MB. We recommend that a single PDF file has no more than 5 pages.

      2. Images must be in JPG, JPEG, or PNG format and no larger than 10 MB.

      3. Compressed packages must be in ZIP format and no larger than 20 MB.

    2. For a single image, the longest edge cannot exceed 8192 pixels, and the shortest edge must be at least 15 pixels. If the longest edge exceeds 1024 pixels, the aspect ratio cannot exceed 50:1.

    3. Prepare at least 50 to 60 data samples of the same task type for model training and evaluation.

  2. How can I obtain better recognition results?

    1. For tasks within the product's scope, higher data quality leads to better recognition and extraction results. Clear and neat handwriting results in higher accuracy.

    2. Keep the size of single characters between 10 and 50 pixels for better recognition.

    3. The data is derived from real business scenarios and includes a wide variety of types and formats.

Step 2: Annotate data

Data annotation is divided into three main steps: creating an annotation task, annotating data, and performing a quality check.

Create an annotation task

On the **Data Center** > **Annotation Tasks** page, click Create Annotation Task to open the creation page. Edit the Task Name and select a dataset or upload local files in the Upload Data section. After you finish, proceed to Field Settings.

Pre-annotation: After you enable OCR pre-annotation, the text content within a drawn box is automatically detected during annotation. This improves annotation efficiency.

Question Bank: A collection of predefined questions that you can use to create an annotation task.

Field Name: The name of the recognition field that is exposed externally. This corresponds to the name in the API operation. The field name must be globally unique.

Field Type: The property definition of the field. Selecting an appropriate field type can improve the end-to-end recognition result. You can select a common field or add a custom field. If no post-processing is needed, select a regular field type.

Tips:

  1. How to upload data for an annotation task:

    1. You can upload data from your local machine or select it from a dataset. If you have already uploaded data to a dataset, you can select it from there. Click **Add Dataset** above the list and select the dataset you want to annotate.

  2. How to fill in key-value information for extraction:

    1. Enter all the field names you need to annotate into the content box. Selecting the appropriate field type can improve field recognition accuracy.

    2. You can select a created **Field Library** to reference its fields. For the same model, we recommend selecting the same **Field Library** to ensure that the field settings are identical.

Annotate

In the **Data Center** > **Annotation Tasks** section, select the created annotation task and click Annotate to go to the Data Annotation page. In the annotation tool, use the Draw Box button to draw boxes around the fields to be detected. Select the corresponding field and carefully check the automatically detected text. After all images and their fields are annotated, click Submit Task to complete the annotation for this part.

Important

The quality of the annotated data, including text and position, directly impacts the model's training performance and evaluation metrics.

If you encounter incorrect or un-annotatable data, you can skip that image.

Tips:

  1. Long Document Information Extraction currently supports only rectangular boxes for field annotation. When drawing a box, make sure its four corners are as close to the text as possible.

  2. Ensure that the data is clear, neat, and unobstructed, and that all data layout types are covered.

  3. OCR pre-annotation supports printed text recognition but does not currently support handwriting recognition. Carefully check the text content automatically detected by pre-annotation to ensure accuracy.

  4. How to annotate across multiple lines?

    1. Annotate the parts that span multiple lines separately and assign them the correct field.

    2. Press and hold the Shift key, and then click the edges of the two previously created annotation boxes.

    3. Release the Shift key. The data that spans multiple lines is automatically merged. Then, select the corresponding field.

    4. Click OK. A "group" appears on the right. You do not need to delete the individual small boxes.

Quality check

Go to the **Data Center** > **Annotation Tasks** page and select a completed annotation task to perform a quality check. The quality checker can modify or reject annotations. After all annotation tasks are completed, submit the task. Make sure that all fields have been annotated.

Tips

  1. How to perform a quality check?

    1. The quality checker can directly use the annotation tool to add, delete, or modify annotation information.

    2. When the quality checker selects **Submit**, all images are considered to have passed the quality check.

  2. What if the annotation quality is low during the quality check?

    1. Select the corresponding field and edit it directly.

    2. Select **Reject** to return the task to the annotator for re-annotation.

Step 3: Train and evaluate the model

Go to the **Model Hub** and click Create Model to open the model creation page. Select the annotation results for the training dataset and test dataset, and fill in the basic information. After the model is created successfully, model training starts automatically.

Pre-trained Model: Long Document Information Extraction currently supports two versions: **High-performance** and **Hybrid-layout**.

High-performance model: Suitable for documents with simple styles or formats, such as documents that contain only titles and paragraphs. Supported document formats include PDF and images.

Hybrid-layout model: Suitable for documents with rich styles or formats, such as documents that include titles, paragraphs, tables, and forms. Supported document formats include PDF and images.

Training Dataset: The data source used to train the model. You can only select datasets that have been annotated and quality-checked as the training dataset. Datasets already selected as a test dataset cannot be selected again. We recommend selecting more than 20 valid data samples for model training.

Test Dataset: The data source used to test the model. You can only select datasets that have been annotated and quality-checked as the test dataset. Datasets already selected as a training dataset cannot be selected again.

Automatically use 1/10 of the training dataset as the test dataset: If you enable this option, you do not need to manually select a test dataset. The system automatically partitions 1/10 of the training dataset to use as the test dataset. If a test dataset was already uploaded before you enabled this feature, the system will ignore the manually uploaded test dataset.

Model training fees and the estimated duration vary based on the task type and data volume. The values displayed on the interface are final.

Training Duration: This is determined by multiple factors, such as data volume, annotation status, and machine resources. For example, using a V100 machine, it takes about 1 minute to train on 60,000 characters.

Tips:

  1. You must verify that the fields in the test dataset and training dataset are consistent.

  2. During the model training configuration, you can select the field type corresponding to each field or delete fields as needed to improve training accuracy.

  3. The larger the data volume and the more precise the annotations, the better the model training and evaluation results.

  4. Configure appropriate field types to improve end-to-end accuracy.

Algorithm evaluation metrics: The metrics include overall metrics and field-specific metrics.

Overall metrics - Mean

  • Precision: The precision of the algorithm model before post-processing rules are applied. It is the ratio of correctly identified positive class samples to all samples identified as a positive class. The formula is: Number of correctly predicted fields / Total number of predicted fields. This metric measures how many of the predicted fields match the annotated content and position in the test dataset.image

  • Recall: The recall rate of the algorithm model before post-processing rules are applied. It is the ratio of correctly identified positive class samples to all actual positive class samples. The formula is: Number of correctly predicted fields / Total number of actual correct fields.image

  • F1-score: A comprehensive evaluation metric (F1-Measure). It is the weighted harmonic mean of precision and recall and is often used to evaluate the performance of a classification model.image

Field-specific metrics

  • Precision: The precision of the algorithm model before post-processing rules are applied. In the test dataset, this is the number of correct fields for a specific field type / the total number of predicted fields for that field type.image

  • Recall: The recall rate of the algorithm model before post-processing rules are applied. In the test dataset, this is the number of correct fields for a specific field type / the total number of actual correct fields for that field type.image

  • F1-score: A comprehensive evaluation metric (F1-Measure) before post-processing rules are applied. In the test dataset, it is the weighted harmonic mean of precision and recall.image

Fine-tuning suggestions:

How to fine-tune the model?

  1. Ensure that the task type is within the scope of the feature and that the data quality is as clear and neat as possible.

  2. During data annotation, annotate each recognition field completely. The annotation boxes should fit the field text as closely as possible. You can also adjust field types or advanced options to improve accuracy.

  3. The larger the data volume, the better the model training result. Generally, a model with more than 200 data samples will perform well.

  4. The distribution of data types and layouts should match real business scenarios. The model can only achieve good recognition results for data types and layouts it has been trained on.

Step 4: Deploy the model

After model training is complete, go to **Model Hub** > **Model Details** and click the **Deploy** button at the bottom of the page to start model deployment. Model deployment takes some time. After the deployment is successful, you can test the model's performance on the online testing page or call the online service directly using the API.

Note

After you publish and deploy the model service, you can use the service. For more information, see the API overview document. You can also go to the Online Use page to test the model's performance.

Important

OCR Document AutoML started its full commercialization on August 23, 2023. Model training is billed by duration, and model inference is billed by the number of calls. For more information, see OCR Document AutoML billing.

Widget - Question Bank

Field Library: The field library is used in the **Field Settings** step. You can pre-create fields for annotation tasks, and these fields can be referenced multiple times. If multiple annotation tasks require the same fields, you can select them from the field library. This reduces the manual effort of editing fields multiple times and lowers the risk of editing errors.

Note

Modifying the field library does not affect annotation tasks or models that have already been initiated.