Feature overview
Table Information Extraction is a deep learning feature that lets you create self-learning models. These models extract information from data with fixed layouts, such as tables and forms. You can define custom fields for extraction. The platform provides visual guides for data annotation and model training.
With high-quality images and annotations for over 100 training samples, a fine-tuned model can achieve an accuracy rate of over 95%.
The Toolbox also provides a classifier management tool and a field type management tool. These tools allow you to automatically classify and route different data types and achieve high-precision recognition through a single interface.
Advantages
High precision: Based on powerful pre-trained models from Alibaba Cloud, the multi-layout model can achieve over 95% accuracy after fine-tuning and training.
Few-shot learning: You only need to annotate a small amount of data to optimize and iterate the model. The model also has excellent generalization capabilities.
Low barrier to entry: No coding is required. The feature is out-of-the-box and lets you configure rules through a user-friendly and controllable interface.
High efficiency: The feature provides intelligent pre-annotation capabilities. Multiple users can collaborate on annotation, which reduces the required time.
Scenarios
Supports list-type tables and key-value (KV) pair-type tables. This feature is suitable for images or single-page PDF files that contain lined, semi-bordered, or borderless tables with minor or enumerable layout variations and require structured information extraction.
This feature is suitable for table and form data that requires continuous model optimization and has many data samples available for model training. Example scenarios include financial and tax reimbursement, qualification verification, and internal business document structuring.
Related links
OCR Document AutoML: Console
Table Information Extraction model task development guide: Online Debugging, API Reference, SDK Reference
User Guide
For a video tutorial on how to use Table Information Extraction, see the following video:
The following figure shows the workflow for creating a Table Information Extraction task. A minimum of 20 data samples are required to start model training.
Step 1: Data Preparation
Go to the Data Center - Dataset page of your Table Information Extraction project. You can upload and manage the data required for the model task. Click Add Dataset to go to the upload page. Enter a Dataset Name and upload your table business data.
A custom Table Information Extraction model requires at least 20 training data samples to achieve good recognition and extraction results.
Step 2: Data Annotation
Data annotation consists of three main steps: annotation task creation, annotation, and quality check.
Annotation task creation
On the Data Center - Annotation Task page, click Create Annotation Task. On the creation page, enter a Task Name. In the Upload Data section, select a dataset or upload data from your local machine. Then, proceed to Question Settings.
Pre-annotation: If you enable OCR pre-annotation, the text within a drawn box is automatically recognized during annotation. This improves annotation efficiency.
Question Library: This contains existing questions for the current task. You can view the question library and select suitable questions for your annotation task.
Field Name: The name of the recognition field that is exposed externally. This name corresponds to the name in the API operation. The field name must be globally unique.
Field Type: The property definition of a field. Selecting an appropriate field type can improve the end-to-end recognition effect. You can select a common field type or add a custom one. If no post-processing is needed, you can select a regular field type.
Annotation
Go to the Data Center - Annotation Task page. Select a created annotation task and click Annotate to go to the Data Annotation interface. On the toolbar, click Box Selection to complete a Box-select Question or Create Table. After you finish annotating all images and their fields, click Submit Task to complete this part of the annotation.
The quality of the annotated data (both text and position) directly affects the model training results and evaluation metrics.
If you encounter incorrect or unannotatable data, you can skip that image.
Quality check: Go to the Data Center - Annotation Task page. Select a task that has been annotated and click Quality Check to enter the data quality check interface. You need to verify that all images and their fields in the task are completely and correctly annotated. If there are errors, you must make changes or perform the Reject operation. If the annotations are correct, click Submit Task to complete the quality check.
Step 3: Model Training and Evaluation
Go to the Model Hub. Click Create Model to go to the model creation page. Select the annotation results for the training set and test set, and fill in the basic information. After the model is created, training starts automatically.
Training set: The data source used to train the model. You can only select datasets that have been annotated and have passed the quality check as the training set. Datasets that are already selected as a test set cannot be selected again. We recommend selecting more than 20 valid data samples for model training.
Test set: The data source used to test the model. You can only select datasets that have been annotated and have passed the quality check as the test set. Datasets that are already selected as a training set cannot be selected again.
Automatically use 1/10 of the training set as the test set: If you enable this option, you do not need to manually select a test set. The system automatically uses 1/10 of the training set as the test set. If a test set was already uploaded before you enable this option, the system ignores the manually uploaded test dataset.
Model training fees and estimated duration: The cost and time vary based on the task type and data volume. The values displayed on the interface prevail. For more information, see Billing of OCR Document AutoML.
Training duration: This is determined by multiple factors, such as data volume, annotation status, and machine resources. For example, using a V100 machine, training with 20 images takes about 1 hour, and training with 200 images takes about 2 hours.
Algorithm evaluation metrics: Includes three dimensions: overall metrics, field metrics, and table metrics.
Overall metrics
Accuracy: The average accuracy of the algorithm model before correction by post-processing rules. It is the ratio of correctly predicted annotation boxes (content and position) to all annotation boxes in the test set.
Precision: The average precision of the algorithm model before correction by post-processing rules. It is the ratio of true positive samples among all samples predicted as positive. In other words, it is the proportion of recognized fields in the test set that match the annotation boxes (content and position).
Recall: The average recall rate of the algorithm model before correction by post-processing rules. It is the ratio of correctly identified positive samples among all true positive samples. In other words, it is the proportion of correctly recognized annotation boxes (content and position) in the test set.
Field metrics
Precision: The single-field precision of the algorithm model before correction by post-processing rules. It is the probability that a sample predicted as positive is a true positive. In other words, it is the ratio of fields predicted as a specific field type to the true annotation boxes (content and position) of that field type in the test set.
Recall: The single-field recall rate of the algorithm model before correction by post-processing rules. It is the probability that a true positive sample is correctly identified as positive. In other words, it is the ratio of correctly recognized annotation boxes (content and position) of a specific field type in the test set.
F1-score: A comprehensive evaluation metric (F1-Measure). It is the weighted harmonic mean of precision and recall and is often used to evaluate the quality of a classification model.
Table metrics
Precision: The field precision for the current type in the algorithm model before correction by post-processing rules. It is the probability that a sample predicted as positive is a true positive. In other words, it is the ratio of correctly predicted fields of a specific type to the annotation boxes (content and position) of that field type in the current table type's test set.
Recall: The field recall rate for the current type in the algorithm model before correction by post-processing rules. It is the probability that a true positive sample is correctly identified as positive. In other words, it is the ratio of correctly recognized annotation boxes (content and position) of a specific field type in the current table type's test set.
F1-score: A comprehensive evaluation metric (F1-Measure). It is the weighted harmonic mean of precision and recall and is often used to evaluate the quality of a classification model.
Step 4: Model Deployment
After model training is complete, go to Model Hub - Model Details. Click the Deploy button at the bottom of the page to start model deployment. Model deployment takes some time. After a successful deployment, you can test the model's performance visually through the online experience or call the online service directly using the API.
After you publish and deploy the model service, you can see the API overview to use the service. You can also go to the Online Use page to test the model's performance.
OCR Document AutoML has been fully commercialized since August 23, 2023. Model training is billed by duration, and model inference is billed by the number of calls. For more information, see Billing of OCR Document AutoML.
Tool - Question Library
Question Library: This is used in the Question Settings step to pre-create annotation task fields. These questions can be referenced multiple times. If multiple annotation tasks require the same annotation fields, you can select them from the question library. This reduces the manual effort of editing questions multiple times and lowers the possibility of editing errors.
Modifying the question library does not affect annotation tasks or models that have already been initiated.