General Table Recognition Pipeline User Guide

1. Introduction to General Table Recognition Pipeline

Table recognition is a technology that automatically identifies and extracts table content and structure from documents or images. It is widely used in data entry, information retrieval, and document analysis. By using computer vision and machine learning algorithms, table recognition can convert complex table information into editable formats, facilitating further processing and analysis of data.

The General Table Recognition Pipeline is designed to solve table recognition tasks by identifying tables in images and outputting them in HTML format. This pipeline integrates the well-known SLANet and SLANet_plus table recognition models. Based on this pipeline, precise predictions of tables can be achieved, covering a wide range of applications in general, manufacturing, finance, transportation, and other fields. The pipeline also provides flexible service deployment options, supporting various hardware and programming languages for integration. Moreover, it offers secondary development capabilities, allowing you to train and optimize models on your own dataset, which can then be seamlessly integrated.

The General Table Recognition Pipeline includes essential modules for table structure recognition, text detection, and text recognition, as well as optional modules for layout area detection, document image orientation classification, and text image correction.

If you prioritize model accuracy, choose a high-accuracy model; if you prioritize inference speed, choose a faster model; if you care about model size, choose a smaller model.

Table Recognition Module Models:

Model	Model Download Link	Accuracy (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
SLANet	Inference Model/Training Model	59.52	103.08 / 103.08	197.99 / 197.99	6.9 M	SLANet is a self-developed table structure recognition model by the PaddlePaddle Vision Team. This model significantly improves the accuracy and inference speed of table structure recognition by using a CPU-friendly lightweight backbone network PP-LCNet, a high-low feature fusion module CSP-PAN, and a feature decoding module SLA Head that aligns structure and position information.
SLANet_plus	Inference Model/Training Model	63.69	140.29 / 140.29	195.39 / 195.39	6.9 M

Text Detection Module Models:

Model	Model Download Link	Detection Hmean (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-OCRv4_server_det	Inference Model/Training Model	82.69	83.34 / 80.91	442.58 / 442.58	109	The server text detection model of PP-OCRv4, with higher accuracy, suitable for deployment on high-performance servers.
PP-OCRv4_mobile_det	Inference Model/Training Model	77.79	8.79 / 3.13	51.00 / 28.58	4.7	The mobile text detection model of PP-OCRv4, which is more efficient and suitable for deployment on edge devices.

Text Recognition Module Models:

Model	Model Download Link	Recognition Avg Accuracy(%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-OCRv4_mobile_rec	Inference Model/Training Model	78.20	4.82 / 4.82	16.74 / 4.64	10.6 M	PP-OCRv4 is the next version of the self-developed text recognition model PP-OCRv3 by the PaddlePaddle Vision Team. By introducing data augmentation and GTC-NRTR guidance branches, it further improves text recognition accuracy without changing the model inference speed. This model provides both server and mobile versions to meet industrial needs in different scenarios.
PP-OCRv4_server_rec	Inference Model/Training Model	79.20	6.58 / 6.58	33.17 / 33.17	71.2 M

Model	Model Download Link	Recognition Avg Accuracy(%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
ch_SVTRv2_rec	Inference Model/Training Model	68.81	8.08 / 8.08	50.17 / 42.50	73.9 M	SVTRv2 is a server text recognition model developed by the OpenOCR team from the Visual and Learning Laboratory (FVL) at Fudan University. It won the first prize in the PaddleOCR Algorithm Model Challenge - Task 1: End-to-End OCR Recognition, with a 6% improvement in end-to-end recognition accuracy compared to PP-OCRv4.

Model	Model Download Link	Recognition Avg Accuracy(%)	GPU Inference Time (ms) [Regular Mode / High-Performance Mode]	CPU Inference Time (ms) [Regular Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
ch_RepSVTR_rec	Inference Model/Training Model	65.07	5.93 / 5.93	20.73 / 7.32	22.1 M	The RepSVTR text recognition model is a mobile text recognition model based on SVTRv2. It won the first prize in the PaddleOCR algorithm model competition - Task 1: OCR end-to-end recognition task. Compared with PP-OCRv4, the end-to-end recognition accuracy on the B list was improved by 2.5%, while the inference speed remained the same.

Layout Detection Module Models (Optional):

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Regular Mode / High-Performance Mode]	CPU Inference Time (ms) [Regular Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PicoDet_layout_1x	Inference Model/Training Model	86.8	9.03 / 3.10	25.82 / 20.70	7.4	A high-efficiency layout detection model trained on the PubLayNet dataset based on PicoDet-1x, capable of locating five types of regions: text, title, table, image, and list.
PicoDet_layout_1x_table	Inference Model/Training Model	95.7	8.02 / 3.09	23.70 / 20.41	7.4 M	A high-efficiency layout detection model trained on a custom dataset based on PicoDet-1x, capable of locating one type of region: table.
PicoDet-S_layout_3cls	Inference Model/Training Model	87.1	8.99 / 2.22	16.11 / 8.73	4.8	A high-efficiency layout detection model trained on a custom dataset in Chinese and English papers, magazines, and research reports based on the lightweight PicoDet-S model, including three categories: table, image, and seal.
PicoDet-S_layout_17cls	Inference Model/Training Model	70.3	9.11 / 2.12	15.42 / 9.12	4.8	A high-efficiency layout detection model trained on a custom dataset in Chinese and English papers, magazines, and research reports based on the lightweight PicoDet-S model, including 17 common layout categories: paragraph title, image, text, number, abstract, content, chart title, formula, table, table title, reference, document title, footnote, header, algorithm, footer, seal.
PicoDet-L_layout_3cls	Inference Model/Training Model	89.3	13.05 / 4.50	41.30 / 41.30	22.6	A high-efficiency layout detection model trained on a custom dataset in Chinese and English papers, magazines, and research reports based on PicoDet-L, including three categories: table, image, and seal.
PicoDet-L_layout_17cls	Inference Model/Training Model	79.9	13.50 / 4.69	43.32 / 43.32	22.6	A high-efficiency layout detection model trained on a custom dataset in Chinese and English papers, magazines, and research reports based on PicoDet-L, including 17 common layout categories: paragraph title, image, text, number, abstract, content, chart title, formula, table, table title, reference, document title, footnote, header, algorithm, footer, seal.
RT-DETR-H_layout_3cls	Inference Model/Training Model	95.9	114.93 / 27.71	947.56 / 947.56	470.1	A high-precision layout detection model trained on a custom dataset in Chinese and English papers, magazines, and research reports based on RT-DETR-H, including three categories: table, image, and seal.
RT-DETR-H_layout_17cls	Inference Model/Training Model	92.6	115.29 / 104.09	995.27 / 995.27	470.2	A high-precision layout detection model trained on a custom dataset in Chinese and English papers, magazines, and research reports based on RT-DETR-H, including 17 common layout categories: paragraph title, image, text, number, abstract, content, chart title, formula, table, table title, reference, document title, footnote, header, algorithm, footer, seal.

Text Image Rectification Module Models (Optional):

Model	Model Download Link	MS-SSIM (%)	Model Storage Size (M)	Introduction
UVDoc	Inference Model/Training Model	54.40	30.3 M	High-precision text image rectification model.

Document Image Orientation Classification Module Models (Optional):

Model	Model Download Link	Top-1 Acc (%)	GPU Inference Time (ms) [Regular Mode / High-Performance Mode]	CPU Inference Time (ms) [Regular Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-LCNet_x1_0_doc_ori	Inference Model/Training Model	99.06	2.31 / 0.43	3.37 / 1.27	7	A document image classification model based on PP-LCNet_x1_0, containing four categories: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

Text Recognition Module Models:

Model	Model Download Link	Recognition Avg Accuracy(%)	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-OCRv4_server_rec_doc	Inference Model/Training Model	81.53	6.65 / 2.38	32.92 / 32.92	74.7 M	PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data based on PP-OCRv4_server_rec. It has added the ability to recognize some traditional Chinese characters, Japanese, and special characters, and can support the recognition of more than 15,000 characters. In addition to improving the text recognition capability related to documents, it also enhances the general text recognition capability.
PP-OCRv4_mobile_rec	Inference Model/Training Model	78.74	4.82 / 1.20	16.74 / 4.64	10.6 M	The lightweight recognition model of PP-OCRv4 has high inference efficiency and can be deployed on various hardware devices, including edge devices.
PP-OCRv4_server_rec	Inference Model/Training Model	80.61	6.58 / 2.43	33.17 / 33.17	71.2 M	The server-side model of PP-OCRv4 offers high inference accuracy and can be deployed on various types of servers.
en_PP-OCRv4_mobile_rec	Inference Model/Training Model	70.39	4.81 / 0.75	16.10 / 5.31	6.8 M	The ultra-lightweight English recognition model, trained based on the PP-OCRv4 recognition model, supports the recognition of English letters and numbers.

❗ The above list features the 4 core models that the text recognition module primarily supports. In total, this module supports 18 models. The complete list of models is as follows:

👉Model List Details

* Chinese Recognition Model

Model	Model Download Link	Recognition Avg Accuracy(%)	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-OCRv4_server_rec_doc	Inference Model/Training Model	81.53	6.65 / 2.38	32.92 / 32.92	74.7 M	PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data based on PP-OCRv4_server_rec. It has added the recognition capabilities for some traditional Chinese characters, Japanese, and special characters. The number of recognizable characters is over 15,000. In addition to the improvement in document-related text recognition, it also enhances the general text recognition capability.
PP-OCRv4_mobile_rec	Inference Model/Training Model	78.74	4.82 / 1.20	16.74 / 4.64	10.6 M	The lightweight recognition model of PP-OCRv4 has high inference efficiency and can be deployed on various hardware devices, including edge devices.
PP-OCRv4_server_rec	Inference Model/Trained Model	80.61	6.58 / 2.43	33.17 / 33.17	71.2 M	The server-side model of PP-OCRv4 offers high inference accuracy and can be deployed on various types of servers.
PP-OCRv3_mobile_rec	Inference Model/Training Model	72.96	5.87 / 1.19	9.07 / 4.28	9.2 M	PP-OCRv3’s lightweight recognition model is designed for high inference efficiency and can be deployed on a variety of hardware devices, including edge devices.

Model	Model Download Link	Recognition Avg Accuracy(%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
ch_SVTRv2_rec	Inference Model/Training Model	68.81	8.08 / 2.74	50.17 / 42.50	73.9 M	SVTRv2 is a server text recognition model developed by the OpenOCR team of Fudan University's Visual and Learning Laboratory (FVL). It won the first prize in the PaddleOCR Algorithm Model Challenge - Task One: OCR End-to-End Recognition Task. The end-to-end recognition accuracy on the A list is 6% higher than that of PP-OCRv4.

Model	Model Download Link	Recognition Avg Accuracy(%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
ch_RepSVTR_rec	Inference Model/Training Model	65.07	5.93 / 1.62	20.73 / 7.32	22.1 M	The RepSVTR text recognition model is a mobile text recognition model based on SVTRv2. It won the first prize in the PaddleOCR Algorithm Model Challenge - Task One: OCR End-to-End Recognition Task. The end-to-end recognition accuracy on the B list is 2.5% higher than that of PP-OCRv4, with the same inference speed.

* English Recognition Model

Model	Model Download Link	Recognition Avg Accuracy(%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
en_PP-OCRv4_mobile_rec	Inference Model/Training Model	70.39	4.81 / 0.75	16.10 / 5.31	6.8 M	The ultra-lightweight English recognition model trained based on the PP-OCRv4 recognition model supports the recognition of English and numbers.
en_PP-OCRv3_mobile_rec	Inference Model/Training Model	70.69	5.44 / 0.75	8.65 / 5.57	7.8 M	The ultra-lightweight English recognition model trained based on the PP-OCRv3 recognition model supports the recognition of English and numbers.

* Multilingual Recognition Model

Model	Model Download Link	Recognition Avg Accuracy(%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
korean_PP-OCRv3_mobile_rec	Inference Model/Training Model	60.21	5.40 / 0.97	9.11 / 4.05	8.6 M	The ultra-lightweight Korean recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Korean and numbers.
japan_PP-OCRv3_mobile_rec	Inference Model/Training Model	45.69	5.70 / 1.02	8.48 / 4.07	8.8 M	The ultra-lightweight Japanese recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Japanese and numbers.
chinese_cht_PP-OCRv3_mobile_rec	Inference Model/Training Model	82.06	5.90 / 1.28	9.28 / 4.34	9.7 M	The ultra-lightweight Traditional Chinese recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Traditional Chinese and numbers.
te_PP-OCRv3_mobile_rec	Inference Model/Training Model	95.88	5.42 / 0.82	8.10 / 6.91	7.8 M	The ultra-lightweight Telugu recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Telugu and numbers.
ka_PP-OCRv3_mobile_rec	Inference Model/Training Model	96.96	5.25 / 0.79	9.09 / 3.86	8.0 M	The ultra-lightweight Kannada recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Kannada and numbers.
ta_PP-OCRv3_mobile_rec	Inference Model/Training Model	76.83	5.23 / 0.75	10.13 / 4.30	8.0 M	The ultra-lightweight Tamil recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Tamil and numbers.
latin_PP-OCRv3_mobile_rec	Inference Model/Training Model	76.93	5.20 / 0.79	8.83 / 7.15	7.8 M	The ultra-lightweight Latin recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Latin script and numbers.
arabic_PP-OCRv3_mobile_rec	Inference Model/Training Model	73.55	5.35 / 0.79	8.80 / 4.56	7.8 M	The ultra-lightweight Arabic script recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Arabic script and numbers.
cyrillic_PP-OCRv3_mobile_rec	Inference Model/Training Model	94.28	5.23 / 0.76	8.89 / 3.88	7.9 M	The ultra-lightweight cyrillic alphabet recognition model trained based on the PP-OCRv3 recognition model supports the recognition of cyrillic letters and numbers.
devanagari_PP-OCRv3_mobile_rec	Inference Model/Training Model	96.44	5.22 / 0.79	8.56 / 4.06	7.9 M	The ultra-lightweight Devanagari script recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Devanagari script and numbers.

Test Environment Description:

• Performance Test Environment
• Test Dataset
  • Document Image Orientation Classification Model: A dataset built by PaddleX, covering multiple scenarios such as certificates and documents, containing 1,000 images.
  • Text Image Rectification Model: DocUNet.
  • Layout Region Detection Model: A layout region analysis dataset built by PaddleOCR, containing 10,000 images of common document types such as Chinese and English papers, magazines, and research reports.
  • Table Structure Recognition Model: An internal English table recognition dataset built by PaddleX.
  • Text Detection Model: A Chinese dataset built by PaddleOCR, covering multiple scenarios such as street scenes, web images, documents, and handwriting, with 500 images for detection.
  • Chinese Recognition Model: A Chinese dataset built by PaddleOCR, covering multiple scenarios such as street scenes, web images, documents, and handwriting, with 11,000 images for text recognition.
  • ch_SVTRv2_rec: PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task A-list evaluation set.
  • ch_RepSVTR_rec: PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task B-list evaluation set.
  • English Recognition Model: An English dataset built by PaddleX.
  • Multilingual Recognition Model: A multilingual dataset built by PaddleX.

• Hardware Configuration:

  • GPU: NVIDIA Tesla T4
  • CPU: Intel Xeon Gold 6271C @ 2.60GHz
  • Other Environment: Ubuntu 20.04 / cuDNN 8.6 / TensorRT 8.5.2.2

• Inference Mode Description

Mode	GPU Configuration	CPU Configuration	Acceleration Technology Combination
Normal Mode	FP32 Precision / No TRT Acceleration	FP32 Precision / 8 Threads	PaddleInference
High-Performance Mode	Optimal combination of pre-selected precision types and acceleration strategies	FP32 Precision / 8 Threads	Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)

2. Quick Start

The pre-trained models provided by PaddleX can be quickly experienced. You can experience the effect of the general table recognition production line online on the Galaxy Community, or you can experience it locally using the command line or Python.

2.1 Online Experience

You can experience online the effect of the general table recognition production line using the official demo images, for example:

If you are satisfied with the performance of the production line, you can directly integrate and deploy it. You can choose to download the deployment package from the cloud, or refer to the method in Section 2.2 Local Experience for local deployment. If you are not satisfied with the effect, you can fine-tune the models in the production line using private data. If you have local training hardware resources, you can start training directly on your local machine; if not, the Galaxy Zero-Code platform provides a one-click training service. No code is required; simply upload your data to start the training task.

2.2 Local Experience

Before using the general table recognition production line locally, please ensure that you have completed the installation of the PaddleX wheel package according to the PaddleX Local Installation Guide.

2.1 Command Line Experience

You can quickly experience the table recognition production line with a single command. Use the test file, and replace --input with your local path for prediction.

paddlex --pipeline table_recognition \
        --input table_recognition.jpg \
        --save_path ./output \
        --device gpu:0

The content of the parameters can refer to the parameter description in 2.2 Python Script Method.

After running, the result will be printed to the terminal, as follows:

{'res': {'input_path': 'table_recognition.jpg', 'model_settings': {'use_doc_preprocessor': True, 'use_layout_detection': True, 'use_ocr_model': True}, 'doc_preprocessor_res': {'input_path': '0.jpg', 'model_settings': {'use_doc_orientation_classify': True, 'use_doc_unwarping': True}, 'angle': 0}, 'layout_det_res': {'input_path': None, 'boxes': [{'cls_id': 0, 'label': 'Table', 'score': 0.9196816086769104, 'coordinate': [0, 8.614925, 550.9877, 132]}]}, 'overall_ocr_res': {'input_path': '0.jpg', 'model_settings': {'use_doc_preprocessor': False, 'use_textline_orientation': False}, 'dt_polys': [array([[232,   0],
       [318,   1],
       [318,  24],
       [232,  21]], dtype=int16), array([[32, 38],
       [67, 38],
       [67, 55],
       [32, 55]], dtype=int16), array([[119,  34],
       [196,  34],
       [196,  57],
       [119,  57]], dtype=int16), array([[222,  29],
       [396,  31],
       [396,  60],
       [222,  58]], dtype=int16), array([[420,  30],
       [542,  32],
       [542,  61],
       [419,  59]], dtype=int16), array([[29, 71],
       [72, 71],
       [72, 92],
       [29, 92]], dtype=int16), array([[287,  72],
       [329,  72],
       [329,  93],
       [287,  93]], dtype=int16), array([[458,  68],
       [501,  71],
       [499,  94],
       [456,  91]], dtype=int16), array([[  9, 101],
       [ 89, 103],
       [ 89, 130],
       [  8, 128]], dtype=int16), array([[139, 105],
       [172, 105],
       [172, 126],
       [139, 126]], dtype=int16), array([[274, 103],
       [339, 101],
       [340, 128],
       [275, 130]], dtype=int16), array([[451, 103],
       [508, 103],
       [508, 126],
       [451, 126]], dtype=int16)], 'text_det_params': {'limit_side_len': 960, 'limit_type': 'max', 'thresh': 0.3, 'box_thresh': 0.6, 'unclip_ratio': 2.0}, 'text_type': 'general', 'textline_orientation_angles': [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1], 'text_rec_score_thresh': 0, 'rec_texts': ['CRuncover', 'Dres', '连续工作3', '取出来放在网上，没想', '江、江等八大', 'Abstr', 'rSrivi', '$709.', 'cludingGiv', '2.72', 'Ingcubic', '$744.78'], 'rec_scores': [0.9943075180053711, 0.9951075315475464, 0.9907732009887695, 0.9975494146347046, 0.9974043369293213, 0.9983242750167847, 0.991967499256134, 0.9898287653923035, 0.9961177110671997, 0.9975040555000305, 0.9986456632614136, 0.9987970590591431], 'rec_polys': [array([[232,   0],
       [318,   1],
       [318,  24],
       [232,  21]], dtype=int16), array([[32, 38],
       [67, 38],
       [67, 55],
       [32, 55]], dtype=int16), array([[119,  34],
       [196,  34],
       [196,  57],
       [119,  57]], dtype=int16), array([[222,  29],
       [396,  31],
       [396,  60],
       [222,  58]], dtype=int16), array([[420,  30],
       [542,  32],
       [542,  61],
       [419,  59]], dtype=int16), array([[29, 71],
       [72, 71],
       [72, 92],
       [29, 92]], dtype=int16), array([[287,  72],
       [329,  72],
       [329,  93],
       [287,  93]], dtype=int16), array([[458,  68],
       [501,  71],
       [499,  94],
       [456,  91]], dtype=int16), array([[  9, 101],
       [ 89, 103],
       [ 89, 130],
       [  8, 128]], dtype=int16), array([[139, 105],
       [172, 105],
       [172, 126],
       [139, 126]], dtype=int16), array([[274, 103],
       [339, 101],
       [340, 128],
       [275, 130]], dtype=int16), array([[451, 103],
       [508, 103],
       [508, 126],
       [451, 126]], dtype=int16)], 'rec_boxes': array([[232,   0, 318,  24],
       [ 32,  38,  67,  55],
       [119,  34, 196,  57],
       [222,  29, 396,  60],
       [419,  30, 542,  61],
       [ 29,  71,  72,  92],
       [287,  72, 329,  93],
       [456,  68, 501,  94],
       [  8, 101,  89, 130],
       [139, 105, 172, 126],
       [274, 101, 340, 130],
       [451, 103, 508, 126]], dtype=int16)}, 'table_res_list': [{'cell_box_list': array([[  8.        ,   9.61492538, 532.        ,  26.61492538],
       [  3.        ,  27.61492538, 104.        ,  65.61492538],
       [109.        ,  28.61492538, 215.        ,  66.61492538],
       [219.        ,  28.61492538, 396.        ,  64.61492538],
       [396.        ,  29.61492538, 546.        ,  66.61492538],
       [  1.        ,  65.61492538, 110.        ,  93.61492538],
       [111.        ,  65.61492538, 215.        ,  94.61492538],
       [220.        ,  66.61492538, 397.        ,  94.61492538],
       [398.        ,  67.61492538, 544.        ,  94.61492538],
       [  2.        ,  98.61492538, 111.        , 131.61492538],
       [113.        ,  98.61492538, 216.        , 131.61492538],
       [219.        ,  98.61492538, 400.        , 131.61492538],
       [403.        ,  99.61492538, 545.        , 130.61492538]]), 'pred_html': '<html><body><table><tr><td colspan="4">CRuncover</td></tr><tr><td>Dres</td><td>连续工作3</td><td>取出来放在网上，没想</td><td>江、江等八大</td></tr><tr><td>Abstr</td><td></td><td>rSrivi</td><td>$709.</td></tr><tr><td>cludingGiv</td><td>2.72</td><td>Ingcubic</td><td>$744.78</td></tr></table></body></html>', 'table_ocr_pred': {'rec_polys': [array([[232,   0],
       [318,   1],
       [318,  24],
       [232,  21]], dtype=int16), array([[32, 38],
       [67, 38],
       [67, 55],
       [32, 55]], dtype=int16), array([[119,  34],
       [196,  34],
       [196,  57],
       [119,  57]], dtype=int16), array([[222,  29],
       [396,  31],
       [396,  60],
       [222,  58]], dtype=int16), array([[420,  30],
       [542,  32],
       [542,  61],
       [419,  59]], dtype=int16), array([[29, 71],
       [72, 71],
       [72, 92],
       [29, 92]], dtype=int16), array([[287,  72],
       [329,  72],
       [329,  93],
       [287,  93]], dtype=int16), array([[458,  68],
       [501,  71],
       [499,  94],
       [456,  91]], dtype=int16), array([[  9, 101],
       [ 89, 103],
       [ 89, 130],
       [  8, 128]], dtype=int16), array([[139, 105],
       [172, 105],
       [172, 126],
       [139, 126]], dtype=int16), array([[274, 103],
       [339, 101],
       [340, 128],
       [275, 130]], dtype=int16), array([[451, 103],
       [508, 103],
       [508, 126],
       [451, 126]], dtype=int16)], 'rec_texts': ['CRuncover', 'Dres', '连续工作3', '取出来放在网上，没想', '江、江等八大', 'Abstr', 'rSrivi', '$709.', 'cludingGiv', '2.72', 'Ingcubic', '$744.78'], 'rec_scores': [0.9943075180053711, 0.9951075315475464, 0.9907732009887695, 0.9975494146347046, 0.9974043369293213, 0.9983242750167847, 0.991967499256134, 0.9898287653923035, 0.9961177110671997, 0.9975040555000305, 0.9986456632614136, 0.9987970590591431], 'rec_boxes': [array([232,   0, 318,  24], dtype=int16), array([32, 38, 67, 55], dtype=int16), array([119,  34, 196,  57], dtype=int16), array([222,  29, 396,  60], dtype=int16), array([419,  30, 542,  61], dtype=int16), array([29, 71, 72, 92], dtype=int16), array([287,  72, 329,  93], dtype=int16), array([456,  68, 501,  94], dtype=int16), array([  8, 101,  89, 130], dtype=int16), array([139, 105, 172, 126], dtype=int16), array([274, 101, 340, 130], dtype=int16), array([451, 103, 508, 126], dtype=int16)]}}]}}

The running result parameters can be referred to the result explanation in 2.2 Python Script Integration.

The visualization results are saved under save_path, where the visualization result of table recognition is as follows:

2.2 Python Script Integration

• The above command line is for a quick experience to view the results. Generally, in a project, it is often necessary to integrate through code. You can complete the fast inference of the production line with just a few lines of code. The inference code is as follows:

from paddlex import create_pipeline

pipeline = create_pipeline(pipeline="table_recognition")

output = pipeline.predict(
    input="table_recognition.jpg",
    use_doc_orientation_classify=False,
    use_doc_unwarping=False,
)

for res in output:
    res.print()
    res.save_to_img("./output/")
    res.save_to_xlsx("./output/")
    res.save_to_html("./output/")
    res.save_to_json("./output/")

In the above Python script, the following steps are executed:

(1) The create_pipeline() function is used to instantiate a general table recognition pipeline object, with the specific parameters described as follows:

Parameter	Parameter Description	Parameter Type	Default Value
pipeline	The name of the pipeline or the path to the pipeline configuration file. If it is a pipeline name, it must be supported by PaddleX.	str	None
config	The specific configuration information of the pipeline (if set simultaneously with pipeline, it has a higher priority than pipeline, and the pipeline name must be consistent with pipeline).	dict[str, Any]	None
device	The inference device for the pipeline. It supports specifying the specific GPU card number, such as "gpu:0", other hardware card numbers, such as "npu:0", and CPU as "cpu".	str	gpu:0
use_hpip	Whether to enable high-performance inference, which is only available if the pipeline supports high-performance inference.	bool	False

(2) The predict() method of the general table recognition pipeline object is called to perform inference prediction. This method returns a generator. Below are the parameters and their descriptions for the predict() method:

Parameter	Parameter Description	Parameter Type	Options	Default Value
input	Data to be predicted, supporting multiple input types (required).	Python Var|str|list	Python Var: Image data represented by numpy.ndarray. str: Local path of an image file or a PDF file, such as /root/data/img.jpg; URL link, such as the network URL of an image or PDF file: example; local directory, which must contain images to be predicted, such as the local path /root/data/ (currently, prediction of PDF files in a directory is not supported; PDF files must be specified with an exact file path). List: The elements of the list must be of the above types, such as [numpy.ndarray, numpy.ndarray], [\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"], or [\"/root/data1\", \"/root/data2\"].	None
device	The inference device for the pipeline.	str|None	CPU: cpu indicates using the CPU for inference; GPU: gpu:0 indicates using the first GPU for inference; NPU: npu:0 indicates using the first NPU for inference; XPU: xpu:0 indicates using the first XPU for inference; MLU: mlu:0 indicates using the first MLU for inference; DCU: dcu:0 indicates using the first DCU for inference; None: If set to None, the default value from the pipeline initialization will be used. During initialization, the local GPU 0 will be prioritized; if unavailable, the CPU will be used.	None
use_doc_orientation_classify	Whether to use the document orientation classification module.	bool|None	bool: True or False; None: If set to None, the default value from the pipeline initialization will be used, initialized as True.	None
use_doc_unwarping	Whether to use the document unwarping module	bool|None	bool: True or False; None: If set to None, the default value from the pipeline initialization will be used, initialized as True.	None
text_det_limit_side_len	Image side length limit for text detection	int|None	int: Any integer greater than 0; None: If set to None, the default value from the pipeline initialization will be used, initialized as 960.	None	text_det_limit_type	Type of image side length limit for text detection	str|None	str: Supports min and max. min ensures that the shortest side of the image is not less than det_limit_side_len, and max ensures that the longest side of the image is not greater than limit_side_len. None: If set to None, the default value from the pipeline initialization will be used, initialized as max.	None	text_det_thresh	Pixel detection threshold; pixels with scores greater than this threshold in the output probability map will be considered as text pixels	float|None	float: Any floating-point number greater than 0. None: If set to None, the default value from the pipeline initialization will be used, initialized as 0.3.	None	text_det_box_thresh	Detection box threshold; the detection result will be considered as a text region if the average score of all pixels within the detection box is greater than this threshold	float|None	float: Any floating-point number greater than 0. None: If set to None, the default value from the pipeline initialization will be used, initialized as 0.6.	None	text_det_unclip_ratio	Expansion ratio for text detection; the larger this value, the larger the expanded area of the text region	float|None	float: Any floating-point number greater than 0. None: If set to None, the default value from the pipeline initialization will be used, initialized as 2.0.	None	text_rec_score_thresh	Text recognition threshold; text results with scores greater than this threshold will be retained	float|None	float: Any floating-point number greater than 0. None: If set to None, the default value from the pipeline initialization will be used, initialized as 0.0 (i.e., no threshold).	None

(3) Process the prediction results. Each sample's prediction result is a corresponding Result object, and it supports operations such as printing, saving as an image, saving as an xlsx file, saving as an HTML file, and saving as a json file:

Method	Method Description	Parameter	Parameter Type	Parameter Description	Default Value
print()	Print the result to the terminal	format_json	bool	Whether to format the output content using JSON indentation	True
		indent	int	Specify the indentation level to beautify the output JSON data for better readability, effective only when format_json is True	4
		ensure_ascii	bool	Control whether to escape non-ASCII characters to Unicode. If set to True, all non-ASCII characters will be escaped; False retains the original characters, effective only when format_json is True	False
save_to_json()	Save the result as a JSON file	save_path	str	The file path for saving. When it is a directory, the saved file name is consistent with the input file type	None
		indent	int	Specify the indentation level to beautify the output JSON data for better readability, effective only when format_json is True	4
		ensure_ascii	bool	Control whether to escape non-ASCII characters to Unicode. If set to True, all non-ASCII characters will be escaped; False retains the original characters, effective only when format_json is True	False
save_to_img()	Save the result as an image file	save_path	str	The file path for saving, supporting both directory and file paths	None
save_to_xlsx()	Save the result as an xlsx file	save_path	str	The file path for saving, supporting both directory and file paths	None
save_to_html()	Save the result as an HTML file	save_path	str	The file path for saving, supporting both directory and file paths	None

• Calling the print() method will print the result to the terminal, and the content printed to the terminal is explained as follows:

  • input_path: (str) The input path of the image to be predicted

  • page_index: (Union[int, None]) If the input is a PDF file, it indicates which page of the PDF is currently being processed; otherwise, it is None

  • model_settings: (Dict[str, bool]) Model parameters required for the production line configuration

    • use_doc_preprocessor: (bool) Controls whether to enable the document preprocessor sub-line
      • doc_preprocessor_res: (Dict[str, Union[str, Dict[str, bool], int]]) Output result of the document preprocessor sub-line. It exists only when use_doc_preprocessor=True
    • input_path: (Union[str, None]) The image path accepted by the image preprocessor sub-line. When the input is numpy.ndarray, it is saved as None
    • model_settings: (Dict) Model configuration parameters for the preprocessor sub-line
      • use_doc_orientation_classify: (bool) Controls whether to enable document orientation classification
      • use_doc_unwarping: (bool) Controls whether to enable document warping correction
    • angle: (int) The prediction result of document orientation classification. When enabled, the values are [0,1,2,3], corresponding to [0°,90°,180°,270°]; when not enabled, it is -1

  • dt_polys: (List[numpy.ndarray]) A list of polygon boxes for text detection. Each detection box is represented by a numpy array of four vertex coordinates, with a shape of (4, 2) and a data type of int16

  • dt_scores: (List[float]) A list of confidence scores for text detection boxes

  • text_det_params: (Dict[str, Dict[str, int, float]]) Configuration parameters for the text detection module

    • limit_side_len: (int) The side length limit value during image preprocessing
    • limit_type: (str) The processing method for the side length limit
    • thresh: (float) The confidence threshold for text pixel classification
    • box_thresh: (float) The confidence threshold for text detection boxes
    • unclip_ratio: (float) The expansion ratio for text detection boxes
    • text_type: (str) The type of text detection, currently fixed as "general"

  • text_rec_score_thresh: (float) The filtering threshold for text recognition results

  • rec_texts: (List[str]) A list of text recognition results, containing only texts with confidence scores exceeding text_rec_score_thresh

  • rec_scores: (List[float]) A list of confidence scores for text recognition, filtered by text_rec_score_thresh

  • rec_polys: (List[numpy.ndarray]) A list of text detection boxes after confidence filtering, in the same format as dt_polys

  • rec_boxes: (numpy.ndarray) An array of rectangular bounding boxes for detection boxes, with a shape of (n, 4) and a dtype of int16. Each row represents the coordinates [x_min, y_min, x_max, y_max] of a rectangular box, where (x_min, y_min) is the top-left corner and (x_max, y_max) is the bottom-right corner

• Calling the save_to_json() method will save the above content to the specified save_path. If it is a directory, the saved path will be save_path/{your_img_basename}.json; if it is a file, it will be saved directly to that file. Since JSON files do not support saving numpy arrays, the numpy.array type will be converted to a list format.

• Calling the save_to_img() method will save the visualization results to the specified save_path. If it is a directory, the saved path will be save_path/{your_img_basename}_ocr_res_img.{your_img_extension}; if it is a file, it will be saved directly to that file. (The production line usually contains many result images, so it is not recommended to specify a specific file path directly, otherwise multiple images will be overwritten and only the last one will be retained)
• Calling the save_to_html() method will save the above content to the specified save_path. If it is a directory, the saved path will be save_path/{your_img_basename}.html; if it is a file, it will be saved directly to that file. In the general table recognition production line, the HTML form of the table in the image will be written to the specified HTML file.

• Calling the save_to_xlsx() method will save the above content to the specified save_path. If it is a directory, the saved path will be save_path/{your_img_basename}.xlsx; if it is a file, it will be saved directly to that file. In the general table recognition production line, the Excel table form of the table in the image will be written to the specified xlsx file.

• In addition, it also supports obtaining visualized images with results and prediction results through attributes, as follows:

Attribute	Attribute Description
json	Get the prediction result in json format
img	Get the visualized image in dict format

• The prediction result obtained through the json attribute is data of the dict type, and its content is consistent with the content saved by calling the save_to_json() method.
• The prediction result returned by the img attribute is data of the dictionary type. The keys are table_res_img, ocr_res_img, layout_res_img, and preprocessed_img, and the corresponding values are four Image.Image objects, in order: the visualized image of table recognition results, the visualized image of OCR results, the visualized image of layout area detection results, and the visualized image of image preprocessing. If a sub-module is not used, the corresponding result image will not be included in the dictionary.

In addition, you can obtain the configuration file for the general table recognition production line and load the configuration file for prediction. You can execute the following command to save the result in my_path:

paddlex --get_pipeline_config table_recognition --save_path ./my_path

If you have obtained the configuration file, you can customize the settings for the general table recognition production line. You just need to modify the value of the pipeline parameter in the create_pipeline method to the path of the production line configuration file. The example is as follows:

from paddlex import create_pipeline

pipeline = create_pipeline(pipeline="./my_path/table_recognition.yaml")

output = pipeline.predict(
    input="table_recognition.jpg",
    use_doc_orientation_classify=False,
    use_doc_unwarping=False,
)

for res in output:
    res.print()
    res.save_to_img("./output/")
    res.save_to_xlsx("./output/")
    res.save_to_html("./output/")
    res.save_to_json("./output/")

Note: The parameters in the configuration file are the initialization parameters for the production line. If you want to change the general table recognition production line initialization parameters, you can directly modify the parameters in the configuration file and load the configuration file for prediction. At the same time, CLI prediction also supports passing in a configuration file, and you can specify the path of the configuration file with --pipeline.

3. Development Integration/Deployment

If the production line meets your requirements for inference speed and accuracy, you can directly proceed with development integration/deployment.

If you need to apply the production line directly in your Python project, you can refer to the example code in 2.2 Python Script Method.

In addition, PaddleX also provides three other deployment methods, which are detailed as follows:

🚀 High-Performance Inference: In actual production environments, many applications have strict performance requirements (especially response speed) for deployment strategies to ensure efficient system operation and smooth user experience. To this end, PaddleX provides a high-performance inference plugin, which aims to deeply optimize the performance of model inference and pre/post-processing to significantly speed up the end-to-end process. For detailed high-performance inference procedures, please refer to the PaddleX High-Performance Inference Guide.

☁️ Service-Oriented Deployment: Service-oriented deployment is a common form of deployment in actual production environments. By encapsulating inference functions as services, clients can access these services through network requests to obtain inference results. PaddleX supports multiple production line service-oriented deployment solutions. For detailed production line service-oriented deployment procedures, please refer to the PaddleX Service-Oriented Deployment Guide.

Below are the API references for basic service-oriented deployment and multi-language service invocation examples:

API Reference

For the main operations provided by the service:

• The HTTP request method is POST.
• Both the request body and response body are JSON data (JSON objects).
• When the request is processed successfully, the response status code is 200, and the attributes of the response body are as follows:

Name	Type	Meaning
logId	string	The UUID of the request.
errorCode	integer	Error code. Fixed as 0.
errorMsg	string	Error message. Fixed as "Success".
result	object	The result of the operation.

• When the request is not processed successfully, the attributes of the response body are as follows:

Name	Type	Meaning
logId	string	The UUID of the request.
errorCode	integer	Error code. Same as the response status code.
errorMsg	string	Error message.

The main operations provided by the service are as follows:

• infer

Locate and recognize tables in the image.

POST /table-recognition

• The properties of the request body are as follows:

Name	Type	Description	Required
file	string	The URL of an image or PDF file accessible by the server, or the Base64-encoded content of the above file types. For PDF files exceeding 10 pages, only the content of the first 10 pages will be used.	Yes
fileType	integer | null	The file type. 0 indicates a PDF file, and 1 indicates an image file. If this attribute is missing in the request body, the file type will be inferred based on the URL.	No
useDocOrientationClassify	boolean | null	See the description of the use_doc_orientation_classify parameter in the production predict method.	No
useDocUnwarping	boolean | null	See the description of the use_doc_unwarping parameter in the production predict method.	No
useLayoutDetection	boolean | null	See the description of the use_layout_detection parameter in the production predict method.	No
useOcrModel	boolean | null	See the description of the use_ocr_model parameter in the production predict method.	No
layoutThreshold	number | null	See the description of the layout_threshold parameter in the production predict method.	No
layoutNms	boolean | null	See the description of the layout_nms parameter in the production predict method.	No
layoutUnclipRatio	number | array | null	See the description of the layout_unclip_ratio parameter in the production predict method.	No
layoutMergeBboxesMode	string | null	See the description of the layout_merge_bboxes_mode parameter in the production predict method.	No
textDetLimitSideLen	integer | null	See the description of the text_det_limit_side_len parameter in the production predict method.	No
textDetLimitType	string | null	See the description of the text_det_limit_type parameter in the production predict method.	No
textDetThresh	number | null	See the description of the text_det_thresh parameter in the production predict method.	No
textDetBoxThresh	number | null	See the description of the text_det_box_thresh parameter in the production predict method.	No
textDetUnclipRatio	number | null	See the description of the text_det_unclip_ratio parameter in the production predict method.	No
textRecScoreThresh	number | null	See the description of the text_rec_score_thresh parameter in the production predict method.	No

• When the request is processed successfully, the result in the response body has the following properties:

Name	Type	Meaning
tableRecResults	object	The table recognition results. The array length is 1 (for image input) or the smaller of the number of document pages and 10 (for PDF input). For PDF input, each element in the array represents the processing result of each page in the PDF file.
dataInfo	object	Information about the input data.

Each element in tableRecResults is an object with the following properties:

Name	Type	Meaning
prunedResult	object	A simplified version of the res field in the JSON representation of the result generated by the predict method of the production object, with the input_path field removed.
outputImages	object | null	Refer to the description of the img attribute in the production prediction results. The image is in JPEG format and encoded in Base64.
inputImage	string | null	The input image. The image is in JPEG format and encoded in Base64.

Multi-language Service Call Example

Python

import base64
import requests

API_URL = "http://localhost:8080/table-recognition"
file_path = "./demo.jpg"

with open(file_path, "rb") as file:
    file_bytes = file.read()
    file_data = base64.b64encode(file_bytes).decode("ascii")

payload = {"file": file_data, "fileType": 1}

response = requests.post(API_URL, json=payload)

assert response.status_code == 200
result = response.json()["result"]
for i, res in enumerate(result["tableRecResults"]):
    print(res["prunedResult"])
    for img_name, img in res["outputImages"].items():
        img_path = f"{img_name}_{i}.jpg"
        with open(img_path, "wb") as f:
            f.write(base64.b64decode(img))
        print(f"Output image saved at {img_path}")

📱 Edge Deployment: Edge deployment is a method of placing computing and data processing capabilities on the user's device itself, allowing the device to process data directly without relying on remote servers. PaddleX supports deploying models on edge devices such as Android. For detailed edge deployment procedures, please refer to the PaddleX Edge Deployment Guide. You can choose the appropriate deployment method for your model production line according to your needs, and then proceed with the subsequent AI application integration.

4. Secondary Development

If the default model weights provided by the general table recognition production line are not satisfactory in terms of accuracy or speed in your scenario, you can try to further fine-tune the existing model using your own specific domain or application scenario data to improve the recognition performance of the general table recognition production line in your scenario.

4.1 Model Fine-tuning

Since the general table recognition production line consists of several modules, if the performance of the model production line does not meet expectations, the issue may originate from any one of these modules. You can analyze the images with poor recognition results to determine which module is problematic, and then refer to the corresponding fine-tuning tutorial links in the table below for model fine-tuning.

Scenario	Fine-tuning Module	Fine-tuning Reference Link
Table structure recognition error or cell positioning error	Table Structure Recognition Module	Link
Failed to detect the table area	Layout Area Detection Module	Link
Text detection omission	Text Detection Module	Link
Text content is inaccurate	Text Recognition Module	Link
Whole image rotation correction is inaccurate	Document Image Orientation Classification Module	Link
Image distortion correction is inaccurate	Text Image Correction Module	Fine-tuning not supported yet

4.2 Model Application

After you complete fine-tuning with your private dataset, you will obtain a local model weight file.

If you need to use the fine-tuned model weights, simply modify the production configuration file by replacing the local path of the fine-tuned model weights to the corresponding position in the production configuration file.

SubModules:
  LayoutDetection:
    module_name: layout_detection
    model_name: PicoDet_layout_1x_table
    model_dir: null # 替换为微调后的版面区域检测模型权重路径

  TableStructureRecognition:
    module_name: table_structure_recognition
    model_name: SLANet_plus
    model_dir: null # 替换为微调后的表格结构识别模型权重路径

SubPipelines:
  DocPreprocessor:
    pipeline_name: doc_preprocessor
    use_doc_orientation_classify: True
    use_doc_unwarping: True
    SubModules:
      DocOrientationClassify:
        module_name: doc_text_orientation
        model_name: PP-LCNet_x1_0_doc_ori
        model_dir: null # 替换为微调后的文档图像方向分类模型权重路径

      DocUnwarping:
        module_name: image_unwarping
        model_name: UVDoc
        model_dir: null

  GeneralOCR:
    pipeline_name: OCR
    text_type: general
    use_doc_preprocessor: False
    use_textline_orientation: False
    SubModules:
      TextDetection:
        module_name: text_detection
        model_name: PP-OCRv4_server_det
        model_dir: null # 替换为微调后的文本检测模型权重路径
        limit_side_len: 960
        limit_type: max
        thresh: 0.3
        box_thresh: 0.6
        unclip_ratio: 2.0
      TextRecognition:
        module_name: text_recognition
        model_name: PP-OCRv4_server_rec
        model_dir: null # 替换为微调后文本识别的模型权重路径
        batch_size: 1
        score_thresh: 0

Subsequently, refer to the command line method or Python script method in 2.2 Local Experience to load the modified production configuration file.

5. Multi-Hardware Support

PaddleX supports a variety of mainstream hardware devices such as NVIDIA GPU, Kunlunxin XPU, Ascend NPU, and Cambricon MLU. Simply modify the --device parameter to seamlessly switch between different hardware.

For example, if you use Ascend NPU for OCR production inference, the CLI command is:

paddlex --pipeline table_recognition \
        --input table_recognition.jpg \
        --save_path ./output \
        --device npu:0

Of course, you can also specify the hardware device when calling create_pipeline() or predict() in the Python script.

If you want to use the universal table recognition pipeline on a wider range of hardware, please refer to the PaddleX Multi-Device Usage Guide.

Comments