Layout Detection Module Tutorial¶

I. Overview¶

The core task of structure analysis is to parse and segment the content of input document images. By identifying different elements in the image (such as text, charts, images, etc.), they are classified into predefined categories (e.g., pure text area, title area, table area, image area, list area, etc.), and the position and size of these regions in the document are determined.

II. Supported Model List¶

The layout detection model includes 20 common categories: document title, paragraph title, text, page number, abstract, table, references, footnotes, header, footer, algorithm, formula, formula number, image, table, seal, figure_table title, chart, and sidebar text and lists of references

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-DocLayout_plus-L	Inference Model/Training Model	83.2	34.6244 / 10.3945	510.57 / -	126.01 M	A higher-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L

The layout detection model includes 1 category: Block:

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-DocBlockLayout	Inference Model/Training Model	95.9	34.6244 / 10.3945	510.57 / -	123.92 M	A layout block localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L

The layout detection model includes 23 common categories: document title, paragraph title, text, page number, abstract, table of contents, references, footnotes, header, footer, algorithm, formula, formula number, image, figure caption, table, table caption, seal, figure title, figure, header image, footer image, and sidebar text

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PP-DocLayout-L	Inference Model/Training Model	90.4	34.6244 / 10.3945	510.57 / -	123.76 M	A high-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using RT-DETR-L.
PP-DocLayout-M	Inference Model/Training Model	75.2	13.3259 / 4.8685	44.0680 / 44.0680	22.578	A layout area localization model with balanced precision and efficiency, trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-L.
PP-DocLayout-S	Inference Model/Training Model	70.9	8.3008 / 2.3794	10.0623 / 9.9296	4.834	A high-efficiency layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-S.

❗ The above list includes the 4 core models that are key supported by the text recognition module. The module actually supports a total of 12 full models, including several predefined models with different categories. The complete model list is as follows:

👉 Details of Model List

* Table Layout Detection Model

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PicoDet_layout_1x_table	Inference Model/Training Model	97.5	8.02 / 3.09	23.70 / 20.41	7.4 M	A high-efficiency layout area localization model trained on a self-built dataset using PicoDet-1x, capable of detecting table regions.

* 3-Class Layout Detection Model, including Table, Image, and Stamp

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PicoDet-S_layout_3cls	Inference Model/Training Model	88.2	8.99 / 2.22	16.11 / 8.73	4.8	A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.
PicoDet-L_layout_3cls	Inference Model/Training Model	89.0	13.05 / 4.50	41.30 / 41.30	22.6	A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.
RT-DETR-H_layout_3cls	Inference Model/Training Model	95.8	114.93 / 27.71	947.56 / 947.56	470.1	A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.

* 5-Class English Document Area Detection Model, including Text, Title, Table, Image, and List

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PicoDet_layout_1x	Inference Model/Training Model	97.8	9.03 / 3.10	25.82 / 20.70	7.4	A high-efficiency English document layout area localization model trained on the PubLayNet dataset using PicoDet-1x.

* 17-Class Area Detection Model, including 17 common layout categories: Paragraph Title, Image, Text, Number, Abstract, Content, Figure Caption, Formula, Table, Table Caption, References, Document Title, Footnote, Header, Algorithm, Footer, and Stamp

Model	Model Download Link	mAP(0.5) (%)	GPU Inference Time (ms) [Normal Mode / High-Performance Mode]	CPU Inference Time (ms) [Normal Mode / High-Performance Mode]	Model Storage Size (M)	Introduction
PicoDet-S_layout_17cls	Inference Model/Training Model	87.4	9.11 / 2.12	15.42 / 9.12	4.8	A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.
PicoDet-L_layout_17cls	Inference Model/Training Model	89.0	13.50 / 4.69	43.32 / 43.32	22.6	A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.
RT-DETR-H_layout_17cls	Inference Model/Training Model	98.3	115.29 / 104.09	995.27 / 995.27	470.2	A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.

Test Environment Description:

Performance Test Environment
- Test Dataset：
  - 20 types of layout detection models: PaddleOCR's self built layout area detection dataset, including Chinese and English papers, magazines, newspapers, research papers PPT、 1300 images of document types such as test papers and textbooks.
  - Type 1 version face region detection model: PaddleOCR's self built version face region detection dataset, including Chinese and English papers, magazines, newspapers, research reports PPT、 1000 document type images such as test papers and textbooks.
  - 23 categories Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, containing 500 common document type images such as Chinese and English papers, magazines, contracts, books, exam papers, and research reports.
  - Table Layout Detection Model: A self-built table area detection dataset by PaddleOCR, including 7,835 Chinese and English paper document type images with tables.
  - 3-Class Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, comprising 1,154 common document type images such as Chinese and English papers, magazines, and research reports.
  - 5-Class English Document Area Detection Model: The evaluation dataset of PubLayNet, containing 11,245 images of English documents.
  - 17-Class Area Detection Model: A self-built layout area detection dataset by PaddleOCR, including 892 common document type images such as Chinese and English papers, magazines, and research reports.
- Hardware Configuration：
  - GPU: NVIDIA Tesla T4
  - CPU: Intel Xeon Gold 6271C @ 2.60GHz
  - Other Environments: Ubuntu 20.04 / cuDNN 8.6 / TensorRT 8.5.2.2
Inference Mode Description

Mode	GPU Configuration	CPU Configuration	Acceleration Technology Combination
Normal Mode	FP32 Precision / No TRT Acceleration	FP32 Precision / 8 Threads	PaddleInference
High-Performance Mode	Optimal combination of pre-selected precision types and acceleration strategies	FP32 Precision / 8 Threads	Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)

III. Quick Integration ¶

❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to PaddleOCR Local Installation Tutorial。

Quickly experience with just one command:

paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg

You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download Example Image Go to the local area.

from paddleocr import LayoutDetection

model = LayoutDetection(model_name="PP-DocLayout_plus-L")
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
for res in output:
    res.print()
    res.save_to_img(save_path="./output/")
    res.save_to_json(save_path="./output/res.json")

After running, the result obtained is:

{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}

The meanings of the parameters are as follows: - input_path: The path to the input image for prediction. - page_index: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is None. - boxes: Information about the predicted bounding boxes, a list of dictionaries. Each dictionary represents a detected object and contains the following information: - cls_id: Class ID, an integer. - label: Class label, a string. - score: Confidence score of the bounding box, a float. - coordinate: Coordinates of the bounding box, a list of floats in the format [xmin, ymin, xmax, ymax].

The visualized image is as follows:

Relevant methods, parameters, and explanations are as follows:

LayoutDetection instantiates a target detection model (here, PP-DocLayout_plus-L is used as an example). The detailed explanation is as follows:

Parameter	Description	Type	Options	Default Value
`model_name`	Name of the model	`str`	None	None
`model_dir`	Path to store the model	`str`	None	None
`device`	The device used for model inference	`str`	It supports specifying specific GPU card numbers, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu".	`gpu:0`
`img_size`	Size of the input image; if not specified, the default 800x800 be used	`int/list/None`	int, e.g., 640, means resizing the input image to 640x640 List, e.g., [640, 512], means resizing the input image to a width of 640 and a height of 512 None, not specified, will use the default 0.5	None
`threshold`	Threshold for filtering low-confidence prediction results; if not specified, the default 0.5 will be used	`float/dict/None`	float, e.g., 0.2, means filtering out all bounding boxes with a confidence score less than 0.2 Dictionary, with keys as int representing `cls_id` and values as float thresholds. For example, `{0: 0.45, 2: 0.48, 7: 0.4}` means applying a threshold of 0.45 for cls_id 0, 0.48 for cls_id 2, and 0.4 for cls_id 7 None, not specified, will use the default 0.5	None
`layout_nms`	Whether to use NMS post-processing to filter overlapping boxes; if not specified, the default False will be used	`bool/None`	bool, True/False, indicates whether to use NMS for post-processing to filter overlapping boxes None, not specified, will use the default False	None
`layout_unclip_ratio`	Scaling factor for the side length of the detection box; if not specified, the default 1.0 will be used	`float/list/dict/None`	float, a positive float number, e.g., 1.1, means expanding the width and height of the detection box by 1.1 times while keeping the center unchanged List, e.g., [1.2, 1.5], means expanding the width by 1.2 times and the height by 1.5 times while keeping the center unchanged dict, keys as int representing `cls_id`, values as float scaling factors, e.g., `{0: (1.1, 2.0)}` means cls_id 0 expanding the width by 1.1 times and the height by 2.0 times while keeping the center unchanged None, not specified, will use the default 1.0
`layout_merge_bboxes_mode`	Merging mode for the detection boxes output by the model; if not specified, the default union will be used	`string/dict/None`	large, when set to large, only the largest external box will be retained for overlapping detection boxes, and the internal overlapping boxes will be deleted small, when set to small, only the smallest internal box will be retained for overlapping detection boxes, and the external overlapping boxes will be deleted union, no filtering of boxes will be performed, and both internal and external boxes will be retained dict, keys as int representing `cls_id` and values as merging modes, e.g., `{0: "large", 2: "small"}` `None, not specified, will use the default union`	None
`use_hpip`	Whether to enable the high-performance inference plugin	`bool`	None	`False`
`hpi_config`	High-performance inference configuration	`dict` \| `None`	None	`None`

The predict() method of the target detection model is called for inference prediction. The parameters of the predict() method are input, batch_size, and threshold, which are explained as follows:

Parameter	Description	Type	Options	Default Value
`input`	Data for prediction, supporting multiple input types	`Python Var`/`str`/`list`	Python Variable, such as image data represented by `numpy.ndarray` File Path, such as the local path of an image file: `/root/data/img.jpg` URL link, such as the network URL of an image file: 示例 Local Directory, the directory should contain the data files to be predicted, such as the local path: `/root/data/` List, the elements of the list should be of the above-mentioned data types, such as `[numpy.ndarray, numpy.ndarray]`, `[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"]`, `[\"/root/data1\", \"/root/data2\"]`	None
`batch_size`	Batch size	`int`	Any integer greater than 0	1
`threshold`	Threshold for filtering low-confidence prediction results	`float/dict/None`	float, e.g., 0.2, means filtering out all bounding boxes with a confidence score less than 0.2 Dictionary, with keys as int representing `cls_id` and values as float thresholds. For example, `{0: 0.45, 2: 0.48, 7: 0.4}` means applying a threshold of 0.45 for cls_id 0, 0.48 for cls_id 2, and 0.4 for cls_id 7 None, not specified, will use the `threshold` parameter specified in `create_model`. If not specified in `create_model`, the default 0.5 will be used
`layout_nms`	Whether to use NMS post-processing to filter overlapping boxes; if not specified, the default False will be used	`bool/None`	bool, True/False, indicates whether to use NMS for post-processing to filter overlapping boxes None, not specified, will use the `layout_nms` parameter specified in `create_model`. If not specified in `create_model`, the default False will be used	None
`layout_unclip_ratio`	Scaling factor for the side length of the detection box; if not specified, the default 1.0 will be used	`float/list/dict/None`	float, a positive float number, e.g., 1.1, means expanding the width and height of the detection box by 1.1 times while keeping the center unchanged List, e.g., [1.2, 1.5], means expanding the width by 1.2 times and the height by 1.5 times while keeping the center unchanged dict, keys as int representing `cls_id`, values as float scaling factors, e.g., `{0: (1.1, 2.0)}` means cls_id 0 expanding the width by 1.1 times and the height by 2.0 times while keeping the center unchanged None, not specified, will use the `layout_unclip_ratio` parameter specified in `create_model`. If not specified in `create_model`, the default 1.0 will be used
`layout_merge_bboxes_mode`	Merging mode for the detection boxes output by the model; if not specified, the default union will be used	`string/dict/None`	large, when set to large, only the largest external box will be retained for overlapping detection boxes, and the internal overlapping boxes will be deleted small, when set to small, only the smallest internal box will be retained for overlapping detection boxes, and the external overlapping boxes will be deleted union, no filtering of boxes will be performed, and both internal and external boxes will be retained dict, keys as int representing `cls_id` and values as merging modes, e.g., `{0: "large", 2: "small"}` `None, not specified, will use the layout_merge_bboxes_mode parameter specified in create_model. If not specified in create_model, the default union will be used`	None

Process the prediction results, with each sample's prediction result being the corresponding Result object, and supporting operations such as printing, saving as an image, and saving as a 'json' file:

Method	Method Description	Parameters	Parameter type	Parameter Description	Default value
`print()`	Print the result to the terminal	`format_json`	`bool`	Do you want to use `JSON` indentation formatting for the output content	`True`
		`indent`	`int`	Specify the indentation level to enhance the readability of the `JSON` data output, only valid when `format_json` is `True`	4
		`ensure_ascii`	`bool`	Control whether to escape non ASCII characters to Unicode characters. When set to `True`, all non ASCII characters will be escaped; `False` preserves the original characters and is only valid when `format_json` is `True`	`False`
`save_to_json()`	Save the result as a JSON format file	`save_path`	`str`	The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type	None
		`indent`	`int`	Specify the indentation level to enhance the readability of the `JSON` data output, only valid when `format_json` is `True`	4
		`ensure_ascii`	`bool`	Control whether to escape non ASCII characters to Unicode characters. When set to `True`, all non `ASCII` characters will be escaped; `False` preserves the original characters and is only valid when`format_json` is `True`	`False`
`save_to_img()`	Save the results as an image format file	`save_path`	`str`	The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type	None

Additionally, it also supports obtaining the visualized image with results and the prediction results via attributes, as follows:

Attribute	Description
`json`	Get the prediction result in `json` format
`img`	Get the visualized image in `dict` format

IV. Custom Development¶

Since PaddleOCR does not directly provide training for the layout detection module, if you need to train the layout area detection model, you can refer to PaddleX Layout Detection Module Secondary DevelopmentPartially conduct training. The trained model can be seamlessly integrated into PaddleOCR's API for inference.

Layout Detection Module Tutorial¶

I. Overview¶

II. Supported Model List¶

III. Quick Integration ¶

IV. Custom Development¶

V. FAQ¶

Comments