Skip to content

CPPD

1. Introduction

Paper:

Context Perception Parallel Decoder for Scene Text Recognition Yongkun Du and Zhineng Chen and Caiyan Jia and Xiaoting Yin and Chenxia Li and Yuning Du and Yu-Gang Jiang

Scene text recognition models based on deep learning typically follow an Encoder-Decoder structure, where the decoder can be categorized into two types: (1) CTC and (2) Attention-based. Currently, most state-of-the-art (SOTA) models use an Attention-based decoder, which can be further divided into AR and PD types. In general, AR decoders achieve higher recognition accuracy than PD, while PD decoders are faster than AR. CPPD, with carefully designed CO and CC modules, achieves a balance between the accuracy of AR and the speed of PD.

The accuracy (%) and model files of CPPD on the public dataset of scene text recognition are as follows:：

  • English dataset from PARSeq.
Model IC13
857		 SVT IIIT5k
3000		 IC15
1811		 SVTP CUTE80 Avg Download
CPPD Tiny 97.1 94.4 96.6 86.6 88.5 90.3 92.25 en
CPPD Base 98.2 95.5 97.6 87.9 90.0 92.7 93.80 en
CPPD Base 48*160 97.5 95.5 97.7 87.7 92.4 93.7 94.10 en
  • Trained on Synth dataset(MJ+ST), Test on Union14M-L benchmark from U14m.
Model Curve Multi-
Oriented		 Artistic Contextless Salient Multi-
word		 General Avg Download
CPPD Tiny 52.4 12.3 48.2 54.4 61.5 53.4 61.4 49.10 Same as the table above.
CPPD Base 65.5 18.6 56.0 61.9 71.0 57.5 65.8 56.63 Same as the table above.
CPPD Base 48*160 71.9 22.1 60.5 67.9 78.3 63.9 67.1 61.69 Same as the table above.
  • Trained on Union14M-L training dataset.
Model IC13
857		 SVT IIIT5k
3000		 IC15
1811		 SVTP CUTE80 Avg Download
CPPD Base 32*128 98.7 98.5 99.4 91.7 96.7 99.7 97.44 en
Model Curve Multi-
Oriented		 Artistic Contextless Salient Multi-
word		 General Avg Download
CPPD Base 32*128 87.5 70.7 78.2 82.9 85.5 85.4 84.3 82.08 Same as the table above.
Model Scene Web Document Handwriting Avg Download
CPPD Base 74.4 76.1 98.6 55.3 76.10 ch
CPPD Base + STN 78.4 79.3 98.9 57.6 78.55 ch

2. Environment

Please refer to "Environment Preparation" to configure the PaddleOCR environment, and refer to "Project Clone"to clone the project code.

Dataset Preparation

English dataset download Union14M-Benchmark download Chinese dataset download

3. Model Training / Evaluation / Prediction

Please refer to Text Recognition Tutorial. PaddleOCR modularizes the code, and training different recognition models only requires changing the configuration file.

Training

Specifically, after the data preparation is completed, the training can be started. The training command is as follows:

1
2
3
4
5
# Single GPU training (long training period, not recommended)
python3 tools/train.py -c configs/rec/rec_svtrnet_cppd_base_en.yml

# Multi GPU training, specify the gpu number through the --gpus parameter
python3 -m paddle.distributed.launch --gpus '0,1,2,3'  tools/train.py -c configs/rec/rec_svtrnet_cppd_base_en.yml

Evaluation

You can download the model files and configuration files provided by CPPD: download link, take CPPD-B as an example, using the following command to evaluate:

1
2
3
4
# Download the tar archive containing the model files and configuration files of CPPD-B and extract it
wget https://paddleocr.bj.bcebos.com/CCPD/rec_svtr_cppd_base_en_train.tar && tar xf rec_svtr_cppd_base_en_train.tar
# GPU evaluation
python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c ./rec_svtr_cppd_base_en_train/rec_svtrnet_cppd_base_en.yml -o Global.pretrained_model=./rec_svtr_cppd_base_en_train/best_model

Prediction

1
python3 tools/infer_rec.py -c ./rec_svtr_cppd_base_en_train/rec_svtrnet_cppd_base_en.yml -o Global.infer_img='./doc/imgs_words_en/word_10.png' Global.pretrained_model=./rec_svtr_cppd_base_en_train/best_model

4. Inference and Deployment

4.1 Python Inference

First, the model saved during the CPPD text recognition training process is converted into an inference model. ( Model download link ), you can use the following command to convert:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# export model
# en
python3 tools/export_model.py -c configs/rec/rec_svtrnet_cppd_base_en.yml -o Global.pretrained_model=./rec_svtr_cppd_base_en_train/best_model.pdparams Global.save_inference_dir=./rec_svtr_cppd_base_en_infer
# ch
python3 tools/export_model.py -c configs/rec/rec_svtrnet_cppd_base_ch.yml -o Global.pretrained_model=./rec_svtr_cppd_base_ch_train/best_model.pdparams Global.save_inference_dir=./rec_svtr_cppd_base_ch_infer

# speed test
# docker image https://hub.docker.com/r/paddlepaddle/paddle/tags/: sudo docker pull paddlepaddle/paddle:2.4.2-gpu-cuda11.2-cudnn8.2-trt8.0
# install auto_log: pip install https://paddleocr.bj.bcebos.com/libs/auto_log-1.2.0-py3-none-any.whl
# en
python3 tools/infer/predict_rec.py --image_dir='../iiik' --rec_model_dir='./rec_svtr_cppd_base_en_infer/' --rec_algorithm='CPPD' --rec_image_shape='3,32,100' --rec_char_dict_path='./ppocr/utils/ic15_dict.txt' --warmup=True --benchmark=True --rec_batch_num=1 --use_tensorrt=True
# ch
python3 tools/infer/predict_rec.py --image_dir='../iiik' --rec_model_dir='./rec_svtr_cppd_base_ch_infer/' --rec_algorithm='CPPDPadding' --rec_image_shape='3,32,256' --warmup=True --benchmark=True --rec_batch_num=1 --use_tensorrt=True
# stn_ch
python3 tools/infer/predict_rec.py --image_dir='../iiik' --rec_model_dir='./rec_svtr_cppd_base_stn_ch_infer/' --rec_algorithm='CPPD' --rec_image_shape='3,64,256' --warmup=True --benchmark=True --rec_batch_num=1 --use_tensorrt=True

Note: If you are training the model on your own dataset and have modified the dictionary file, please pay attention to modify the character_dict_path in the configuration file to the modified dictionary file.

After the conversion is successful, there are three files in the directory:

1
2
3
4
/inference/rec_svtr_cppd_base_en_infer/
    ├── inference.pdiparams
    ├── inference.pdiparams.info
    └── inference.pdmodel

4.2 C++ Inference

Not supported

4.3 Serving

Not supported

4.4 More

Not supported

Citation

@article{Du2023CPPD,
  title     = {Context Perception Parallel Decoder for Scene Text Recognition},
  author    = {Du, Yongkun and Chen, Zhineng and Jia, Caiyan and Yin, Xiaoting and Li, Chenxia and Du, Yuning and Jiang, Yu-Gang},
  booktitle = {Arxiv},
  year      = {2023},
  url       = {https://arxiv.org/abs/2307.12270}
}

Comments