OCR processing with deep learning: Apply to Vietnamese documents
14 likes4,733 views
This document discusses using deep learning techniques like LSTM and CTC for optical character recognition (OCR), specifically for Vietnamese documents. It provides an overview of OCR, the history including Tesseract, and challenges with traditional approaches. Connectionist temporal classification (CTC) is introduced as a way to directly train RNNs on unsegmented sequence data. CTC combined with LSTM networks allows for end-to-end training of OCR without needing pre-segmented text. The document demonstrates how this approach can be applied to perform OCR on Vietnamese documents.
![ARCHITECTURE [CONT’D]](https://image.slidesharecdn.com/deeplearningforocrprocessing-160128173640/85/OCR-processing-with-deep-learning-Apply-to-Vietnamese-documents-7-320.jpg)
OCR processing with deep learning: Apply to Vietnamese documents
- 1. OCR PROCESSING WITH DEEP LEARNING: APPLY TO VIETNAMESE DOCUMENTS VIET-TRUNG TRAN, ANH PHI NGUYEN, KHUYEN NGUYEN
- 2. OUTLINE • OCR overview • History • Pipelining • Deep learning for OCR • Motivation • Connectionist temporal classification (CTC) network • LSTM + CTC for sequence recognition
- 3. WHAT IS OCR • Optical character recognition (optical character reader) (OCR) is the mechanical or electronic conversion of images of typed, handwritten or printed text into machine-encoded text
- 4. OCR TYPES • Optical Character Recognition (OCR) • Targets typewritten text, one character at a time • Optical Word Recognition (OWR) • Typewritten text, one word at a time • Intelligent Character Recognition (ICR) • Handwritten print script, one character at a time • Intelligent Word Recognition (IWR) • Handwritten, one word at a time
- 5. HISTORY OF OCR: TESSERACT OCR ENGINE TIMELINE
- 9. PAGE LAYOUT ANALYSIS Smith, Ray. "Hybrid page layout analysis via tab-stop detection." Document Analysis and Recognition, 2009. ICDAR'09. 10th International Conference on. IEEE, 2009.
- 10. IMAGE LEVEL PAGE LAYOUT ANALYSIS • Using the morphological processing from Leptonica • http://www.slideshare.net/versae/javier-de-larosacs9883-5912825
- 12. COLUMN FINDING
- 13. BLOCK FINDING
- 15. FEATURES AND WORD CLASSIFIER Classical character classification
- 17. CHAR SEGMENTATION, LANGUAGE MODEL AND BEAM SEARCH
- 18. OCR CHALLENGES 1. Fonts specifics Never overcome their ability to understand a limited numbers of fonts and page formats 2. Character bounding boxes 3. Extracting features unreliable 4. Slow performance
- 19. TESSERACT TUTORIAL @ DAS 2014
- 20. RECENT IMPROVEMENTS 1. Multilanguages 2. Full layout analysis 3. Table detection 4. Equation detection 5. Better language models 6. Hand-written text
- 21. LSTM FOR TEXT RECOGNITION
- 22. MOTIVATION • Segmentation is difficult for cursive or unconstrained text • R. Smith, “History of the Tesseract OCR engine: what worked and what didn’t ,” in DRR XX, San Francisco, USA, Feb. 2013. • there was not a single method proposed for OCR, that can achieve very low error rates without using aforementioned sophisticated post-processing techniques.
- 23. RESEARCH BREAKTHROUGH A. Graves, M. Liwicki, S. Fernandez, Bertolami, H. Bunke, and J. Schmidhuber, “A Novel Connectionist System for Unconstrained Handwriting Recognition,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 855–868, May 2008.
- 26. MOTIVATION • Real-world sequence learning task • OCR (Optical character recognition) • ASR (Automatic speech recognition) • Requires • prediction of sequences of labels from noisy, unsegmented input data • Recurrent neural networks (RNN) can be used for sequence learning, but ask for • pre-segmented training data • post-processing to transform outputs into label sequences
- 27. CONNECTIONIST TEMPORAL CLASSIFICATION (CTC) • Graves, Alex, et al. "Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks." Proceedings of the 23rd international conference on Machine learning. ACM, 2006. • WHAT CTC IS ALL ABOUT? •a novel method for training RNNs to label unsegmented sequences directly
- 28. THE SPEECH RECOGNITION PROBLEM
- 34. DYNAMIC TIME WRAPERING • Because the length of y might differ from (often longer than) l, so the inference of l from y is actually a dynamic time warping problem.
- 37. CONNECTIONIST TEMPORAL CLASSIFICATION • o transform the network outputs into a conditional probability distribution over label sequences • A CTC network has a softmax output layer with one more unit than there are labels in L • activations of the first |L| units are interpreted as the probabilities of observing the corresponding labels at particular times • activation of the extra unit is the probability of observing a ‘blank’, or no label
- 38. PREFIX SEARCH DECODING ON THE LABEL ALPHABET X,Y
- 40. LONG SHORT-TERM MEMORY (LSTM) • One type of RNN networks • RNN vanishing gradient problem • influence of a given input on the hidden layer, and therefore on the network output, either decays or blows up exponentially as it cycles around the network’s recurrent connections • LSTM is designed to address vanishing gradient problem • An LSTM hidden layer consists of recurrently connected subnets, called memory blocks • Each block contains a set of internal units, or cells, whose activation is controlled by three multiplicative gates: the input gate, forget gate and output gate
- 42. FORGET GATE
- 43. INPUT GATE
- 44. OUTPUT GATE
- 45. LSTM -> CTC OUTPUT LAYER: OCR
- 46. DEMO TIME: OCR FOR VIETNAMESE DOCUMENTS Thank you!
- 47. REFERENCES - CREDITS • https://github.com/yiwangbaidu/notes/blob/master/CTC/CTC.pdf • http://colah.github.io/posts/2015-08-Understanding-LSTMs/ • Ray Smith. Everything you always wanted to know about Tesseract. Tesseract tutorial @ DAS 2014
Editor's Notes
- #8: http://www.slideshare.net/100002968637682/seminar-p2
- #9: http://www.slideshare.net/DocuFi/improve-ocr-accuracy-with-cleanup?qid=a439a1d4-ad96-4fb3-b376-9b301e58674b&v=default&b=&from_search=13
- #18: http://www.slideshare.net/temsolin/6-char-segmentation