Motivation

Recognizing scene text is a challenging problem, even more so than the recognition of scanned documents. Given the rapid growth of camera-based applications readily available on mobile phones, understanding scene text is more important than ever. One could, for instance, foresee an application to answer questions such as, “What does this sign say?”. This is related to the problem of Optical Character Recognition (OCR), which has a long history in the computer vision community. However, the success of OCR systems is largely restricted to text from scanned documents. Scene text exhibits a large variability in appearances, and can prove to be challenging even for the state-of-the-art OCR methods. Many scene understanding methods recognize objects and regions like roads, trees, sky in the image successfully, but tend to ignore the text on the sign board. Our goal is to fill this gap in understanding the scene.

Highlights

• Binarization as a labelling problem (see our ICDAR'11 paper)
• Both open and closed vocabulary word recognition (see our CVPR'12 paper and BMVC'12 paper)
• Use of both top-down (lexicons) and bottom-up cues (character detection)
• Holistic recognition approach of lexicon-driven scene text recognition (see our ICDAR'13 paper)
• Applications in image retrieval (see our ICCV'13 paper)
• Word recognition for large lexicions and cropped word image retrieval (see our ACCV'14 paper)

Code

Generating exemplars (based on our ICDAR'13 paper)  README

Scene Text Binarization

Scene Character Recognition