Reading text in consumer digital photographs

We present a distributed system to extract text contained in natural scenes within consumer photographs. The objective is to automatically annotate pictures in order to make consumer photo sets searchable based on the image content. The system is designed to process a large volume of photos, by quickly isolating candidate text regions, and successively cascading them through a series of text recognition engines which jointly make a decision on whether or not the region contains text that is readable by OCR. In addition, a dedicated rejection engine is built on top of each text recognizer to adapt its confidence measure to the specifics of the task. The resulting system achieves very high text retrieval rate and data throughput with very small false detection rates.