SCENE TEXT RECOGNITION IN MOBILE APPLICATION BY CHARACTER DESCRIPTOR AND STRUCTURE CONFIGURATION

SCENE TEXT RECOGNITION IN MOBILE
APPLICATION BY CHARACTER
DESCRIPTOR AND STRUCTURE
COMFIGURATION
CHERIYAN K M

INTRODUCING…..
 Valuable information form an image.
 To extract an information.
 Automatic and Effective scene text detection.
 Recognition algorithm.
 Factors affecting on extraction.
 Cluttered background.
 Difference in text pattern.
 Difficult to model the structure of character.
 Lake of discriminative pixel level appearance.
 Structure features from non-text background outliers.
 Different word , may diff. characters , in various fonts ,
styles and size.

 Two activities;
 Text detection.
 Localize the image region containing the text characters.
 Based on
 Color uniformity and
 Horizontal alignment of text char.
 Text recognition.
 Transform pixel-based text into reliable codes.
 Distinguish diff. text characters , Properly compose the text
word.
 62 identity category of text characters.
9 (0-9)
26 (a-z)
26 (A-Z)
 Two schemes;
 Character recognizer to predict the category of text char.
 Binary character classifier to predict the existence of ctgry.

RELATED WORKS
 Optical Character Recognizer (OCR) system.
 Many algorithms are proposed;
 Weinmen:- combined the Gabor-based appearance
model.
 Neumann:- based on extremal region.
 Smith:- based on SIFT.
 Mishra:- adopted conditional random field.
 Lu:- modeled the inner character structure.
 Coates:- extracted local features of character patches.

LAYOUT BASED SCENE TEXT DETECTION
 A text;
 Instruction
 Identifier
 Uniform color
 Aligned arrangement
Two processes are employed to complete layout
analysis
1. Color Decomposition
2. Horizontal Alignment
Improved to compatible with mobile app

LAYOUT ANALYSIS OF COLOR
DECOMPOSITION
 Boundary clustering algorithm base on bigram color
uniformity.
 Group pixels of same color into a layer.
 Character boundary boarder b/w txt and bg.(color
pair)
 Create a vector of color pair (txt and bg).

LAYOUT ANALYSIS OF HORIZONTAL
ALIGNMENT
Text information(string)
Several
character
members
In similar
size
Approximately
horizontal
alignment
The geometrical properties to detect the existence of text characters

 Adjacent character grouping algorithm
Bounding box>siblings>similar size & vertical location>merge

 For non-horizontal strings-> ± /6 degree set as
range.

STRUCTURE BASED SCENE TEXT
RECOGNITION
 To extract text information.
 Binary classification problem.
 Character classes(Queried characters).
 Binary classifier:- to distinguish character class
from other classes or bg outliers.
 Eg: Character class A predict patches containing A as
positive. And other as negative.
 Two activities;
1. Character descriptor.
2. Stroke configuration.

CHARACTER DESCRIPTOR
 Extract structure features.
 4 different key points features;
1. Harris Detector:- To extract Key points from corner and
junction.
2. MSER Detector:-To extract Key point from stroke
component.
3. Dense Detector:- To extract Key point uniformly.
4. Random Detector:- To extract the preset number of
Key points in a random pattern.

Flowchart of our proposed character descriptor
HOG:-features are Calculated as observed feature vector x.
(Histogram of Oriented Gradient)
•Selected as local feature descriptor( compatibility
with all 4 key point detectors).

 SIFT and SURF are not employed
 Normalization of character patches(128x128).
 Feature Quantization: to aggregate the extracted
features
 Bag-of-Words(BOW) Medel:- Applied to key points from
all 4 feature detector.
 Gaussian Mixture Model(GMM):Applied to key points
from DD & RD.(fixed number and location of key point)
 Now mapping both into characteristic Histogram as
feature representation.
 Cascading BOW and GMM-based feature repr. ,we
get Character Descriptor.

CHARACTER STROKE CONFIGURATION
 Stroke:- Region bounded by two parallel boundary
segments.
 Stroke width
 Stroke orientation
 Characters are connected strokes with
configuration.
 Structure Map of Strokes is stroke configuration.(
is consistant)
Eg: B have 1 vertical stroke
2 arc strokes.
B

 Synthesized character generator: Estimate stroke
configuration from computer s/w.(Provide accurate
skeleton and boundary)
 Synthetic font training dataset(20000 are selected
out off 67400 character patches)
 Contain 62 class of characters(128x128 pixel)
 Compose Stroke Configuration
Step1
 Discrete Contour Evaluation(DCE):obtain boundary and
skeleton. Skeleton pruning on the basis of DCE.
 DCE simplifies the character(using polygon and small
no. of vertices)
 DCE and Skeleton pruning are invariant to deformation
and scaling.

Step2
 Estimate stroke width and orientation
 Width: length along normal
 Orientation: tangent
 Sampling from character boundary
 128 samples.
 So that no. of samples = length
 Estimating
 Taking two neighboring sample point to fit a line.
 Approximately collinear.
 A quadratic curve.

Step3
 Calculate Skeleton-based stroke map
 Consistency of stroke width and orientation.
Width no larger than 3 Orientation no larger than /8
 Construct stroke section: If sample point satisfying
the stroke related features.
 Construct junction sections: If they are not.
 Skeleton points are extracted.

STROKE ALIGNMENT METHOD
 To handle various fonts, styles …..etc
 Mean value of all stroke configuration.
 Mean value,

 D=Distance b/w stroke configurations of two
samples
 S=Mean value of stroke configurations.
 Ti=Transformations applied on strokes of i-th stroke
configuration.
 g(Ti)=Amplitude of the transformation.

SCENE TEXT RECOGNITION IN MOBILE APPLICATION BY CHARACTER DESCRIPTOR AND STRUCTURE CONFIGURATION

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to SCENE TEXT RECOGNITION IN MOBILE APPLICATION BY CHARACTER DESCRIPTOR AND STRUCTURE CONFIGURATION (20)

Recently uploaded (20)

SCENE TEXT RECOGNITION IN MOBILE APPLICATION BY CHARACTER DESCRIPTOR AND STRUCTURE CONFIGURATION