Segmentation and recognition of handwritten digit numeral string using a multi layer perceptron neural networks

International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.1, January 2016
DOI:10.5121/ijfcst.2016.6104 49
SEGMENTATION AND RECOGNITION OF
HANDWRITTEN DIGIT NUMERAL STRING USING A
MULTI LAYER PERCEPTRON NEURAL NETWORKS
N. Venkateswara Rao1
and Dr. B. Raveendra Babu2
1
Dept. of Computer Science & Engineering, R.V.R. & J.C. College of Engineering,
Guntur, INDIA
2
Professor, Dept. of Computer Science & Engineering VNR Vignana Jyothi Institute of
Engineering and Technology, Hyderabad, INDIA
ABSTRACT
In this paper, the use of Multi-Layer Perceptron (MLP) Neural Network model is proposed for recognizing
unconstrained offline handwritten Numeral strings. The Numeral strings are segmented and isolated
numerals are obtained using a connected component labeling (CCL) algorithm approach. The structural
part of the models has been modeled using a Multilayer Perceptron Neural Network. This paper also
presents a new technique to remove slope and slant from handwritten numeral string and to normalize the
size of text images and classify with supervised learning methods. Experimental results on a database of
102 numeral string patterns written by 3 different people show that a recognition rate of 99.7% is obtained
on independent digits contained in the numeral string of digits includes both the skewed and slant data.
KEYWORDS
Connected Components Labeling, Multi-Layer Perceptron Neural Networks, Segmentation, Feature
Extraction, Handwritten recognition.
1. INTRODUCTION
Recognizing writer-independent handwritten numeral string of digits is still a difficult problem
for a computer although it has been a research topic for over so many decades. The key problem
is due to an unlimited number of styles, sizes and variations of digit patterns used by different
people. As a critical preprocessing stage in handwritten numerical string recognition, the
numerical string digits segmentation exports results which will affect the performance of the
overall recognition system. There still exist several challenges in the numerical string
segmentation for hand written document. Unlike machine printed document, the free-style
handwritten numerical string of digits is often curved, have various skew angles, no uniform
direction, connect or overlap with each other. A study of the present literature exposes that an
excessive amount of research work has been made to resolve the problem. Different sets of
features in topological and spatial domain, as well as in frequency domain, have been proposed
for the recognition of handwritten characters [1]. Moreover, a variety of classification methods,
such as template matching, structural, syntactic and neural network approaches are adopted for

50
classification and/or recognition [2, 11]. However, most of these algorithms could not yield
satisfactory recognition performance in the cases of rotated and noisy patterns. Multi-module
(multi-expert) neural networks have been studied for the last few years in order to reduce the
learning complexity and to improve the system performance [3]. The neural network classifiers
are generally classified into two categories. In one category, each module of the neural network is
used to look after a sub-region of the feature space or a subgroup of classes in the hope that each
of these local expert can take better care of its own domain so that the overall performance can be
improved [4, 5]. In the other category, each module of the neural network is used to deal with the
input from one of the multiple sensors that are used to collect all possible data. Handwritten
character recognition task is very complex. Normally recognition of isolated characters is a
simple task than a string of characters [6-10].These systems are used in many areas like automatic
processing of bank checks, tax forms, postal identification numbers, optical response sheets etc.,.
In this work, a connected component labeling algorithm [12] is used to segment the individual
numerals of the image. For any character recognition problems to segment the characters, it is
required to do some preprocessing methods for getting better features.
2. SEGMENTATION AND RECOGNITION OF HAND WRITTEN NUMERAL
STRING
The procedure done before processing by correcting images from different errors is called
preprocessing. The preprocessing is to be done before image enhancement. It includes conversion
to a binary image, applying Median filtering to remove noise, and Thinning etc.
2.1 Binarization:
Scanned input image is given as input. It is checked whether the image is color or grayscale or
binary. If the image is not binary image it is converted to binary image.
1. Calculating the size of the image.
2. Finding the sum of the pixels.
3. Calculating the average threshold value using the sum of the pixels and the size of the
image.
4. Comparing the generated threshold value with the pixel value of the image. If the pixel is
value is greater than the threshold value, then assign the pixel a value of ‘1’ otherwise
‘0’.
2.2 Noise Removal:
Median filters are commonly used methods to remove the noise. Median filter is one of the most
popular non-linear filters to remove the salt & pepper noise. The noise is removed by substituting
the mask center value by the median value of midpoint neighborhood.
2.3 Skew Correction
In mathematical terms skew means, lines that are neither parallel nor intersecting. The skew
correction is performed on such lines. The image is rotated with an angle to remove the skew.
Skew correction is used to align the image base line with the x-axis.In this system, a lower
baseline is drawn where the maximum pixels are located. An example is shown in Figure 1.

51
1) The lowest black pixel is determined in every column of the image to populate the set S.
S = {si = (xi; yi)/lowest black pixel in column xi}
2) To fit as the baseline of the current line, the data set S is used to calculate a least-squares
linear regression to find a base line of the form y = mx+c
3) Computing the arctangent of the line slope that produces the rotation angle, theta= arctan(m)
4) Rotate the image by negative theta value computed in step 3 to remove the skew.
Figure 1 Sample Skew Correction Image
2.4 Slant Removal
It is used to normalize writing where the text is at an angle with the goal of making the text
upright. Here based on a line, image is rotated with the angle so that the slant present in the image
is removed.
The slant correction algorithm is as follows:
1) Calculate the new transformations from -45 degrees to 45 degrees angle to shear and
transform the line
2) Generate a vertical projection histogram for each shear angle calculated in the above step.
3) Calculate Time Frequency distribution of each vertical histogram computed in the above step.
4) Remove the slant of the image by choosing angle representing the largest distribution
intensity from the histogram computed in the above step.
Figure 2 Sample Slant Correction Image
2.5 Segmentation
Segmentation is the process of dividing the string of numeral digits into individual digit images.
Normally there are three approaches for dividing the string of numerals into individual digits.
They are 1) External segmentation, where digit boundaries are found prior to segmentation, 2)
Internal segmentation, in which letter boundaries are determined as part of recognition, and 3) No
segmentation, where recognition occurs at string level. In this paper an internal segmentation is
done by using a connected components labeling (CCL) approach method.

52
The algorithm for finding the connected components of an image is
1) Scan the image by moving along a row until it comes to a point p (where p denotes the
pixel to be labeled at any stage in the scanning process) for which V= {1}.
2) When this is true, examine the four neighbors of p which have already been encountered
in the scan
a. To the left of p.
b. Above it.
c. The two upper diagonal terms.
3) The labeling of p occurs as follows:
a. If all four neighbors are 0, assign a new label top, else
b. if only one neighbor has V={1}, assign its label to p, else
c. If more than one of the neighbors have V={1}, assign one of the labels to p and
make a note of the equivalences.
2.6 Feature Extraction
For achieving high recognition rate, the selection of appropriate feature extraction method is very
important. After pre-processing i.e., skew correction, slant correction, and segmentation is done
the image is normalized to a 15x15 without changing the aspect ratio. A total of 225 features are
taken from the image to classify the image. The features of the characters that are important for
classifying them at recognition stage are extracted. This is an important stage as its effective
functioning increases the recognition rate and decreases the misclassification. The features are
extracted for all images after segmenting the string of numerals into individual digits where one
sample of the image is shown in Figure 1 which is a skew corrected image and Figure 2 is after
slant correction.
2.7 Classification
As the features are extracted, an appropriate classifier must be selected. A number of classifiers
are used and each classifier suitable to classify a specific kind of feature vector depending upon
its characteristics is found. The Neural Network classifier is used commonly for classification. A
multi-layer perceptron neural network classifier is used here for recognition. The most common
classifier model is multi-layer perceptron neural networks. In order to learn this neural network, it
requires a desired output because it is a supervised network. The goal of this network is to create
a model which correctly maps from input to the output with historical data. A sample
representation of a multi-layer perceptron is shown in Figure 3.

53
Figure 3 Architecture of Multi Layer Perceptron (MLP) Neural Network
3. RESULTS AND DISCUSSIONS
Experiments are conducted on different numerical string of characters written by three different
writers. In this work, 40 samples from writer1 and 38 samples from writer2 and 24 samples from
writer3 are collected. A total of 102 numerical strings of characters from these three different
writers are collected. These samples contain a total number of 617 individual digits covering all
the digits approximately equal which is shown in table 1. Each sample of numerical string is
segmented into individual digits. Each digit is normalized to a size of 15X15 pixels. From the
normalized image the features are extracted and training is performed with the multi layer
perceptron back propagation neural network. From the confusion matrix, shown in Figure 4, it is
observed that a recognition rate of 99.7% and an error rate of 0.3% is obtained. Some of the
sample images written by different writers are shown in Figure 5.
Table 1 Sample Images database
Writer1 Writer2 Writer3 Total
No: of samples 40 38 24 102
Number of 0’s 22 21 14 57
Number of 1’s 29 22 15 66
Number of 2’s 28 22 15 65
Number of 3’s 19 23 17 57
Number of 4’s 25 22 15 62
Number of 5’s 25 24 12 63
Number of 6’s 27 24 14 65
Number of 7’s 26 22 14 62
Number of 8’s 27 22 15 63
Number of 9’s 23 21 13 59
Total Digits 251 223 144 617

54
Figure 4 Confusion Matrix
Figure 5 Sample Images
4. CONCLUSION
Previous work of the authors of this paper is limited to an isolated digit character. In this paper, a
new method for handwritten digits recognition from a string of numerals using a multi-layer
perceptron neural network is presented. The effectiveness of this proposed method was evaluated
by computing the recognition rate and error rate. It is observed that a recognition rate of 99.7%
and an error rate of 0.3% on handwritten string of numerals is obtained.
REFERENCES
[l] C. Y. Suen. Distinctive features in automatic recognition of hand printed characters. Signal
Processing, 4(2 & 3):193 - 207, 1982.
[2] S. Mori, C. Y. Suen, and K. Yamamoto. Historical review of OCR research and development.
Proceedings of the IEEE, 80(7):1029 - 1058, 1992.

55
[3] Michael I. Jordan and Robert A. Jacobs. Hierarchies of adaptive experts. In Advance in Neural
Information Processing Systems 4,
[4] Tatsuo Matsuoka, Hiroshi Hamada, and Ryohei Nakatsu. Syllable recognition using integrated neural
networks. In IEEE INNS International Joint Conference on Neural Networks, volume 1, 1989.
[5] Z. Chi and M. Jabri. A Multi-Module Neural Network Approach for ICEG Classification. In
Proceedings of the Third Australian Conference on Neural Networks, Canberra, Australia, 1992.
[6] Thien M. Ha, Matthias Zimmermann, Horst Bunke, Off-line handwritten numeral string recognition
by combining segmentation-based and segmentation-free methods, Journal of Pattern Recognition,
Volume 31, Issue 3, Pages 257–272, March 1998.
[7] Zhixin Shi, Venu Govindaraju, Segmentation and recognition of connected handwritten numeral
strings, Journal of Pattern Recognition, Volume 30, Issue 9, Pages 1501-1504, 1997.
[8] Ashraf Elnagara, Reda Alhajj, Segmentation of connected handwritten numeral strings, Journal of
Pattern Recognition, Volume 36, Issue 3, Pages 625–634, March 2003.
[9] Yi-Kai Chen, Jhing-Fa Wang, Segmentation of single- or multiple-touching handwritten numeral
string using background and foreground analysis, IEEE Transactions on Pattern Analysis and Machine
Intelligence, Volume 22, Issue 11, Pages 1304 – 1317, Nov 2000.
[10] Javad Sadria, Ching Y. Suena, Tien D. Bui, A genetic framework using contextual knowledge for
segmentation and recognition of handwritten numeral strings, Journal of Pattern Recognition, Volume
40, Issue 3, Pages 898–919, March 2007.
[11] N. Venkateswara Rao, Dr. B. Raveendra Babu, G. Rama Mohan Babu, A Radial Basis Function
Neural Network to Recognize Handwritten Numerals with normalized moment features from
skeletons, IEEE International Conference on Recent Trends in Information Technology (ICRTIT),
Pages 68 – 72, 2013.
[12] Di Stefano, L.Bulgarelli, Andrea , A simple and efficient connected components labeling algorithm,
International Conference on Image Analysis and Processing, Pages 322 – 327, 1999.
AUTHORS
N. VenkateswaraRao, received his M.Sc degree in Computer Science Department
from Acharya Nagarjuna University, India. He did his M.Tech in Computer Science
& Technology from Andhra University, India. He is currently working as Associate
Professor, in the Department of Computer Science & Engineering at RVR & JC
College of Engineering, Guntur, India. He has 15 years of teaching experience. His
research areas of interest include Artificial Neural Networks, Image Processing, and
Pattern Recognition. He is life member of ISTE.
Dr. B. Raveendra Babu, obtained his Masters in Computer Science and Engineering
from Anna University, Chennai. He received his Ph.D. in Applied Mathematics at S.
V. University, Tirupati. He is currently working as professor and HOD in department
of Computer Science & Engineering at VNR Vignana Jyothi Institute of Engineering
and Technology, Hyderabad. He has 30 years of teaching experience. He has more
than 40 international & national publications to his credit. His research areas of
interest include VLDB, Image Processing, Pattern analysis and Wavelets. He is life
member in professional bodies like ACM, ISTE and CSI.

Segmentation and recognition of handwritten digit numeral string using a multi layer perceptron neural networks

More Related Content

What's hot (20)

Similar to Segmentation and recognition of handwritten digit numeral string using a multi layer perceptron neural networks (20)

More from ijfcstjournal (20)

Recently uploaded (20)

Segmentation and recognition of handwritten digit numeral string using a multi layer perceptron neural networks