Neural network based numerical digits recognization using nnt in matlab

International Journal of Computer Science & Engineering Survey (IJCSES) Vol.4, No.5, October 2013

Neural network based Numerical digits
Recognization using NNT in Matlab
Amritpal kaur1, Madhavi Arora2
M.tech- ECE1, Assistant Professor2
Global institute of engineering and technology, Amritsar

ABSTRACT
Artificial neural networks are models inspired by human nervous system that is capable of learning. One of
the important applications of artificial neural network is character Recognition. Character Recognition
finds its application in number of areas, such as banking, security products, hospitals, in robotics also.
This paper is based on a system that recognizes a english numeral, given by the user, which is already
trained on the features of the numbers to be recognized using NNT (Neural network toolbox) .The system
has a neural network as its core, which is first trained on a database. The training of the neural network
extracts the features of the English numbers and stores in the database. The next phase of the system is to
recognize the number given by the user. The features of the number given by the user are extracted and
compared with the feature database and the recognized number is displayed.

KEYWORDS
Steepest Descent algorithm, Back propagation network, Artificial neural network, Feature extraction, Least
mean square

1. INTRODUCTION
The handwriting recognition refers to the identification of written characters. Handwriting
recognition has been become a very important and useful research area in recent years for the
ease of access of many applications. Numerous approaches have been proposed for character
recognition and considerable success has been reported [1]. Numeral recognition refers to the
process of translating images of handwritten, typewritten, or printed digits into a format
understood by the user for the purpose of editing, indexing, searching and reduction in storage
size .Number recognition can be online or offline. In online number recognition, data are captured
during the writing process with the help of special pen and an electronic interface. Offline
documents are scanned images of prewritten text, generally on sheet or paper [2]. Offline number
recognition is significantly different from online number recognition, because here, stoke
information is not available [3, 4]. In this paper, we work with offline numerals. In this paper
artificial neural network is used for recognition of numerals. Back propagation network is used
for training of artificial neural network. The learning rule used here is Steepest Descent algorithm
with adaptive learning capabilities.

1.1. STEEPEST DESCENT ALGORITHM
Steepest Descent algorithm calculates the gradient of function. Gradient means change of error
energy with respect to weight value. This algorithm is used to calculate the error. Training in
neural networks involves finding the minimum of a complicated non linear function (called error
DOI : 10.5121/ijcses.2013.4502

19


function).This algorithm describes the error that a neural networks makes in approximating or
classifying the training data. The error that mainly occurs in training of network is mean square
error (mse). Mse is a network performance function. It measures the network performance
according to mean of squared errors.

1.2. ARTIFICAL NEURAL NETWORK
Artificial neural networks have been developed as generalizations of mathematical models of
biological nervous systems. A first wave of interest in neural networks emerged after the
introduction of simplified neurons by McCulloch and Pitts (1943). The basic processing elements
of neural networks are called artificial neurons [5] or simply neurons or nodes. In a simple
mathematical model of neuron, the effects of the synapses are represented by connection weights
that modulate the effect of associated input signals and the non linear characteristics exhibited by
the neurons that are represented by transfer function [6]. The neuron impulse function is
calculated as the weighted sum of input signals, with the help of the transfer function. The
Learning capability of artificial neurons is achieved by adjusting the weights in accordance with
chosen learning algorithm. The artificial neural network can be trained into two main groups that
are supervised and unsupervised learning. In supervised learning, the network learns by example
with help of some training algorithm and a target value is set, whereas in unsupervised learning,
no target value is set or no example is given. Unsupervised learning is very complex and difficult
to implement [7].

Figure 1.shows artificial neural network

1.3. BACK PROPAGATION NETWORK
Back propagation, an abbreviation for ‘backward propagation of errors’, is a method used for
training of artificial neural network. From the desired or actual output, the network learns from
number of inputs, similar to the way a child learns to identify a car from group of toys. It is
supervised learning method. It requires a dataset of desired output from set of many inputs,
making up the training set. A back propagation network consists of three layers of units- input
layer, hidden layer and output layer as shown in figure 2. These units are connected in feed
forward manner with input units fully connected with units in hidden layer and units in hidden
layer fully connected with units in output layer. When back propagation network is cycled, an
input pattern is propagated forward to the output units through input to hidden and hidden to
output weights. Back propagation is an iterative process that starts with last layer and moves
backward through the layers until the first layer is reached. This algorithm is based upon Widrow
hoff delta learning rule in which the weight adjustment is done through mean square error of
output response to sample input. The rest of these sample patters are repeatedly presented to the

20


network until the error value is minimised. Weights are adjusted according to the error present in
the network.

Figure 2. shows Back Propagation Network

2. ADAPTIVE LEARNING RATE
The back propagation algorithms are basically of two types, gradient descent and gradient descent
with momentum. With standard steepest descent, the learning rate is held constant throughout
training. The performance of this algorithm is dependent on the proper setting of learning rate. If
the learning rate is set too high, the algorithm can starts oscillate and becomes unstable. If
learning rate is kept small, algorithm takes long time to converge. The performance of steepest
learning can be improve if learning rate is allowed to change during training process. An
adaptive learning rate has an advantage of keeping the learning step size as large as possible
while keeping learning process stable. The learning rate is made sensitive to the complexity of
mean square error. This process can increase the learning rate, but up to the extent that the
network can learn without having large number of errors.
One of the most popular algorithms in adaptive learning is least mean square (LMS) algorithm of
Widrow and Hoff. The LMS algorithm has been used here because of its robustness and
simplicity. Key parameter in this algorithm is step size. If the step size is made larger, the rate of
convergence of LMS algorithm is fast but on the other side, mean square error is increased. But,
if the step size is made smaller, the convergence rate of LMS algorithm is slow. Thus step size
provides a trade off between convergence rate and MSE. An intuitive way to improve the
performance of LMS algorithm is to make the step size variable rather than fixed, choose large
step size values during the initial convergence and use small step size values when the system is
close to its steady state , which results in variable step size (adaptive learning rate) LMS
algorithms. By using such approach both fast convergences rate and least MSE can be obtained.
In the VSSLMS algorithm, the step size µ is replaced with variable parameter µ(n) and
coefficient vector w(n) is updated as:
w(n+1) = w(n) + µ(n) e(n) x(n)
Where e(n) is error and x(n) is input to network.
21


3. METHODOLOGY USED
This paper recognizes the English numerals given as an image to the system automatically using
the intelligence of the two layered neural network. The neural network applied in the system
utilizes the Steepest Descent Algorithm with Adaptive Learning Rate as the learning rule, which
is used for the training of the neural network. The basic flow of the system is shown as below:
Neural Network
Initialization and Training

Extracted Feature
Database

Input Test Image

Select the image of the
number to recognize

Image Pre-processing

Test Image Feature
Extraction

Recognizing the Number
based on feature extracted
The methodology used here can be explained as:

3.1. Neural Network Initialization and Training
In the Ist step, neural network is initialised and trained with the help of back propagation
network. Once the network has been initialised, the network is ready to train. Some issues that
need to address upon training the network are: What should be the values of- learning rate,
sigmoid shape, gradient, weight bias and how many values of Iterations (Epochs) are to be taken
for the training of network for given set of inputs. Here the network learns what we have taught to
it.

22


Figure 3.shows NN toolbox for English numeral recognition

Neural network is trained as shown in fig.4. Here we have set 5000 iterations for the network to
train. Following figure.5 shows that our performance goal is met with only 313 iterations out of
5000 iterations. Other parameters of network are also displayed. Time taken for training of
network is displayed as 0:05:26 as well as performance and gradient are also displayed here.

Figure 4.shows training of ANN

23


3.2. Feature Extraction
Feature Extraction is used to extract relevant features for recognition of characters. First features
are computed and extracted and then most relevant features are selected to construct feature
vector which is used for recognition. The computation of features is based on statistical,
structural, directional, moment, transformation like approaches. Feature extraction is extracting
information from raw data which is more relevant for classification purpose and that minimises
the variation within a class and maximizes the variations between classes[8].

3.3. Input Test Image
Input the image which is to be tested. Input images maybe any numerals between 0 to 9. It may
be 0,1,2,3,4,5,6,7,8,9 as well as combination of two or three numbers. The arrow indicates to load
image by which image can be loaded into the system. Here we take an image of 5*7 matrixes
which is displayed in fig.6.

Figure.5 shows Ist step of loading of image in ANN

Figure.6 shows 2nd step of loading image in ANN

24


3.4. Image Pre-Processing
Raw image may carry some unwanted noise. In Pre-processing stage, we basically remove the
unwanted noise and distortion of image. In Image pre processing, the input image is first
converted into gray scale image and then to binary image. This process is called digitization of
image. The scanned image has some noise, the noise is due to some unnecessary details present in
the image. Image Pre processing is used for skeletonization (thinning) and noise reduction or
removal .Skeletonization is necessary here as it decreases the line width of text from many pixels
to single pixel. Noise removal is necessary as it remove unwanted details which do not play any
important role. Image dilation and filling of holes are performed to produce the pre processed
image suitable for segmentation [9]. In segmentation stage, an image consisting of sequence of
characters is decomposed into sub images of individual character [10].In Segmentation, the
position of the object that is the number in the image is find out and size of number is cropped as
shown in figure.7.

Figure.7 shows pre processing of image

3.5. Test image feature extraction
The image from the segmented stage is matched with all the number which is preload into the
system. In this step, the nearest match of the stored and input number is finding out. Once, the
matching is completed the number with maximum matching is declared as the number present in
the image.

3.6. Recognizing the number
This is the final stage .Here the recognized number is displayed as shown: The number that is
highlighted in blue is selected and is recognized. In figure 8, a single number ‘6’ is selected and is
displayed as shown. In figure 9, two numbers ‘67’ is selected and is displayed as 67 shown in
figure.9.

25


Figure.8 shows recognition of numeral ‘6’

Figure.9 shows recognition of numeral ‘67’

4. PERFORMANCE PARAMETERS
The performance parameters show the accuracy of the system in terms of its parameters like
gradient, validation, learning rate and training.

26

Gradient = 0.11536, at epoch 313

2

gradient

10

0

10

-2

10

Validation Checks = 0, at epoch 313
val fail

1
0
-1
Learning Rate = 7.6934, at epoch 313

lr

10
5
0

0

50

100

150
313 Epochs

200

250

300

Best Training Performance is 0.098379 at epoch 313
2

10

Sum Squared Error (sse)

Train
Best
Goal
1

10

0

10

-1

10

-2

10

0

50

100

150

200

250

300

313 Epochs

27


Training: R=0.99935
1

Output ~= 0.96*Target + 0.0056

0.9

Data
Fit
Y =T

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0

0.2

0.4

0.6

0.8

1

Target

5. CONCLUSION AND FUTURE SCOPE
The application of neural network in character recognition has been a field of study recently.
Neural networks are used to solve the tasks which are difficult for conventional computers or
human beings to solve. This paper aims at recognition of English numeral recognition using
gradient descent based back propagation training algorithm. The main aim of using this
algorithm is to reduce the error which is difference between the computed value of neural
network and desired value. By using artificial neural network methodology of English numeral
recognition, 99% accuracy is achieved. The method used in this paper can be further extended to
recognize multilingual characters.

REFERENCES
[1]. R. Plamondon
and S.N. Srihari., 2000, “Online and off-line handwriting recognition: a
comprehensive survey.”, Pattern Analysis and Machine Intelligence, IEEE Transac-tions on, vol.
22(1), pp 63–84.
[2]. Munish kumar, M.K.Jindal and R.K Sharma,”Classification of characters and grading writers in
offline handwritten gurmukhi script”,2011 International Conference on image information processing
(ICIIP 2011)
[3]. M. Lorigo and V. Govindaraju, “Offline Arabic handwriting recognition: a survey”, IEEE
Transactions on PAMI, Vol. 28(5), pp. 712-724, 2006.
[4]. R. Plamondon and S. N. Srihari, ''On-line and offline handwritten character recognition: A
comprehensive survey'', IEEE Transactions on PAMI, Vol. 22 (1), pp. 63-84, 2000.

28

[5]. M. Schuster and K. K. Paliwal, November 1997 , “Bidirectional recurrent neural networks”, IEEE
Transactions on Signal Processing,vol 45, pp 2673–2681.”.
[6]. Yusuf Perwej , November 2012, “ Recurrent Neural Network Method in Arabic Words Recognition
System” ,International Journal of Computer Science and Telecommunications (IJCST), published by
Sysbase Solution (Ltd), UK, London (http://www.ijcst.org) , vol. 3, Issue 11, pp 43-48.
[7]. Sang Sung park,Won Gyo Jung,Young Geun Shin,Dong Sik Jang, Department of Industrial system
and Information Engineering, Korea University, South Korea, “Optical character system Using BP
Algorithm
[8]. O. D. Trier, A. K. Jain and T. Taxt, 1996, ”Feature Extraction Methods for Character Recognition A
Survey”, Pattern Recognition, vol. 29, pp. 641-662.
[9]. Gonzalez ,R.C.woods, R.E.and Eddins, S.L. ,”Digital image processing using MATLAB”, Pearson
Education, Dorling Kindersley,South Asia,(2002).
[10]. Casey, R.G. and Lecolinet, E.,”A Survey of Methods and Strategies in Character Segmentation.”
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.18, No. 7, (1996),690-706.

Authors
Amritpal kaur is pursuing M. Tech in Electronics and Communication Engineering from
Global Institute of Engineering & Technology, Amritsar. She did her B.Tech degree in
Electronics and Communication Engineering from Sai Institute of Engineering &
Technology, Amritsar, Punjab, India. Her area of interest is Image Processing and fuzzy
logic.
Madhavi Arora is presently working as a A.P in Global college of Management and
Emerging Technology. She has done her M. Tech in Electronics and Communication
Engineering from Amritsar College of Engineering & Technology .She did her B.Tech
degree in Electronics and Communication Engineering from Guru Nanak Dev University,
Amritsar, Punjab, India. Her area of interest is Image Processing and fuzzy logic.

29

Neural network based numerical digits recognization using nnt in matlab

Recommended

More Related Content

What's hot (20)

Similar to Neural network based numerical digits recognization using nnt in matlab (20)

More from ijcses (14)

Recently uploaded (20)

Neural network based numerical digits recognization using nnt in matlab