Facial Expression Recognition Using Local Binary Pattern and Support Vector Machine

International Journal of Innovative Research in Advanced Engineering (IJIRAE) ISSN: 2349-2763
Issue 06, Volume 3 (June 2016) www.ijirae.com
________________________________________________________________________________________________
IJIRAE: Impact Factor Value – SJIF: Innospace, Morocco (2015): 3.361 | PIF: 2.469 | Jour Info: 4.085 |
Index Copernicus 2014 = 6.57
© 2014- 16, IJIRAE- All Rights Reserved Page -103
Facial Expression Recognition Using
Local Binary Pattern and Support Vector Machine
Nivedita Chitra
Department of Electronics and Communication Engineering, MRIU, Faridabad, India
Geeta Nijhawan
Department of Electronics and Communication Engineering, MRIU, Faridabad, India
Abstract— Facial expression analysis is a remarkable and demanding problem, and impacts significant applications
in various fields like human-computer interaction and data-driven animation. Developing an efficient facial
representation from the original face images is a crucial step for achieving facial expression recognition. Facial
representation based on statistical local features, Local Binary Patterns (LBP) is practically assessed. Several
machine learning techniques were thoroughly observed on various databases. LBP features- which are effectual and
competent for facial expression recognition are generally used by researchers Cohn Kanade is the database for
present work and the programming language used is MATLAB. Firstly, face area is divided in small regions, by
which histograms, Local Binary Patterns (LBP) are extracted and then concatenated into single feature vector. This
feature vector outlines a well-organized representation of face and is helpful in determining the resemblance among
images.
Keywords— Face Recognition, Facial Expression Recognition, Local Binary Pattern, Principal Component Analysis,
Support Vector Machine.
I. INTRODUCTION
In the present day scenario, lot of research is going in the field of face recognition. The aim of face recognition is to
match the input image and the image stored in database [1]. Face recognition has played a vital role in solving the
problem of criminal identification, human computer interaction. The procedure of dimensionality reduction is a crucial
phase in face recognition task where the data have an inherent high dimensionality. For diminishing the dimensionality
of image space, PCA is applied [2]. The representation of face by means of Gabor features has caused huge interest in
computer vision, image processing, and pattern recognition and further. The major inspiration for using the Gabor filter
is its biological significance. The accessible field profile of neurons in the principal visual cortex of mammals is
oriented having the features of spatial frequencies. Gabor filter can also make use of prominent visual properties such as
spatial frequency feature, orientation selectivity and spatial localization [3]. Face conveys emotions more rapidly than
people realize their feelings. Facial expression reveals one's emotions and provides essential communicative indications
throughout social communication. This shows that facial expression figures a foremost modality. This research turns out
to be progressively more admired in computer vision or in area of robotics. Face recognition and identification have
applications like image retrieval, surveillance and automated access control. The analysis of facial expressions has
extensive variety of applications such as image understanding, psychological area, face animation, human computer
interaction (HCI), etc. Interaction takes place both verbally and non-verbally in humans. The automatic facial
expression recognition comprises of three main components for attaining this objective. Firstly, the face is identified
and established. Facial features are extracted from the identified face region. Lastly, the facial expression category is
classified on the basis of extracted features. For identification of face regions from image, various practices are
implemented now-a-days [4]. A human face has important and exclusive features due to which the detection of
expressions of human faces is possible. The transformation that takes place in answer to human internal emotional
status is referred as facial expression recognition (FER). This make use of diverse applications of Human Computer
Interaction (HCI) like facial video surveillance system, facial animation and face image processing in the area of
Artificial Intelligence, computer vision and digital image processing. For the past few years Automatic Recognition has
become an area of great concern and is a difficult task too. The feature extraction phase plays an important role in FER.
The facial expression, puts 55% to the total communication, while verbal and vocal contributes 7% and 38% in given
order [5]. Facial expression conveys rich information regarding human emotions and plays vital role in human
communication. They regulate to smoothen the process for more intellectual communication [6]. Regular facial
expression recognition usually comprises of three fundamental steps: face acquisition, facial feature extraction and
facial expression classification. Face acquisition is a preprocessing phase to perceive or establish face from the entered
series or images. The most extensively preferred face detector is the real-time face detection algorithm. Different
classifiers are used which have Harr-wavelet characteristics. Once the face is identified in database images, the
equivalent face regions are frequently standardized to have the same gray level and same eye distance. The facial
characteristics withdrawal efforts are done to discover the most suitable illustration of facial images for detection. There
are mainly two approaches: appearance features-based systems and geometric features-based systems.

________________________________________________________________________________________________
In the Geometric features-based systems, the location and shape of main facial features such as nose, mouth, eyes and
eye brows are identified in the images. However, the geometric features-based systems oblige the precise and consistent
facial characteristics recognition, which is complicated to comprehend in real-time application. While the appearance
features-based systems, changes appearance of the facial images such as wrinkles and bulges [7]. Since 1872, the efforts
of Darwin have fascinated great awareness of automatic facial expression to behavioral scientists. In 1978, Suwa et al.
was the first to challenge automatic facial expression commencing image sequences for analysis [8].
II. FACIAL REPRESENTATION
For extortion of facial appearance changes, Linear Discriminate Analysis [LDA], Gabor wavelet analysis, Principal
Component Analysis [PCA] and Independent Analysis [ICA] are applied to specific face regions or sometimes the
whole face. Donato et al. investigate various methods to characterize face images for facial action recognition, which
comprises of Local Feature Analysis (LFA), ICA, local schemes like Gabor-wavelet representation and local principal
components, PCA and LDA. Excellent results were achieved by using ICA and Gabor-wavelet representation. Gabor-
wavelet representations have been extensively implemented in face image analysis because of their better
representation. Liao et al. launched a progressed LBP operator for extortion of features in gradient maps and intensity
for facial expression recognition; in addition they tested their techniques on facial images of abridged resolution.
Though, their trial was tested on a awfully undersized database. The encouraging work on real-time squashed video
sequences demonstrated various applications in real-world environments [8].
III.FACE RECOGNITION
The principal phase which is considered in this work is Face Recognition. Therefore, it is separated in three steps -
1. Gabor filtering
2. Principal component analysis
3. Biogeography based optimization.
1. GABOR FILTERING:
Gabor filters are usually known as Gabor wavelets or kernels. They are the composite band pass filters. The shape of
the cells of the visual cortex of mammalian brains is similar to the shape of Gabor filters. These are applied in numerous
applications like extraction of multi resolution, spatially local features of a confined frequency band [1].
A well planned Gabor filter bank is able to detain relevant frequency spectrum within all directions. It can be taken in
use like a feature since this consists of information of the edge locations and additional particulars in images. Amplitude
on each pixel can be taken in use as feature because it consist a number of oriented frequency spectrum on each point of
image. Various important features can also be extracted from this with the help of Gabor filters [4]. The output image of
Gabor filter can be described as relationship of input image I(x) with Gabor kernel Pk(x)
ak(x0) = ∫∫ I (x)Pk(x − x0) dx ............. (1)
2. PRINCIPAL COMPONENT ANALYSIS:
The primary step in face recognition is extraction of features and standard technique applied to it is PCA. Principal
component analysis (PCA) is a well-recognized and extensively utilized linear dimensionality reduction technique. PCA
intends to generate a dimensional representation of high dimensional data so as to conserve the utmost foundations of
variation in the data set. This is attained by performing the linear transformation of the data, projecting it onto the axes
of greatest variance, known as principal components [9].
3. BIOGEOGRAPHY BASED OPTIMIZATION:
The study of geographical allocation of biological organism is known as biogeography. This seeks to find out how
species moves around from one island to another in search of shelter. Biogeography is articulated in terms of many
factors like immigration rate, emigration rate, habitat area, Habitat Suitably Index (HSI) and Suitably Index Variable
(SIV). Every individual is judged as a ''habitat'' among a habitat suitably index to determine individual's HSI. Those
variables of an individual through which the habitability can be characterized are Suitability Index Variable (SIV). An
excellent resolution constantly has a high HSI, while a deprived resolution has a low HSI. Features are also shared by
the good resolution with the poor one through migration. [1]

________________________________________________________________________________________________
IV.FACE EXPRESSION RECOGNITION
Figure 1 gives the various steps involved in face expression recognition:
Figure 1: Steps for face expression recognition
(i) PREPROCESSING :
A very significant step in facial expression recognition is preprocessing procedure. The initial step in processing is to
acquire pure facial expression images, which has normalized intensity, uniform shape and size. The effect of
illumination and lighting should also be removed by it [6]. First the original images are transformed into gray scale
images. Positioning the centers of eyes on every face, the entire images are appropriately rotated, scaled, translated and
cropped to 100×100 pixels. Images are subsequently subjected to a few image pre-processing operations. The image
pre-processing step consists of contrast and illumination equalization, fuzzy filtering and histogram equalization which
are described below:
CONTRAST AND ILLUMINATION EQUALIZATION: Contrast is an assess of human visual system sensitivity. The job of
contrast is important in visual processing of computer displays. The entire ancient times literatures, tackles the face
recognition process in various lightening conditions among various illumination and contrast. For acquiring an efficient
and psychologically-meaningful representation, all images are practiced with similar rms contrast and illumination.
FUZZILY SKEWED FILTERING: At times images get corrupted by different sources of noise. The particulars of images
which correspond to high frequencies mix up with noise. Therefore, to eliminate some noise effect in images low-pass
filters are used.
HISTOGRAM EQUALIZATION: Due to restrictions of the lighting conditions, the face images might be of poor contrast.
Hence, histogram equalization is used to pay off for the lighting conditions and to develop the contrast of images. [3]
(ii) FACE DETECTION:
Detecting a face in an image has to decide which pixels in the image is part of the face and which are not. Methods that
focus on facial landmarks (such as eyes, nose etc) that detect face-like colors in circular region or that use standard
feature templates, are used to detect faces. In face detection, it has to choose which pixels in images are the part of face
and which are not the part of it [10].
(iii) FEATURE EXTRACTION:
The extraction of the feature matrix is the primary step in any face recognition system. A characteristics feature
extraction algorithm, construct a computational model with a few linear or non-linear data. Hence, to make the
extracted feature as representative as possible or when the input data to an algorithm is very big to be processed and is
supposed to be infamously outmoded then the input data will be changed to a concentrated representation set of
features. Transformation of input data in set of features is known as feature extraction. When the extracted features are
cautiously selected, then it is estimated that features set will distort the appropriate detail from input data in order to
carry out preferred work using the reduced representation in place of full-size input. However, if there is non-
availability of specialist knowledge, general dimensionality reduction methods may be used.
These consist of:
 geometrical features extraction
 statistical (algebraic) features extraction [11]

________________________________________________________________________________________________
(iv) CLASSIFICATION:
For classification of expressions such as happy, surprise, anger, sad, fear, disgust, neutral, etc. multiclass SVM is taken
in use. Seven classes are used here to classify data. For conversion of Gabor features to vector form SVM is used. When
the image for test is given as input, Gabor is created in support of that image, and then get transformed into a vector.
The data is divided in two parts in SVM - Training set and Testing set, each comprising the illustration of attribute.
Every example is containing one target means class label and some attributes. The objective of SVM is to manufacture
the model that could foretell the target value of illustrations in testing sets that are only given by attributes value. It is
based on organized learning techniques. Support Vector Machine has an exclusive property through which it builds the
set of hyper plane or a hyper plane that has been take in concern for classification. This describes the functional margin
which means a fine division is attained by hyper plane that have the largest distance to the closest data of all classes.
Larger the margin would be, easier it would be to find accuracy and minimal errors [12].
(v) LOCAL BINARY PATTERN:
Many techniques are used for the extraction, which is the most important step to carry out face recognition. Local
Binary Pattern (LBP) is also one of the methods used for the feature extraction. A relatively new approach came
forward through Ojala et al. in 1996. Through LBP it becomes achievable for the illustration of shape and texture. It is
done by the separation of image into many small regions commencing the feature extraction. These features comprises
of binary patterns so as to express the surroundings of pixel in region. The images obtained can be distinguished by
evaluating the corresponding distance between their histograms. As per observations from many studies it is concluded
that face recognition shows extremely high quality results in terms of both speed and discrimination concert. The LBP
operator works among eight neighbors of pixel, by means of center pixel's value as threshold. When the neighbor pixel
is having gray value higher than center pixel than the assigned value of that pixel, gets one. The same case happens
when the values are same. Then the LBP code is created by concatenation of zeroes to a binary code as shown in Figure
2 or eight ones for the center pixel. In Figure 3 original and cropped images are shown [13].
Figure 2: The LBP operator
Figure 3: The original face and cropped face image
(vi) SUPPORT VECTOR MACHINE: SVM also being an influential machine learning method or technique for the data
classification, it presents mapping of data into a elevated dimensional feature space, and later discovers a linear
separation of hyper plane among the maximum margin to divide data in the given higher dimensional space.
Binary decisions are made by SVM, due to which the multi-class classification is proficient, this technique
instructs binary classifiers to differentiate one expression as of the entire, and yields the class of largest yield of
binary classification [8].

________________________________________________________________________________________________
(vii) The prepared data by SVM algorithms is divided in feature space by a hyper plane which is characterized through
the kind of kernel function used. They come across the hyper plane of maximal margin. The dimensions of the
margin limit the complications of hyper plane function. These are effectively used in number of categorization
jobs like text categorization, face detection and genetic analysis. They presently do better than artificial neutral
networks in a diversity of applications.
V. RESULTS
Table 1 and Table 2 give the result.
TABLE 1: RECOGNITION SCORE CALCULATED IN 5 SUBJECTS
SUBJECTS RESULTS RECOGNITION RESULT (%) CONFUSION RESULT (%)
S1 6/6 100 0
S2 6/6 100 0
S3 4/6 67 33
S4 5/6 83 17
S5 5/6 83 17
TABLE 2: CONFUSION MATRIX OF 7- CLASS FACIAL EXPRESSION RECOGNITION
Neutral
(%)
Angry
(%)
Surprise
(%)
Happiness
(%)
Sadness
(%)
Disgust
(%)
Fear
(%)
NEUTRAL 93.75 6.25 0 0 0 0 0
ANGRY 0 100 0 0 0 0 0
SURPRISE 0 0 100 0 0 0 0
HAPPINESS 50 0 0 50 0 0 0
SADNESS 0 0 0 0 100 0 0
DISGUST 0 0 0 0 0 100 0
FEAR 8.33 0 0 0 0 0 91.66
VI. CONCLUSION:
Through this work an inclusive practical study of facial expression recognition based on LBP features is presented. It
primarily comprises of three parts, i.e. face representation, feature extraction, and classification. Face representation
shows how to mock-up the face and resolves the succeeding algorithms of recognition and detection. The distinctive
features of face images are extracted in feature extraction step. In the categorization or classification, the face image is
evaluated among the images present in database. Face recognition gives approximately 90% of exactness; on the other
hand face expression recognition also lies in the similar range of 90% accuracy.
VI. REFERENCES
[1]. Daya Gupta, LavikaGoel “An Efficient Biogeography based Face Recognition Algorithm” 2nd International
Conference on Advances in Computer Science and Engineering (CSE 2013).
[2]. D Murugan, Dr. S Arumugam, K Rajalakshmi ''Performance evaluation of face recognition using Gabor filter, Log
Gabor filter and Discrete wavelet transform'' International Journal of computer science and information
technology(IJCSIT), Vol. 2, No. 1, February 2010.
[3]. Al-Amin Bhuiyan, and Chang Hong Liu '' On Face Recognition using Gabor Filters'' International Journal of
Computer, Electrical, Automation, Control and Information Engineering Vol:1, No:4, 2007.
[4]. Praseeda Lekshmi.V , Dr.M. Sasikumar '' Analysis of Facial Expression using Gabor and SVM'' International
Journal of Recent Trends in Engineering, Vol. 1, No. 2, May 2009.
[5]. Jaimini Suthar, Narendra Limbad'' A Literature Survey on Facial Expression Recognition techniques
usingAppearance based features'' International Journal of Computer Trends and Technology (IJCTT) – volume 17
Number 4 Nov 2014.
[6]. Hong-Bo Deng, Lian-Wen Jin, Li-Xin Zhen, Jian-Cheng Huang “A New Facial Expression Recognition Method
Based on Local Gabor Filter Bank and PCA plus LDA” International Journal of Information Technology Vol. 11
No. 11 2005.
[7]. Xiaoming Zhao and Shiqing Zhang '' Facial expression recognition using local binary patterns and discriminant
kernel locally linearembedding'' Zhao and Zhang EURASIP Journal on Advances in Signal Processing 2012.

________________________________________________________________________________________________
[8]. Caifeng Shan , Shaogang Gong , Peter W. McOwan “Facial expression recognition based on Local Binary
Patterns” Image and Vision Computing 27 (2009) 803–816.
[9]. Shiqing Zhang , Xiaoming Zhao , Bicheng Lei '' Facial Expression Recognition Based on Local Binary Patterns
andLocal Fisher Discriminant Analysis'' WSEAS TRANSACTIONS on SIGNAL PROCESSING E-ISSN: 2224-
3488 Issue 1, Volume 8, January 2012.
[10]. Ambika Ramchandra, Ravindra Kumar ''Overview of Face Recognition System Challenges'' international journal
of scientific & technology research volume 2, issue 8, august 2013.
[11]. Er.Navdeep Kaur Johal Er.Poonam Gupta Er. Amanpreet Kaur' Face Recognition Using Biogeography
BasedOptimization'' (IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 5,
May 2011.
[12]. Sisodia, Priya, AkhileshVerma, and SachinKansal. "Human Facial Expression Recognition using Gabor Filter
Bank with Minimum Number of Feature Vectors” International Journal of Applied Information Systems, Volume
5 – No. 9 July 2013 pp. 9-13.
[13]. Md. Abdur Rahim, Md. Najmul Hossain, Tanzillah Wahid & Md. ShafiulAzam “Face Recognition using Local
Binary Patterns (LBP)” Global Journal of Computer Science and Technology Graphics & Vision Volume 13 Issue
4 Version 1.0 Year 2013.

Facial Expression Recognition Using Local Binary Pattern and Support Vector Machine

More Related Content

What's hot (18)

Viewers also liked (19)

Similar to Facial Expression Recognition Using Local Binary Pattern and Support Vector Machine (20)

More from AM Publications (20)

Recently uploaded (20)

Facial Expression Recognition Using Local Binary Pattern and Support Vector Machine