Facial expression recongnition Techniques, Database and Classifiers

ISSN 2347-6788 International Journal of Advances in Computer Science and Communication Engineering (IJACSCE)
Vol 2 Issue2 (June 2014)
www.sciencepublication.org
15
FACIAL EXPRESSION RECOGNITION
Techniques, Database & Classifiers
1
Rupinder Saini, 2
Narinder Rana
1,2
Rayat Institute of Engineering and IT
E-mail: errupindersaini27@gmail.com , narinderkrana@gmail.com
Abstract
Facial Expression Recognition (FER) is really a
speedily growing and an ever green research
field in the region of Computer Vision, Artificial
Intelligent and Automation. There are various
application programs which use Facial
Expression to evaluate human character,
feelings, judgment, and viewpoint. Recognizing
Human Facial Expression is not just an easy
and straightforward task due to sveral
circumstances like illumination, facial
occlusions, face shape/color etc. In this paper,
we present some method/techniques such as
Eigen face approach, principal component
analysis (PCA), Gabor wavelet, principal
component analysis with singular value
decomposition etc which will directly or/and
indirectly used to recognize human expression in
several situations.
Keywords
Techniques, Classifier, Face, Expression, PCA,
JAFFE.
1. Introduction
Expression is an important mode of non-verbal
conversation among people. Recently, the facial
expression recognition technology attracts more
and more attention with people’s growing
interesting in expression information. Facial
expression provides essential information about
the mental, emotive and in many cases even
physical states of the conversation. Face
expression recognition possesses practically
significant importance; it offers vast application
prospects, such as user-friendly interface
between people and machine, humanistic design
of products, and an automatic robot for example.
Face perception is an important component of
human knowledge. Faces contain much
information about ones id and also about mood
and state of mind. Facial expression interactions
usually relevant in social life, teacher-student
interaction, credibility in numerous contexts,
medicine etc. however people can easily
recognize facial expression easily, but it is quite
hard for a machine to do this.
2. Techniques
2.1 Eigen face approach
Eigen faces is the name provided to some of
eigenvectors when they are used in the computer
vision problem of human face recognition.
Eigenvector based features are extracted from
the pictures. Jeemoni Kalita and Karen Das
present a paper “Recognition Of Facial
Expression Using Eigenvector Based
Distributed Features And Euclidean Distance
Based Decision Making Technique” where they
present a method to design an Eigenvector based
facial expression recognition system to
recognize face expressions from digital facial
images. The Eigenvectors for the database
images and test images are extracted, computed,
and input facial images recognized when
similarity obtained by calculating the minimum
Euclidean distance between the test image and

16
the different expressions [1].The recognition rate
obtained for the proposed system is 95%.
2.2 Principal component analysis
Principal component analysis (PCA) involves
some sort of numerical procedure that changes
several (possibly) correlated variables into a
(smaller) number of uncorrelated variables
called principal components. (PCA) is a
technique of identifying patterns in data, and
expressing the data in such a way so as to
highlight their differences and similarities.
Akshat Garg and Vishakha Choudhary in
their paper “Facial Expression Recognition
Using Principal Component Analysis” use PCA
to recognize face expression. They find a subset
of principal directions (principal components)
from the set of training faces. Then project faces
into this principal components space and get
feature vectors. Comparison is performed by
calculating the distance between these kinds of
vectors. Generally comparison of face images is
carried out by computing the Euclidean distance
between these feature vectors [2].
2.3 Gabor Wavelet
The next technique introduces is Gabor wavelet.
Mahesh Kumbhar, Manasi Patil and Ashish
Jadhav proposed a paper “Facial Expression
Recognition using Gabor Wavelet” in which
they discusses the application of Gabor filter
based feature extraction by using feed-forward
neural networks (classifier) for recognition of
four different facial expressions. The
Recognition process start firstly by acquiring the
image using an image capturing device like a
camera. The image that is captured then required
to be preprocessed such that environmental and
other variations in different images are
minimized. The image preprocessing steps
comprises with operations like image scaling,
image brightness and contrast adjustment and
other image enhancement operations. Processing
is done on the same image for obtaining best
feature representation. Then these feature points
are selected. A discrete set of Gabor kernels is
applied to image. Convolution of real Gabor
with Image is taken over selected fiducial points
to generate feature vector. Length of feature
vector is reduced by using PCA. Reduced
feature vector is applied to NN classifier to get
the results [3]. Results obtained by using Gabor
wavelet for randomly selected images are
around 72.50%.
2.4 Principal Component Analysis with
Singular Value Decomposition
The next proposed technique is PCA with SVD
algorithm for classification of facial expressions.
Ajit P.Gosavi and S.R. Khot implements
hybrid facial expression recognition technique
using Principal Component analysis (PCA) with
Singular Value Decomposition (SVD) in their
paper “Facial Expression Recognition uses
Principal Component Analysis with Singular
Value Decomposition”. They performed
experiments on real database images. They used
universally accepted five principal emotions to
be recognized are: Happy, Disgust, Sad, Angry
and Surprise along with neutral. They used
Euclidean distance based matching Classifier for
finding the closest match. This algorithm can
effectively distinguish different expressions by
identifying features [4]. The average Accuracy
of the system obtained is about 89.70% and
65.42% average recognition rate for all five
principal emotions Happy, Disgust, Sad, Angry
and Surprise along with neutral.
2.5 Independent Component Analysis with
Principal Component Analysis
Roman W. ´Swiniarski1 and Andrzej
Skowron presents a paper ‘‘Independent
Component Analysis, Principal Component
Analysis and Rough Sets in Face Recognition’’

17
that contains description of hybrid methods of
face recognition which are based on independent
component analysis, principal component
analysis and rough set theory. Independent
Component Analysis and Principal Component
Analysis provide feature extraction and pattern
forming from face images. The feature se-
lection/reduction has been realized using the
rough set technique. Rough-sets rule based
classifier used to design face recognition system.
The rough sets rule based classifier provides
88.75% of classification accuracy for the test set
[5].
2.6 Local Gabor Binary Pattern
Appearence based features are useful for
encounter face identification because it encodes
certain information about human faces. In this
face image is divided into sub blocks and
similarities among the sub blocks is attained[5].
A significant advantage of Local Binary Pattern
(LBP) is its illumination tolerance. In Local
Gabor Binary Pattern(LGBP) method, for
generation of feature vectors, LBP is extracted
from gabor filters. LGBP achieves better
performance than gabor filter method [10]. S.
M. Lajevardi and H. R. Wu introduces a
Tensor Perceptual Color Framework (TPCF) in
their paper ‘‘Facial Expression Recognition in
Perceptual Color Space’’ where color image
components are horizontally unfolded to 2D
tensors using multilinear algebra and tensor
concepts. For feature extraction Log-gabor
filters are used as it overcome the limitations of
gabor filter based method. For feature selection
mutual information quotient method is used.
Multiclass linear discriminant analysis classifier
is used for classifying the selected features.
TPCF can efficiently recognize the facial
expressions under different illumination
conditions therefore overall performance could
be enhanced. [6].
2.7 Using SVM Classification in Perceptual
Color Space
In this module to perform automated expression
recognition, system requires to deal with the
issues of face localization, facial feature
extraction and training as well as the
classification stages of the SVM. Ms.
Aswathy.R in his paper ‘‘Facial Expression
Recognition Using SVM Classification in
Perceptual Color Space ‘introduces a new facial
expression recognition system which uses tensor
concept. A tensor perceptual color framework
for FER based on information contained in color
facial images is introduced. Perceptual color
space is used for improving the performance
instead of using RGB color space. Further the
classification is performed using support vector
machine just because the Support Vector
Machine (SVM )performed better than the other
classifiers and resolution of the face did not
affect the classification rate with the SVM [7].
2.8 Facial expression recognition using LBP
Caifeng Shan ,Shaogang Gong , Peter W.
McOwan in their paper , ‘‘Facial expression
recognition based on Local Binary Patterns: A
comprehensive study’’ used (Local Binary
Patterns) LBP features to perform person-
independent facial expression recognition He
used the concept of template matching . A
template is generated for each class of face
images, then to match the input image with the
closest template a nearest-neighbour classifier is
used. Firstly they adopted template matching to
classify facial expressions for its simplicity. In
training, the histograms of expression images in
a given test class were averaged to generate a
template for this class. The template matching
achieved the generalization performance of

18
79.1% for the 7-class task and 84.5% for the 6-class task [8].
3. Database
Name Image
Size
Color
Images
Number of
pictures per
person
Number of
unique people
Available
AR Face Database 576 x
768
Yes 26 126; 70 Male,
56 Female
Yes
Richard's MIT database 480 x
640
Yes 6 154; 82 Male,
72 Female
Yes
The MUCT Face Database 480 x
640
Yes 10-15 276 Yes
The Yale Face Database 320 x
243
No 11 15 Yes
The Japanese Female
Facial Expression (JAFFE)
Database
256 x
256
No 7 10 Yes
The University of Oulu
Physics-Based Face
Database
428 x
569
Yes 16 125 No - Cost
$50
FEI Face Database 640x480 Yes 14 200 Yes
4. Classifier
4.1 Euclidean Distance Classifier
Euclidean distance based classifier is used which
is obtained by calculating of distance between
image to test and available images that are taken
as training images. Using the given set of values
minimum distance can be found.
In testing, for every expression computation of
Euclidean distance (ED) is done between new
image (testing) Eigenvector and Eigen
subspaces, to find the input image expression
classification based on minimum Euclidean
distance is done. The formula for the Euclidean
distance is given by:
ED =
The recognition rate for the system proposed is
found to be 95%.
4.2 The Back propagation Algorithm
To design a class of feed forward networks with
layers called multilayer perceptrons (MLP)
algorithm called back-propagation is used. Its
input layer has source nodes and output layer is
of neurons and these layers connect the world
outside to the network easily. Along with these
layers it has other layers with hidden neurons are
there. These are hidden as are not accessible
directly. Features of input data are extracted by
hidden neurons. For images selected randomly
results are around 72.50%.
4.3 PCA
Gray-level pixel values in image when
concatenated give raw feature vector. Let us
suppose that given are m images and n pixel

19
values are there per image and Z be a matrix of
(m,n), where m is the number of images and n is
the number of pixels (raw feature vector). The
mean image from Z is subtracted from every
image from the training set, ∆ = −E [ ].
Let the matrix M is representing
resulting”centered” images; M
=(∆ ,∆ ,..∆ ) T. The covariance matrix
can then be represented as: Ω = M. . Ω is
symmetric and can be expressed in terms of the
singular value decomposition Ω = U.Λ. ,
where U is an m x m unitary matrix and Λ =
diag(λ1,...,λm). The vectors U1,...,Um are basis
for the m-dimensional subspace. The covariance
matrix can now be re-written as [9]:
Ω =m
The coordinate ζi, i ∈ 1,2,...m, is called the ζth i
principal component. It shows the projection of
∆Z onto the basis vector U. Principal
components of training set are vectors . After
constructing the subspace centered probe image
is projected into subspace for recognition. As a
match gallery image that is closest to the probe
is selected. Images are also cropped along with
normalization before PCA is applied, resulted
image being of size of 130x150. When image is
unwrapped a vector of size 19,500 is resulted.
PCA also reduces it to a basis vector count of
m−1; here m represents the number of images.
PCA approach drops a few vectors while face
recognition in order to form a face space.
Usually from the beginning it is small number
and from the end a larger number.
4.4 Distance Measure
Nearest neighbor classifier is simple method of
classification in 2-D face recognition. A label is
assigned to image from the probe set which is
also close in galley set. Evaluation of many
distance measures in the field of face recognition
has been done [12, 13]. In our experiments, we
use the Mah- Cosine distance metric [11]. Initial
experiments showed that MahCosine
outperformed the other used distance measures,
such as Euclidean or Mahalanobis distance
measures. When images are transformed to the
Mahalanobis space The MahCosine measure is
cosine of the angle between them [11].
Formally, the MahCosine between the images i
and j having projections a and b in the
Mahalanobis space is computed as:
MahCosine(i,j)=cos( ) =
4.5 Linear Discriminant Analysis (LDA)
To discriminate different subject’s projection is
achieved using LDA. Before using it,
dimensions can be reduced by using PCA. In
first d principal components a dimensional
subspace is defined and construction of
Fisherface is done [14]. In Fisher’s method the
projecting vectors W is so that its basis vectors
maximize the ratio between the determinants of
the inter-class scatter matrix and intra-class
scatter matrix .
W = argmax
Suppose number of subjects is m and the
number of images (samples) per subject
available for training to be , where i is the
subject index. Then and can be deﬁned as:
=
=
And where is the mean of vector of samples
belonging to the class (or subject) i, µ is the
mean vector of all the samples. When samples
are small in number may be less well
estimated.
5. Conclusion
In this paper, we observe many techniques such
as Eigen face PCA, Gabor wavelet, principal
component analysis with singular value
decomposition etc, with the use of appropriate
Datasets for detection of Human Facial
expression and their recognition based on
accuracy and computational time. Some
methods we see contain drawbacks as of
recognition rate or timing. To achieve accurate
recognition two or more techniques can be

20
combined, then features are extracted as per
need and to evaluate results final comparison is
done. The success of technique is dependent on
pre-processing of the images because of
illumination and feature extraction.
References
[1] Jeemoni Kalita and Karen Das, “Recognition Of Facial
Expression Using Eigenvector Based Distributed Features
And Euclidean Distance Based Decision Making
Technique”; International Journal of Advanced Computer
Science and Applications, Vol. 4, No. 2, 2013.
[10] Akshat Garg and Vishakha Choudhary, “Facial
Expression Recognition Using Principal Component
Analysis” ; International Journal of Scientific Research
Engineering &Technology (IJSRET)Volume 1 Issue4 pp
039-042 July 2012.
[2] Mahesh Kumbhar, Manasi Patil and Ashish Jadhav
,“Facial Expression Recognition using Gabor Wavelet” ;
International Journal of Computer Applications (0975 –
8887) Volume 68– No.23, April 2013.
[3] Ajit P.Gosavi and S.R. Khot ,“Facial Expression
Recognition uses Principal Component Analysis with
Singular Value Decomposition” ; International Journal of
Advance Research in Computer Science and Management
Studies Volume 1, Issue 6, November 2013.
[4] Roman W. ´Swiniarski1 and Andrzej Skowron,
‘‘Independent Component Analysis, Principal Component
Analysis and Rough Sets in Face Recognition’’.
[5] S. M. Lajevardi and H. R. Wu,“Facial Expression
Recognition in Perceptual Color Space”; IEEE
Transactions on Image Processing, vol. 21, no. 8, pp. 3721-
3732, 2012.
[6] Ms. Aswathy.R ‘‘Facial Expression Recognition Using
SVM Classification in Perceptual Color Space’’; IJCSMC,
Vol. 2, Issue. 6, June 2013, pg.363 – 368.
[7] Caifeng Shan ,Shaogang Gong , Peter W. McOwan
,‘‘Facial expression recognition based on Local Binary
Patterns:A comprehensive study’’; Image and Vision
Computing 27 (2009) 803–816.
[8] Nitesh V. Chawla and Kevin W. Bowyer, ‘‘Designing
Multiple Classiﬁer Systems for Face Recognition’’.
[9] S. Moore and R. Bowden, “Local binary patterns for
multi-view facial expression recognition”; Comput. Vis.
Image Understand., vol. 115, no. 4, pp. 541–558, 2011.

Facial expression recongnition Techniques, Database and Classifiers

More Related Content

What's hot (20)

Similar to Facial expression recongnition Techniques, Database and Classifiers (20)

Recently uploaded (20)

Facial expression recongnition Techniques, Database and Classifiers