The aim of this project was to develop a computer vision system that
recognises facial expression from still images.
How are you feeling today?
Platform
The system was implemented in Matlab, using the Neural Networks
Toolbox, in Microsoft Windows Environment.
Data
The Yale
Face Database was used for training and testing the system. 60
greyscale images (15 human subjects, 4 images for each subject: normal,
sad, happy, surprise) were selected for the database for training and
testing the system.
Solution
A feed-forward back-propagation neural network solution was adopted,
inspired by the paper: "A Neural Network Facial Expression Recognition
System using Unsupervised Local Processing" by L. Franco, A.
Treves, in Proceedings of the 2nd International Symposium on Image and
Signal Processing and Analysis, ISPA 2001, Pula, Croatia.
Initially, a 96x32 pixel image of the left side of each facial image
was cropped. Then, PCA was used to reduce the dimensionality from 3072
to 55 elements per image. This data was used as input to train a
variety of Neural Networks, using different versions of the Back
Propagation algorithm (gradient descent, adaptive gradient descent,
gradient descent with momentum, adaptive gradiant descent with
momentum), with one or two hidden layers, with 3 or 4 outputs,
corresponding to 3 or 4 facial expressions.
Each time, the images of 14 human subjects were used for training and
the image of the remaining human subject were used to test the system.
Conclusions
Results were promising (48% in average for all implementations,
therefore better than luck), but inconsistent over different
implementation and/or different training sessions. The diagrams below
show
the best success rate achieved. "Happy" and "Surprised" expressions
were easiest to recognise, while "Normal" and "Sad" were confused often.
Contrasting
"Surprised" expressions
"Normal"
and "Sad" similarities
Neural
Networks with best results
over all expressions.
Neural
netwroks with best results for
particular expressions
About this work
The above system was implemented by Tom Botley for his final
year
project at Kingston University, under the supervision of
Dimitrios Makris,
during the academic year 2005-2006.
