Automated facial expression recognition – an integrated approach with optical flow analysis and Support Vector Machines

In this article, an automated facial expression recognition system capable of classifying expressions by analysing the shape of the strongest facial feature responsible for the expression, namely the eyebrows or the mouth region is presented. This novel approach integrates Lucas-Kanade method for optical flow analysis to determine the motion flow vectors between two neighbouring images from a video sequence and support vector machines (SVM) for expression classification. The expressions classified are anger, disgust, fear, sad, smile with open mouth and closed mouth, surprise and neutral expression. By using just the strongest facial feature to train 16 SVM classifiers optimises the expression recognition irrespective of the degree of expressiveness and makes the system less computationally expensive. The SVM has been trained with 300 faces (2,400 images) and evaluated using 80 faces (640 images) to produce an average classification accuracy of 95.5 and 92.8% using mouth and eyebrows, respectively. The simulation results of testing the data set with LibSVM tool based on sequential minimal optimisation (SMO) algorithm and iterative single data algorithm (ISDA) proves that ISDA is faster than other SMO-based algorithms while delivering the same generalisation results.