Facial Expression Recognition from Masked Faces



Facial expression recognition helps to understand others’ internal states in human social activities. Generally, the facial expression is recognised by analysing face videos and/ or images. However, because of the spread of COVID-19, many people wear masks as a common anti-epidemic measure, and it leads to heavy loss of  facial information. The primary goal of the project is to build facial recognition computer vision model from the masked faces. To achieve this goal, the research will use the exposed part of face (above the nose bridge) image sequences and physiological signals (electroencephalogram, electrocardiogram, galvanic skin response, respiration pattern, skin temperature, and so on).


  1. Build a facial expression recognition deep learing model using whole face images/ videos.
  2. Build a facial expression recognition model using physiological signals.
  3. Build a facial expression recognition model based on masked faces.
  4. Do fusion to improve the model performance. 

Background Literature

  1. N Samadiani, G Huang, B Cai, W Luo, C Chi, Y Xiang, and J He (2019). A Review on Automatic Facial Expression Recognition Systems Assisted by Multimodal Sensor DataSensors (Basel), 19(8):1-27.
  2. G R Alexandre, J M Soares, and G A Pereira (2020). Systematic review of 3D facial expression recognition methodsPattern Recognition, 11(107108):1-16.
  3. L Ricciardi, F Visco-Comandini, R Erro, F Morgante, M Bologna, A Fasano, D Ricciardi, M J. Edwards, and J Kilner (2017). Facial Emotion Recognition and Expression in Parkinson’s Disease: An Emotional Mirror Mechanism?PLoS ONE, 12(1):1-16.

Updated:  1 June 2019/Responsible Officer:  Dean, CECS/Page Contact:  CECS Marketing