The Hitchhiker’s Guide to Bias and Fairness in Facial Affective Signal Processing: Overview and techniques

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

The Hitchhiker’s Guide to Bias and Fairness in Facial Affective Signal Processing: Overview and techniques

Jiaee Cheong; Sinan Kalkan; Hatice Gunes

Given the increasing prevalence of facial analysis technology, the problem of bias in the tools is now becoming an even greater source of concern. Several studies have highlighted the pervasiveness of such discrimination, and many have sought to address the problem by proposing solutions to mitigate it. Despite this effort, to date, understanding, investigating, and mitigating bias for facial affect analysis remain an understudied problem. In this work we aim to provide a guide by 1) providing an overview of the various definitions of bias and measures of fairness within the field of facial affective signal processing and 2) categorizing the algorithms and techniques that can be used to investigate and mitigate bias in facial affective signal processing. We present the opportunities and limitations within the current body of work, discuss the gathered findings, and propose areas that call for further research.


Facial analysis, including identity recognition, facial attribute classification (e.g., age, gender, and race), and affect prediction from facial images, has been widely studied in the literature, with increasing applications in various domains ranging from medicine, marketing, and surveillance [1]. With advances in machine learning, facial analysis is now dominantly performed using learning-based approaches. However, these data-driven, data-hungry, and black-box learning-based approaches make facial analysis biased and unfair toward certain demographic groups. Left unaddressed, bias can lead to concerning unfairness issues, such as misidentifying people of a certain demographic group (e.g., female African-American, would be misclassified as male more often than someone of another demographic group (e.g., male Caucasian as female) [1], or predicting a higher probability of criminality for certain groups of people [2]. A biased depression analyzer ( or chronic pain detector  ( can have serious consequences if such technology is incorporated in the health-care domain. Given the far-reaching, real-life consequences that such threats pose, this issue has become a major concern. Within the public, this pressing issue has elicited widespread activism that eventually prompted governmental institutes such as the European Commission to set up a regulatory framework to address it [3].

SPS on Twitter

  • The IEEE Journal of Selected Topics in Signal Processing is now accepting papers for a Special Issue on Learning-Ba…
  • Last chance to vote for your favorite video in the IEEE SPS 5-Minute Video Contest! Check out the videos from our t…
  • CALL FOR PROPOSALS: The IEEE Workshop on Automatic Speech Recognition and Understanding is now soliciting proposals…
  • authors have started uploading their conference slides and posters to IEEE SPS SigPort! Get a sneak pea…
  • DEADLINE EXTENDED: The IEEE Journal of Selected Topics in Signal Processing is accepting papers for a Special Issue…

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar