Video of the Month: A Mobile Game with Physiologically Aware Virtual Humans

You are here

Inside Signal Processing Newsletter Home Page

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

News and Resources for Members of the IEEE Signal Processing Society

Video of the Month: A Mobile Game with Physiologically Aware Virtual Humans

Researches from the University of Barcelona investigate the interaction with affective and physiologically aware virtual human characters for mood and physiological modulation.

Dancing with Physio: A Mobile Game with Physiologically Aware Virtual Humans.

Courtesy of the researchers.

Depression and anxiety are common mental disorders. According to the World Health Organization (WHO), around 14% of the world population suffers from at least one severe mood disorder during the course of their life. In Europe, depression and anxiety, affects around 25% of the population, cause up to 50% of the total chronic sick leaves, and cost about €170 billion per year [1]. One paradox of this global problem is that these disorders can be treated; however, as reported by WHO, more than 50% of the people suffering from these disorders do not receive treatment. Four factors contributing to this incongruity have already been identified in [2]: the cost of medication or therapy, an insufficient number of mental health professionals, difficulty to access mental health services (due to distance or time), and reluctance of patients to look for professional help. Currently, research organizations, like the National Institute of Mental Health in the United States of America see smartphones, wearable sensors and video games as behavioral intervention technologies that can support and facilitate the access to treatment. Nonetheless, further investigation is needed to understand the capabilities of these technologies and how they can be used more effectively.

Dr. Jorge Arroyo Palacios and Prof. Mel Slater, from the EVENTLAB, University of Barcelona, contribute to these research efforts by investigating the feasibility of interaction with physiologically aware virtual human characters in a mobile game context as one approach to mood and physiological modulation. Their study combines two areas of research: interaction with affective virtual humans and biofeedback. They developed an interactive game system that simulates a dancing scenario for mobile platforms (iOS and Android). The interaction method between the participants and the virtual humans consisted of bidirectional physiological and mood contagion. The participants were told that they would have to change their own psychophysiological state to influence the mood of the virtual humans, and at the same time the observable changes in the affective state and behavior of the virtual characters were aimed to reinforce the psychophysiological changes of the participants.

The video illustrates how users interact with physiologically aware virtual humans in a mobile game, as an approach to modulate the participant’s affective and physiological state. The mobile game presents a virtual scenario where the participants are able to interact with virtual human characters through their psychophysiological activity. Music is played in the background of the scenario and, depending on the game mode, the virtual humans are initially either barely dancing or dancing very euphorically. The task of the participants is to encourage the apathetic virtual humans to dance or to calm down the frenetically dancing characters by modulating their own mood and physiological activity.

The results from their study show that by using this mobile game with the physiologically aware and affective virtual humans, the participants were able to emotionally arouse themselves in the Activation condition and were able to relax themselves in the Relaxation condition, during the same session, with only a brief break between conditions. The self-reported affective data was also corroborated by the physiological data (heart rate, respiration and skin conductance).

“We believe that the expanding use of wearable computing and home health technologies makes this a propitious moment to exploit this proposed natural and intuitive human-computer psychophysiological interaction method. Moreover, the current focus of companies like Apple, Google, Microsoft and Samsung on wearable devices physiological sensors, and software platforms, offers a promising panorama for the development of accessible tools to support the treatment of people with mood disorders”, commented Dr. Arroyo Palacios (former member of the EVENTLAB and currently a postdoctoral scholar at the University of California San Francisco) and Prof. Slater (ICREA Research Professor at the University of Barcelona and leader of the EVENTLAB).

For more details, please visit the EVENTLAB or read their recent paper.


[1] WHO, “Depression in Europe: facts and figures,” 2014. [Online]. Available:

[2] NIMH, “National Institute of Mental Health Strategic Plan for Research.” National Institute of Mental Health, NIH, 2015.

SPS on Twitter

  • DEADLINE EXTENDED: The 2023 IEEE International Workshop on Machine Learning for Signal Processing is now accepting…
  • ONE MONTH OUT! We are celebrating the inaugural SPS Day on 2 June, honoring the date the Society was established in…
  • The new SPS Scholarship Program welcomes applications from students interested in pursuing signal processing educat…
  • CALL FOR PAPERS: The IEEE Journal of Selected Topics in Signal Processing is now seeking submissions for a Special…
  • Test your knowledge of signal processing history with our April trivia! Our 75th anniversary celebration continues:…

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar