Video of the Month: A Mobile Game with Physiologically Aware Virtual Humans

Researches from the University of Barcelona investigate the interaction with affective and physiologically aware virtual human characters for mood and physiological modulation.

Dancing with Physio: A Mobile Game with Physiologically Aware Virtual Humans.

Courtesy of the researchers.

Depression and anxiety are common mental disorders. According to the World Health Organization (WHO), around 14% of the world population suffers from at least one severe mood disorder during the course of their life. In Europe, depression and anxiety, affects around 25% of the population, cause up to 50% of the total chronic sick leaves, and cost about €170 billion per year [1]. One paradox of this global problem is that these disorders can be treated; however, as reported by WHO, more than 50% of the people suffering from these disorders do not receive treatment. Four factors contributing to this incongruity have already been identified in [2]: the cost of medication or therapy, an insufficient number of mental health professionals, difficulty to access mental health services (due to distance or time), and reluctance of patients to look for professional help. Currently, research organizations, like the National Institute of Mental Health in the United States of America see smartphones, wearable sensors and video games as behavioral intervention technologies that can support and facilitate the access to treatment. Nonetheless, further investigation is needed to understand the capabilities of these technologies and how they can be used more effectively.

Dr. Jorge Arroyo Palacios and Prof. Mel Slater, from the EVENTLAB, University of Barcelona, contribute to these research efforts by investigating the feasibility of interaction with physiologically aware virtual human characters in a mobile game context as one approach to mood and physiological modulation. Their study combines two areas of research: interaction with affective virtual humans and biofeedback. They developed an interactive game system that simulates a dancing scenario for mobile platforms (iOS and Android). The interaction method between the participants and the virtual humans consisted of bidirectional physiological and mood contagion. The participants were told that they would have to change their own psychophysiological state to influence the mood of the virtual humans, and at the same time the observable changes in the affective state and behavior of the virtual characters were aimed to reinforce the psychophysiological changes of the participants. The video illustrates how users interact with physiologically aware virtual humans in a mobile game, as an approach to modulate the participant’s affective and physiological state. The mobile game presents a virtual scenario where the participants are able to interact with virtual human characters through their psychophysiological activity. Music is played in the background of the scenario and, depending on the game mode, the virtual humans are initially either barely dancing or dancing very euphorically. The task of the participants is to encourage the apathetic virtual humans to dance or to calm down the frenetically dancing characters by modulating their own mood and physiological activity. The results from their study show that by using this mobile game with the physiologically aware and affective virtual humans, the participants were able to emotionally arouse themselves in the Activation condition and were able to relax themselves in the Relaxation condition, during the same session, with only a brief break between conditions. The self-reported affective data was also corroborated by the physiological data (heart rate, respiration and skin conductance).

“We believe that the expanding use of wearable computing and home health technologies makes this a propitious moment to exploit this proposed natural and intuitive human-computer psychophysiological interaction method. Moreover, the current focus of companies like Apple, Google, Microsoft and Samsung on wearable devices physiological sensors, and software platforms, offers a promising panorama for the development of accessible tools to support the treatment of people with mood disorders”, commented Dr. Arroyo Palacios (former member of the EVENTLAB and currently a postdoctoral scholar at the University of California San Francisco) and Prof. Slater (ICREA Research Professor at the University of Barcelona and leader of the EVENTLAB).

For more details, please visit the EVENTLAB or read their recent paper.

References: [1] WHO, “Depression in Europe: facts and figures,” 2014. [Online]. Available: http://www.euro.who.int/en/health-topics/noncommunicable-diseases/pages/.... [2] NIMH, “National Institute of Mental Health Strategic Plan for Research.” National Institute of Mental Health, NIH, 2015.