NeurIPS 2019 ConvAI Workshop Recap

I recently attended the NeurIPS 2019 ConvAI Workshop held in Vancouver. The title of the workshop was "Today's Practice and Tomorrow's Potential" and in many ways the presentations and talks reflected this exact sentiment. While much progress has been made on the dialogue problem in recent years, conversational artificial intelligence remains notoriously difficult. This is because it synthesizes a number of different complex reasoning modalities from natural language understanding to acoustic prosody to natural language generation to common sense grounding. In isolation, any one of these tasks is incredibly challenging but in combination you get what truly amounts to an AI-complete problem. 

At the workshop, there were a number of papers that offered neat solutions to some of the biggest problems in dialogue. For example, Li et. al. developed a new evaluation technique in their "Acute-Eval: Improved dialogue evaluation with optimized questions and multi-turn comparisons" paper, whereby they had third-party evaluators compare pairs of system models rather than a single one, showing that this improved human correlation scores. 

The best paper award was given to Madotto et. al.'s "Attention Over Parameters For Dialogue Problems" which developed an elegant model for learning chit-chat and task-oriented skills when appropriate in a joint conversational setting. In many ways this award represented a broader desire by the community to ultimately merge the task-oriented and chit-chat conversational domains. This social/task-oriented merger was extensively discussed during breaks, and so I expect to see significantly more work on this front in the coming year. 

Many of the presentations during the workshop emphasized moving to what I call "higher-order" reasoning abilities in dialogue systems. These include things like multimodal awareness, negotiation, and emotional understanding. In his talk, Ryuichiro Higashinaka, described his group's work on developing naturalness and personality in chat-oriented systems. As part of this effort, he described new data-collection platforms whereby users are prompted to give more engaging responses through character role play. 

In a related talk, David Traum emphasized his group's work on multimodal environments whereby dialogue agents are able to incorporate multiple cues for informing their understanding and reasoning abilities. He also discussed ongoing work on developing non-cooperative and deceptive virtual agents, which is a particularly sophisticated form of human behavior to emulate in talking machines. While in some ways, one could argue that focusing on these more complex problems seems premature given that we haven't even successfully built a robust restaurant reservation bot, I think there is value in pursuing these higher order tasks because it will inform our understanding of how diverse conversational abilities are performed by humans. I'm confident these insights will inform our work on even the most "basic" restaurant reservation agents. 

When all is said and done, the NeurIPS workshop made it clear that while dialogue is still very much an unsolved problem, there is more excitement than ever around its challenges, and the community is anxious to tackle it from diverse angles. I'm eager to see the progress we make in the coming year!

***

Mihail Eric is a machine learning scientist at Amazon Alexa AI who is passionate about conversational AI. He can be reached via Twitter @mihail_eric.