AgentGraph: Toward Universal Dialogue Management With Structured Deep Reinforcement Learning

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

AgentGraph: Toward Universal Dialogue Management With Structured Deep Reinforcement Learning

By: 
Lu Chen; Zhi Chen; Bowen Tan; Sishan Long; Milica Gašić; Kai Yu

Dialogue policy plays an important role in task-oriented spoken dialogue systems. It determines how to respond to users. The recently proposed deep reinforcement learning (DRL) approaches have been used for policy optimization. However, these deep models are still challenging for two reasons: first, many DRL-based policies are not sample efficient; and second, most models do not have the capability of policy transfer between different domains. In this paper, we propose a universal framework, AgentGraph , to tackle these two problems. The proposed AgentGraph is the combination of graph neural network (GNN) based architecture and DRL-based algorithm. It can be regarded as one of the multi-agent reinforcement learning approaches. Each agent corresponds to a node in a graph, which is defined according to the dialogue domain ontology. When making a decision, each agent can communicate with its neighbors on the graph. Under AgentGraph framework, we further propose dual GNN-based dialogue policy, which implicitly decomposes the decision in each turn into a high-level global decision and a low-level local decision. Experiments show that AgentGraph models significantly outperform traditional reinforcement learning approaches on most of the 18 tasks of the PyDial benchmark. Moreover, when transferred from the source task to a target task, these models not only have acceptable initial performance but also converge much faster on the target task.

SPS on Twitter

  • We are happy to welcome Prof. Jiebo Luo as the new Editor-in-Chief of IEEE Transactions on Multimedia beginning in… https://t.co/9ZgBrgkFXv
  • wants your talents! Our tenure-track position in engineering applications of information and data science a… https://t.co/QrqTAFGlyM
  • If you’re missing out on , don’t worry - we’ll be tweeting all week long. Follow along with us to see the action!

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar