The Importance of Context in Dialog Understanding
Context is crucial for any task related to understanding conversations. Whether it’s relying on previous exchanges, visual cues from the user’s screen, or background signals like alarms or music, context plays a crucial role.
Introducing MARRS: The Multimodal Reference Resolution System
MARRS, or Multimodal Reference Resolution System, is an on-device framework within a Natural Language Understanding system. Its main responsibility is to handle conversational, visual, and background context.
How MARRS Works with Machine Learning Models
MARRS utilizes different machine learning models to handle contextual queries. One model is used for reference resolution, while another is designed to handle context through query rewriting. These models work together to create a unified and lightweight system that can understand context while prioritizing user privacy.
*All authors listed have contributed equally to this work