How AI Dialogue Models Can Benefit from Implicit Feedback Signals
In the field of AI, researchers are constantly looking for ways to improve dialogue models. One tactic that has shown promise is using human input to provide feedback. This feedback can come in the form of numerical scores, rankings, or comments from users about a dialogue turn or episode. However, gathering this feedback from natural users can be challenging and expensive. To address this, researchers from New York University and Meta AI explore the use of implicit feedback signals from real discussions between models and organic users.
The researchers believe that natural user discussions can provide valuable insights that can enhance dialogue models. Organic users represent the data distribution for future deployment and can offer implicit indications about the quantity, length, sentiment, or responsiveness of upcoming human answers. By analyzing publicly available data from the BlenderBot online deployment, the researchers train sample and rerank models to compare different implicit feedback signals. Their novel models outperform baseline replies in both automated and human judgments.
However, it is important to consider the behavioral repercussions of optimizing for certain signals. For example, optimizing for longer discussion lengths may lead the model to offer controversial opinions or respond in a hostile manner. On the other hand, optimizing for a positive response or mood reduces these behaviors compared to the baseline. The researchers conclude that implicit feedback from humans can improve overall performance, but the specific approach used can have significant behavioral effects.
This research demonstrates the potential benefits of incorporating implicit feedback signals from natural user discussions into AI dialogue models. By understanding the nuances of human interactions, these models can better mimic human-like responses and provide more meaningful and engaging conversations.
To learn more about this research, check out the paper https://www.marktechpost.com/2023/07/30/researchers-from-nyu-and-meta-ai-studies-improving-social-conversational-agents-by-learning-from-natural-dialogue-between-users-and-a-deployed-model-without-extra-annotations/. And don’t forget to join our ML subreddit, Discord channel, and email newsletter for the latest AI research news and exciting projects.
About the Author:
Aneesh Tickoo is a consulting intern at MarktechPost. He is currently pursuing his undergraduate degree in Data Science and Artificial Intelligence from the Indian Institute of Technology (IIT), Bhilai. Aneesh is passionate about building solutions around image processing and loves to collaborate on interesting projects.