How to Teach AI Agents in New Ways
Reinforcement learning is a critical part of training an AI agent to perform new tasks. This means the agent is rewarded for taking actions that get it closer to a goal. Traditionally, a human expert creates a reward function that motivates the agent to explore. However, creating and updating this function is time-consuming and inefficient, making it difficult to scale up for complex tasks.
Researchers from MIT, Harvard University, and the University of Washington have developed a new approach to reinforcement learning that doesn’t rely on an expertly designed reward function. Instead, the method leverages feedback from nonexpert users to guide the agent. Even though this feedback may contain errors, this new approach enables the AI agent to learn more quickly than other methods.
With this new approach, feedback can be gathered from nonexpert users around the world, making the teaching process more scalable. This could be a game-changer for training robots to perform tasks in a user’s home, allowing the robots to explore and learn with the help of nonexpert feedback.
The researchers tested this method on both simulated and real-world tasks and found that it helped agents learn faster than other methods. In future work, the team wants to continue refining the method so that the agent can learn from various forms of communication and teach multiple agents at once.
This research is funded, in part, by the MIT-IBM Watson AI Lab. For more details, you can read the full paper, which will be presented at the Conference on Neural Information Processing Systems next month.
The Use of Crowdsourced Feedback in AI Learning
One way to gather user feedback for reinforcement learning is to show a user two photos of states achieved by the agent, and then ask the user which state is closer to a goal. Some previous approaches tried to use this feedback to optimize a reward function for the agent, but this can be noisy and lead the agent to get stuck and never reach its goal. The researchers of the new AI learning method, HuGE (Human Guided Exploration), have decoupled the process to allow for the continuous and asynchronous collection of feedback, making the learning process more effective and faster.
Faster Learning with HuGE
In simulated and real-world tests, HuGE helped agents learn to achieve their goals faster than other methods. This method also uses feedback crowdsourced from nonexperts to guide the agent’s behavior more effectively than synthetic data provided by the researchers.
What Lies Ahead
The researchers are interested in applying this method to teach multiple agents at once and want to continue refining the method so that the agent can learn from various forms of communication and interaction.
For more details, you can read the full paper, which will be presented at the upcoming Conference on Neural Information Processing Systems.
Overall, this new AI learning method is a promising approach to scaling up the teaching of AI agents and has the potential to revolutionize the way robots learn to perform a wide range of tasks.