New AI Curiosity-Driven Exploration Model Outperforms Previous Methods
AI researchers have developed a new curiosity-driven exploration model that outperforms standard exploration methods in challenging tasks. The simplicity of the design, when applied to a suite of visually complex, hard 3-D exploration tasks, outperforms standard methods. This model, called BYOL-Explore, achieves super-human performance in the most challenging Atari games and has a simpler design compared to other competitive agents.
What is Curiosity-Driven Exploration?
Curiosity-driven exploration is the active process of seeking new information to enhance the agent’s understanding of its environment. When the agent learns a world model that predicts future events based on past events, it can use the prediction mismatch as intrinsic reward for exploration, enhancing the world model with new information.
Inspired by the success of other models in computer vision, graph representation learning, and representation learning in RL, BYOL-Explore is a conceptually simple yet powerful curiosity-driven AI agent. It learns a world representation, world dynamics, and a curiosity-driven exploration policy by optimizing the prediction error at the representation level.
Performance of BYOL-Explore
The performance of BYOL-Explore is remarkable, achieving super-human performance in challenging tasks while having a simple design compared to other competitive agents. It outperforms standard curiosity-driven exploration methods in terms of mean capped human-normalized score (CHNS), measured across all tasks. This is achieved using only a single network concurrently trained across all tasks, whereas previous methods were restricted to the single-task setting and could only make meaningful progress with human expert demonstrations.
The researchers are looking to generalize BYOL-Explore to highly stochastic environments by learning a probabilistic world model to avoid stochastic traps and plan for exploration.
This new curiosity-driven exploration model holds promise for the future of AI and could lead to breakthroughs in various applications.