New AI Curiosity-Driven Exploration Model Outperforms Previous Methods
AI researchers have developed a new curiosity-driven exploration model that outperforms standard exploration methods in challenging tasks. The simplicity of the design, when applied to a suite of visually complex, hard 3-D exploration tasks, outperforms standard methods. This model, called BYOL-Explore, achieves super-human performance in the most challenging Atari games and has a simpler design compared to other competitive agents.
What is Curiosity-Driven Exploration?
Curiosity-driven exploration is the active process of seeking new information to enhance the agent’s understanding of its environment. When the agent learns a world model that predicts future events based on past events, it can use the prediction mismatch as intrinsic reward for exploration, enhancing the world model with new information.
Why BYOL-Explore?
Inspired by the success of other models in computer vision, graph representation learning, and representation learning in RL, BYOL-Explore is a conceptually simple yet powerful curiosity-driven AI agent. It learns a world representation, world dynamics, and a curiosity-driven exploration policy by optimizing the prediction error at the representation level.
Performance of BYOL-Explore
The performance of BYOL-Explore is remarkable, achieving super-human performance in challenging tasks while having a simple design compared to other competitive agents. It outperforms standard curiosity-driven exploration methods in terms of mean capped human-normalized score (CHNS), measured across all tasks. This is achieved using only a single network concurrently trained across all tasks, whereas previous methods were restricted to the single-task setting and could only make meaningful progress with human expert demonstrations.
Moving Forward
The researchers are looking to generalize BYOL-Explore to highly stochastic environments by learning a probabilistic world model to avoid stochastic traps and plan for exploration.
This new curiosity-driven exploration model holds promise for the future of AI and could lead to breakthroughs in various applications.