Published on 1st December 2022, researchers Julien Perolat, Bart De Vylder, Daniel Hennes, Eugene Tarassov, Florian Strub, and Karl Tuyls have taught an AI system called DeepNash to play the complex board game Stratego at an expert level. Utilizing game theory and model-free deep reinforcement learning, DeepNash has mastered the game, becoming a top-three player among human experts on the Gravon games platform.
Understanding the complexity of Stratego
Stratego is a challenging game for AI because it’s a game of imperfect information, where players cannot directly observe their opponent’s pieces. Unlike perfect information games like chess and Go, this complexity has made it difficult for AI systems to excel. The game’s length and the sheer number of possible states make it especially challenging.
Seeking an equilibrium
DeepNash uses a novel approach to learn Stratego, combining game theory and model-free deep reinforcement learning. This approach, known as Regularised Nash Dynamics (R-NaD), guides the AI’s learning behavior towards a Nash equilibrium, making it difficult to exploit. In matches against top Stratego bots and expert human players, DeepNash demonstrated remarkable and unpredictable strategies, achieving an all-time top-three ranking among human experts.
Expect the unexpected
To achieve its success, DeepNash displayed unpredictable behaviors, both in its initial piece-deployment phase and during gameplay. It developed strategies to keep its opponent guessing while valuing information and utilizing bluffing tactics, similar to the game of poker.
In conclusion, mastering Stratego has implications beyond gaming. DeepNash’s ability to operate in complex, real-world situations with limited information makes it a valuable tool for solving complex problems across a wide range of domains.