Title: Artificial Intelligence Enables Cultural Transmission without Human Data
Introduction
Throughout history, humanity has acquired extensive cultural knowledge, ranging from navigation to mathematics and art. Cultural transmission, the efficient exchange of information between individuals, has been instrumental in the advancement of human capabilities. In this article, we explore the significance and features of deep reinforcement learning (DRL) in generating artificial agents capable of cultural transmission, without relying on human data.
Video Demonstrations of Cultural Transmission
Videos showcasing our agents’ abilities to imitate and recall demonstrations by both bots and humans can be found on our website.
DRL for Test-Time Cultural Transmission
Using deep reinforcement learning, we train artificial agents capable of inferring and recalling expert knowledge in real-time. These agents can quickly learn new behaviors by observing a single human demonstration, without any prior training on human data. The trained agents generalize their knowledge across a wide range of previously unseen tasks, allowing for the efficient transfer of cultural information.
Training Environment and Cultural Transmission Components
Our agents are trained and tested in procedurally generated 3D worlds that contain colorful goals embedded in obstacle-filled terrains. To facilitate knowledge transfer, a “bot” always enters the goals in the correct sequence. Through ablations, we determine the essential components for cultural transmission, known as MEDAL-ADR. These components include memory, expert dropout, attentional bias towards the expert, and automatic domain randomization. Our MEDAL(-ADR) agent outperforms other ablation methods and showcases remarkable memory recall abilities.
Interpretable Neurons and Generalization
A closer examination of the agent’s brain reveals neurons responsible for encoding social information and goal states. These findings highlight the interpretability and adaptability of our agents. Furthermore, our agent demonstrates impressive generalization capabilities beyond the training distribution, recalling demonstrations even after the expert’s departure.
Conclusion
Our research presents a novel approach to training artificial agents capable of flexible, real-time cultural transmission, without relying on human data during the training process. This breakthrough opens the door for cultural evolution as an algorithm in developing more intelligent AI agents.
Note: This article is based on joint work by the Cultural General Intelligence Team, including Avishkar Bhoopchand, Bethanie Brownfield, Adrian Collister, Agustin Dal Lago, Ashley Edwards, Richard Everett, Alexandre Fréchette, Edward Hughes, Kory W. Mathewson, Piermaria Mendolicchio, Yanko Oliveira, Julia Pawar, Miruna Pîslar, Alex Platonov, Evan Senter, Sukhdeep Singh, Alexander Zacherl, and Lei M. Zhang. For more information, you can read the full paper [here](https://arxiv.org/abs/2203.00715).