Responsibility and Safety in Artificial Intelligence Language research is crucial in demonstrating and facilitating intelligence among humans. DeepMind is delving into the study of language processing and communication in both artificial agents and humans as part of AI research. The development of more advanced language models has the potential to build safe and efficient AI systems that can summarize information, provide expert advice, and follow instructions naturally.
The Gopher language model, with 280 billion parameters, has been studied to understand its strengths and weaknesses. It outperforms existing language models in key tasks and can engage in coherent interaction. However, it still exhibits failure modes such as repetition, biased reflection, and propagation of incorrect information. Understanding these failure modes is essential to address potential harms from large language models.
DeepMind’s second paper addresses potential ethical and social risks from language models. A comprehensive classification of these risks and failure modes is presented, with a taxonomy of 21 in-depth risks in six thematic areas. The paper emphasizes the need for a broad view to understand and mitigate these risks effectively.
The final paper proposes an improved language model architecture, the Retrieval-Enhanced Transformer (RETRO), which reduces the energy cost of training and makes it easier to trace model outputs. This model obtains comparable performance to a regular transformer with far fewer parameters, showcasing its efficiency and effectiveness.
These papers lay the foundation for DeepMind’s language research, with a focus on evaluating and deploying advanced language models and ensuring safe interactions with AI agents. The research aims to be cautious and thoughtful, drawing on multidisciplinary expertise to create large language models that serve society and benefit humanity.