The thirty-sixth International Conference on Neural Information Processing Systems (NeurIPS 2022) is happening from 28 November – 9 December 2022, in New Orleans, USA. NeurIPS is the biggest conference in artificial intelligence (AI) and machine learning (ML), and DeepMind is a Diamond sponsor of the event, supporting the exchange of research advances in the AI and ML community.
DeepMind teams are presenting 47 papers, including 35 external collaborations, in virtual panels and poster sessions. Here’s an overview of some of the research being presented:
Advancing Best-in-Class Large Models
Large models (LMs) are AI systems trained on massive amounts of data, and they have achieved remarkable performances in language, text, audio, and image generation. DeepMind has created a 70 billion parameter language model called Chinchilla that outperforms many larger models, including Gopher. This work has influenced other models in creating leaner and better models. DeepMind also introduces Flamingo, a family of few-shot learning visual language models that handle images, videos, and textual data. Flamingo sets a new state of the art in few-shot learning on multimodal tasks. Additionally, DeepMind explores data properties that promote in-context learning in transformer models.
Optimising Reinforcement Learning
Reinforcement learning (RL) has shown promise in creating generalised AI systems for complex tasks. DeepMind introduces a new approach to boost the decision-making abilities of RL agents by expanding the scale of available information for retrieval. They also showcase an RL agent called BYOL-Explore, which achieves superhuman performance in visually complex environments through curiosity-driven exploration.
Algorithmic Advances
Algorithms play a fundamental role in modern computing, and incremental improvements can have a significant impact. DeepMind presents a highly scalable method for the automatic configuration of computer networks based on neural algorithmic reasoning. They also explore the relationship between graph neural networks and dynamic programming, and how to combine them to optimise out-of-distribution performance.
Pioneering Responsibly
DeepMind is committed to responsible AI development. They offer a set of desiderata that capture their ambitions for fair and transparent AI systems. They also propose practical ways to explain and understand the behavior of complex AI systems. Furthermore, DeepMind introduces a statistical measure called counterfactual harm, which helps AI agents reason about harm and avoid harmful actions. They also address model fairness failures caused by distribution shifts in real-world medical settings and propose ways to diagnose and mitigate these failures.
To learn more about DeepMind’s work at NeurIPS 2022, you can visit their page here.