DeepMind engineers contribute to AI research by developing tools, scaling algorithms, and creating virtual and physical environments for training AI systems. They have recently discovered the usefulness of JAX, a machine learning framework developed by Google Research teams. JAX is a Python library that simplifies high-performance numerical computing, particularly for machine learning research. It is easy to adopt because it is built on familiar Python and NumPy. JAX offers various features that support machine learning research, such as native support for gradient-based optimization, automatic vectorization, and JIT-compilation for GPU and Cloud TPU accelerators. DeepMind has found JAX to be valuable for rapid experimentation and it has been utilized in many recent publications. To facilitate knowledge sharing, DeepMind is hosting a JAX Roundtable at the NeurIPS virtual conference on December 9th.
DeepMind recognizes the importance of balancing rapid prototyping and scalability in AI research. They have adopted a modular approach, extracting essential building blocks from each project into well-tested and efficient components. This allows researchers to focus on their work while benefiting from code reuse and performance improvements. DeepMind’s JAX Ecosystem aims to provide interoperable but independent libraries, giving researchers flexibility in choosing the right tools. Their libraries, such as Haiku, Optax, RLax, Chex, and Jraph, have been developed to address specific research needs and are open-source for the AI research community to explore.
Haiku, for example, simplifies the use of neural networks with trainable parameters, employing JAX’s functional paradigm while retaining a familiar object-oriented programming model. Optax offers a library of gradient transformations and composition operators for various optimization algorithms. RLax provides building blocks for constructing reinforcement learning agents, covering a wide range of algorithms and exploration methods. Chex is a collection of testing utilities that ensure the correctness and reliability of research code. Jraph supports working with graph neural networks in JAX, providing a standardized data structure, utilities for graph manipulation, and a library of graph neural network models.
DeepMind’s JAX Ecosystem is constantly evolving, and they invite the AI research community to explore their libraries and harness the potential of JAX to accelerate their own research.