Introducing Spherical CNNs: Improving Deep Learning for Scientific Applications
Computer vision models, like CNNs and vision transformers (ViTs), are commonly used for processing images. However, these models are designed to work with flat, planar data, such as digital images represented as a grid of pixels. In scientific applications, we often encounter data that is represented on a spherical surface, such as variables sampled from the Earth’s atmosphere or panoramic photos.
Using planar image processing methods for spherical signals presents two challenges. First, there is a sampling problem. It is difficult to define uniform grids on a sphere without distortion. This distortion affects the accuracy of CNNs and ViTs on spherical inputs. Second, signals and patterns on a sphere are often complicated by rotations. Models need to be able to address these rotations to make accurate predictions.
To overcome these challenges, we have developed an open-source library in JAX called “Scaling Spherical CNNs.” This library enables deep learning on spherical surfaces and has been shown to outperform state-of-the-art models on weather forecasting and molecular property prediction benchmarks.
Spherical CNNs solve the sampling and rotation problems by using spherical convolution and cross-correlation operations. These operations are computed via generalized Fourier transforms. While this approach is computationally more expensive than convolution with small filters on planar surfaces, we have optimized our library for speed and enabled distributed training using data parallelism.
In addition to the library, we have also introduced new techniques such as phase collapse activation, spectral batch normalization, and a new residual block that improve accuracy and efficiency. These advancements have allowed us to train larger and more accurate models, up to 100x larger than before.
We have successfully applied our models to molecular property regression and weather forecasting. In molecular property prediction, our models have achieved state-of-the-art performance on the QM9 benchmark. By mapping molecules to a set of spherical functions, we can leverage the rotation equivariance of spherical CNNs to make accurate predictions.
For weather forecasting, our models have shown promise in providing faster and more accurate forecasts. We have outperformed conventional CNN-based models on several weather forecasting benchmarks, including the ability to predict atmospheric variables six hours ahead and iteratively produce forecasts up to five days ahead.
Our JAX library for efficient spherical CNNs is now available for use. We believe that this library will not only benefit scientific applications but also computer vision and 3D vision tasks. Weather forecasting is a particularly active area of research at Google, and we are constantly working on improving the accuracy and robustness of our models.
Make your deep learning models more effective with spherical CNNs. Try out our library and unlock new possibilities in scientific research and computer vision.