Understanding Model Uncertainty with BYOV Algorithm
A paper accepted at the workshop Self-Supervised Learning – Theory and Practice at NeurIPS 2023 introduces the Bootstrap Your Own Variance (BYOV) algorithm, an innovation in Self-Supervised Learning (SSL) that addresses model uncertainty. -Important for many applications.
Combining BYOL and BBB for Model Uncertainty
The BYOV algorithm combines the Bootstrap Your Own Latent (BYOL) and Bayes by Backprop (BBB) methods. BYOL is a negative-free SSL algorithm, while BBB is a Bayesian method for estimating model posteriors. By integrating these two approaches, BYOV is able to capture the learned predictive std and demonstrate its usefulness for label free uncertainty estimation.
Improving Model Performance
Comparative testing shows that BYOV outperforms the deterministic BYOL baseline, presenting improved calibration and reliability in the face of various augmentations. For example, BYOV performs +2.83% test ECE and +1.03% test Brier over the BYOL baseline, and shows even better results when faced with Salt & Pepper noise (+2.4% test ECE, +1.2% test Brier).