BYOV: Unveiling Model Uncertainty for Improved Self-Supervised Learning

Understanding Model Uncertainty with BYOV Algorithm

A paper accepted at the workshop Self-Supervised Learning – Theory and Practice at NeurIPS 2023 introduces the Bootstrap Your Own Variance (BYOV) algorithm, an innovation in Self-Supervised Learning (SSL) that addresses model uncertainty. -Important for many applications.

Combining BYOL and BBB for Model Uncertainty

The BYOV algorithm combines the Bootstrap Your Own Latent (BYOL) and Bayes by Backprop (BBB) methods. BYOL is a negative-free SSL algorithm, while BBB is a Bayesian method for estimating model posteriors. By integrating these two approaches, BYOV is able to capture the learned predictive std and demonstrate its usefulness for label free uncertainty estimation.

Improving Model Performance

Comparative testing shows that BYOV outperforms the deterministic BYOL baseline, presenting improved calibration and reliability in the face of various augmentations. For example, BYOV performs +2.83% test ECE and +1.03% test Brier over the BYOL baseline, and shows even better results when faced with Salt & Pepper noise (+2.4% test ECE, +1.2% test Brier).

Source link

Stay in the Loop

Get the daily email from AI Headliner that makes reading the news actually enjoyable. Join our mailing list to stay in the loop to stay informed, for free.

Latest stories

You might also like...