Online prediction from experts is a crucial machine learning problem that has been extensively studied with privacy constraints. In this article, we will introduce new algorithms that address this problem and offer improved regret bounds compared to existing algorithms for non-adaptive adversaries. These algorithms are designed to achieve better results in both the stochastic setting and for oblivious adversaries.
For approximate differential privacy, our algorithms provide regret bounds for the stochastic setting and for oblivious adversaries, where the number of experts plays a significant role. In the high-dimensional regime, our algorithms are the first to achieve sub-linear regret for oblivious adversaries in pure differential privacy. Additionally, we have made significant progress in understanding adaptive adversaries by establishing new lower bounds.
Our research highlights an important distinction between the optimal regret for adaptive and non-adaptive adversaries in the context of online prediction from experts with privacy constraints. Unlike the non-private setting, our results demonstrate a strong separation between these two types of adversaries. Furthermore, our lower bounds emphasize the necessity of using approximate differential privacy to achieve the non-private regret when dealing with adaptive adversaries.
By developing these new algorithms and establishing lower bounds for adaptive adversaries, we contribute to the ongoing exploration of privacy-constrained online prediction from experts. Our findings offer valuable insights into the unique challenges and opportunities presented by this problem, particularly in relation to differential privacy.
HTML Subheadings:
1. New Algorithms for Online Prediction from Experts with Privacy Constraints
2. Improved Regret Bounds for Non-adaptive Adversaries
3. Distinction Between Adaptive and Non-adaptive Adversaries in Privacy-constrained Online Prediction