Understanding Conformal Prediction in Machine Learning
The workshop on Regulatable ML at NeurIPS 2023 has accepted a paper on Conformal Prediction (CP) – a method that estimates risk or uncertainty in Machine Learning. This is important for industries like healthcare and finance where risk management is crucial.
Challenges of CP for Regression
When it comes to regression, CP can be challenging, especially when dealing with heteroscedastic, multimodal, or skewed output distributions. Estimating a distribution over the output can help, but it can be sensitive to errors and yield unstable intervals.
A New Approach
To address these challenges, the approach in this paper converts regression to a classification problem. This allows CP for classification to obtain CP sets for regression. To maintain the ordering of the continuous-output space, a new loss function is designed along with necessary modifications to the CP classification techniques.
Empirical results from benchmarks show that this simple approach surprisingly yields good results on practical problems.