Revolutionizing Federated Learning & Differential Privacy with Large-scale ASR Training

Introduction to Federated Learning and Differential Privacy with Automatic Speech Recognition

This article discusses the significance of automatic speech recognition (ASR) within the federated learning (FL) and differential privacy (DP) communities. While ASR has seen great progress, it has not received much attention in FL and DP. However, ASR is a natural fit for FL and DP due to the unique characteristics of speech data.

Challenges of Optimizing Large Models in ASR

State-of-the-art ASR models, such as large conformer and transformer models, pose optimization challenges, even in central training. While FL and DP models typically focus on small models, it is important to understand the differences in optimization and model size behavior for large models compared to small models.

Proposed Benchmark for Federated Learning with Differential Privacy

In this paper, the authors analyze key FL parameters and propose the first benchmark of FL with DP for large models in ASR. They examine the applicability of prior results and provide valuable insights into the differences that may arise when applying FL and DP research to large-scale ASR training.

Source link

Stay in the Loop

Get the daily email from AI Headliner that makes reading the news actually enjoyable. Join our mailing list to stay in the loop to stay informed, for free.

Latest stories

You might also like...