Recent research from the University of Washington and the Allen Institute for AI reveals a new process called proxy-tuning. This approach enhances large language models (LMs) without gaining access to their internal weights.
Proxy-tuning involves using a smaller tuned LM to calculate the difference between its predictions and the untuned version. Then, decoding-time experts adjust the original predictions of the larger base model based on this difference. This method effectively achieves the benefits of direct tuning without altering the base model’s parameters. This innovation aims to close the disparity between a base language model and its directly tuned version without needing direct fine-tuning.
The approach significantly improves performance, achieving 88.0% on AlpacaFarm and 32.0% on GSM for 70B-BASE. It also reduces toxicity to 0% on Toxigen and surpasses CHAT models in truthfulness on TruthfulQA’s open-ended setting.
In summary, proxy-tuning is a promising approach for fine-tuning large language models at decoding time by modifying output logits. It addresses the challenge of customizing proprietary models to diverse use cases and makes large language models more accessible, especially for those with limited resources.
For more information, check out the paper at the link provided and follow the researchers’ work on social media.