Using Large Language Models (LLMs) has revolutionized various tasks that were previously impossible. However, these models require more fine-tuning to improve their performance in specialized areas. There are common methods for fine-tuning big models, including ongoing pretraining, instruction tuning, and reinforcement learning with human feedback.
While there are already several big models available to the public, there is no efficient toolbox that can handle fine-tuning across all these models. To address this, a team from Hong Kong University and Princeton University has developed an easy-to-use toolset for developers and researchers to efficiently finetune and infer huge models with limited resources.
They have successfully trained a custom model based on a 7-billion-parameter LLaMA model using just one Nvidia 3090 GPU and five hours. The team has also shared the model weights for academic research purposes, after finetuning versions of LLaMA with 7, 13, 33, and 65 billion parameters on a single machine.
The optimization process for large language models involves four steps. The first step is domain adaptation, where the model is trained specifically for a certain domain. The second step is task adaptation, which involves training the model to accomplish a specific goal such as summarization or translation. The third step is instruction finetuning, where the model’s parameters are adjusted based on instructional question-answer pairs. And finally, reinforcement learning using human feedback is used to refine the model.
LMFlow is a comprehensive tool that supports all four steps of finetuning for big models. It offers continuous pretraining, instruction tuning, and reinforcement learning with human feedback, along with easy-to-use APIs. With LMFlow, anyone can now train their own personalized language models, even with limited computational resources. This allows for a wide range of applications such as question answering, writing, translation, and expert consultations.
If users have a larger model and dataset, training over a longer period will result in even better performance. The team has recently trained a 33-billion-parameter model that outperforms ChatGPT.
To learn more about LMFlow, you can check out the research paper and the GitHub link. Don’t forget to join our ML SubReddit, Discord Channel, and subscribe to our Email Newsletter for the latest AI research news and cool AI projects. If you have any questions or suggestions, feel free to email us at asif@marktechpost.com.
Also, make sure to check out AI Tools Club for hundreds of AI tools.