Introducing Lemur and Lemur-Chat: State-of-the-Art Language Models for AI
Intelligent agents are problem solvers that can perceive, judge, and take action based on data from their surroundings. Language agents, in particular, have shown promise in performing complex tasks using natural language. One approach to creating language agents is through large language models (LLMs), which can mimic human thought and language. This flexibility allows for the development of multi-agent systems and adaptability to new situations.
The Role of LLMs in Language Agents
LLMs are crucial in constructing language agents that can understand human interaction, reasoning, and planning. While LLMs excel in natural language capabilities, they typically rely on general-purpose code or domain-specific APIs for executing actions in the environment. To bridge this gap, a team of researchers from the University of Hong Kong, XLang Lab, Salesforce Research, Sea AI Lab, University of Washington, and MIT CSAIL have developed Lemur and Lemur-Chat.
About Lemur and Lemur-Chat
Lemur and Lemur-Chat are state-of-the-art, publicly available language models that have been pre-trained and fine-tuned to achieve a balance between text and code. The researchers created Lemur by constructing a code-centric corpus based on The Stack, including 90 billion tokens with a 10:1 text-to-code ratio. For Lemur-Chat, they pretrained the model using both text and code instances. These models underwent extensive examinations and outperformed other open-source models across 8 benchmarks.
The Importance of Combining Linguistic and Computational Skills
Combining linguistic and computational skills is crucial in agent-based settings. While models like Llama-2-70B-Chat excel in natural language processing, they may struggle with coding. Lemur’s superior performance can be attributed to its natural language processing and programming abilities, allowing it to handle complex executable action sequences. This study highlights the importance of optimizing the synergy between natural and programming languages in creating sophisticated language agents.
For more details, you can read the paper and check out the code on Github.
Join our community in our ML SubReddit, Facebook Group, Discord Channel, and Email Newsletter for the latest AI research news and projects.
If you enjoy our work, subscribe to our newsletter for more exciting content.
We are also on WhatsApp. Join our AI Channel on Whatsapp.