Introducing Kani: A Lightweight Framework for Building Language Model Applications
The popularity of large language model applications is on the rise due to their incredible capabilities. These models are becoming more sophisticated by incorporating features like tool usage tracking and retrieval augmentation. However, existing frameworks for building such applications often dictate how developers should format their prompts and impose limitations on customization and reproducibility.
The Solution: Kani
A team of researchers from the University of Pennsylvania has developed Kani, an open-source framework designed specifically for building language model applications. Kani is lightweight, extensible, and model-neutral, making it easy for developers to add a wide range of complex features. It offers support for core elements of chat interaction, such as model interaction, chat management, and robust function calling.
What sets Kani apart is its adaptability and customizability. Developers can create language model applications using Kani’s building blocks without being restricted by predefined structures or limitations. The framework’s fundamental features can be easily altered, and extensive documentation is provided to help developers modify the framework’s functionality according to their unique demands and requirements.
Benefits for Different Users
Kani is a valuable tool for various individuals, including academics, amateurs, and business people. Researchers can use Kani to improve the reproducibility of their work while maintaining fine-grained control over their language model applications. Even with powerful models like GPT-4, users can quickly start designing apps with just a few lines of code. Kani’s versatility and durability also benefit industry workers, especially in areas like chat management and function management.
The Key Components of Kani
Kani simplifies language model installation and querying by requiring Python 3.10+. It can be easily installed via pip and offers core dependencies and optional extras like the OpenAI engine. The fundamental processing unit in the Kani framework is called a ‘Kani’. When building applications with Kani, developers work with and manipulate various Kani objects, which consist of three essential components: inference engine, chat history, and function context.
Inference Engines
Through inference engines, a Kani object communicates with linguistic models. This interaction allows developers to transition between different models seamlessly without changing the application’s code. Kani keeps track of token totals and topic switches, ensuring that the dialogue context stays within the model’s bounds.
Chat Management
Kani provides chat history as a crucial component for managing chat interactions. Developers can easily access and manipulate chat history within the framework, allowing for flexible and dynamic conversation management.
Function Context
Language models can access callable functions through Kani. It verifies function calls, runs the appropriate code, and sends the results back to the inference engine. This feature enables developers to integrate complex functionalities into their language model applications.
In Conclusion
Kani solves the problems faced by language model application developers by offering customization, flexibility, and an open-source approach. It empowers developers to create feature-rich applications while maintaining control and interoperability. With Kani’s fundamental building blocks for chat interaction, developers can build incredible language model applications that meet their specific needs and requirements.
Check out the Paper and Github for more information about Kani. All credit for this research goes to the researchers on this project. Also, don’t forget to join our 30k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
If you like our work, you will love our newsletter. Subscribe to stay updated!