Efficiently Connect with Multiple Tools Using AI
Connecting with various tools (APIs) and completing complex tasks can be challenging, but tool learning aims to make it easier. By utilizing large language models (LLMs), tool learning harnesses the potential of AI to act as a middleman between users and a wide range of applications. However, current instruction tuning mostly focuses on simple language tasks rather than tool usage, limiting the capabilities of open-source LLMs.
On the other hand, closed-source LLMs like GPT-4 have excellent tool usage skills but lack transparency. This hinders community-driven innovation and the democratization of AI technology. It is crucial to enable open-source LLMs to understand various APIs adeptly. Previous studies have attempted to create instruction-tuning data for tool usage, but they have several limitations.
The limitations include:
1. Restricted APIs: The studies either ignore real-world APIs or only consider a narrow range with inadequate diversity.
2. Constrained scenarios: Existing works only focus on instructions that use one tool, while real-world situations may require the use of multiple tools.
3. Subpar planning and reasoning: Previous studies use a straightforward prompting mechanism that cannot handle complex instructions effectively.
To address these issues, ToolLLM is introduced. It is a general tool-use framework that includes data production, model training, and assessment. The framework aims to stimulate tool-use skills in open-source LLMs. The API retriever suggests relevant APIs to ToolLLaMA during instruction inference, and ToolLLaMA makes multiple API calls to generate the final result. ToolEval evaluates the entire deliberation process.
To create a high-quality instruction-tuning dataset called ToolBench, a depth-first search-based decision tree (DFSDT) is used to improve planning and reasoning capabilities. The researchers collect REST APIs from RapidAPI, scrape API documentation, and generate various instructions for single-tool and multitool scenarios.
ToolEval, an automated evaluator supported by ChatGPT, is developed to assess the tool-use abilities of LLMs. It includes win rate and pass rate metrics to measure the value, utility, and successful execution of instructions.
ToolLLaMA, optimized on ToolBench, demonstrates attractive capabilities to handle both simple single-tool and complex multitool instructions. It achieves comparable performance to the “teacher model” ChatGPT in tool usage. DFSDT outperforms traditional reasoning methods like ReACT by considering different reasoning trajectories.
The integration of ToolLLaMA and the API retriever eliminates the manual selection of APIs and improves the decision-making process. The retriever shows high retrieval precision by suggesting closely matched APIs from a vast pool. The study aims to enable open-source LLMs to effectively carry out complex commands using a variety of APIs in real-world settings.
For more details and the source code, check out the paper and GitHub. Join our ML SubReddit, Facebook Community, Discord Channel, and Email Newsletter for the latest AI research news and projects.
About the Author:
Aneesh Tickoo is currently pursuing his undergraduate degree in Data Science and Artificial Intelligence. He is a consulting intern at MarktechPost, passionate about image processing and building machine learning solutions. Connect with him for interesting collaborations and projects.
(End of Article)
Use SQL to predict the future (Sponsored)