Introducing the Graph of Thoughts (GoT) Framework
Artificial Intelligence (AI) is revolutionizing the use of Large Language Models (LLMs). One particular type of LLM, based on the Transformer architecture’s decoder-only design, has gained popularity. Models like GPT, PaLM, and LLaMA have become widely used. Prompt engineering is a strategic technique that leverages LLMs to address various problems by embedding task-specific instructions in the input text. By properly writing these instructions, the LLM can generate relevant text and complete the task using its autoregressive token-based approach.
The Chain-of-Thought (CoT) Method
The Chain-of-Thought (CoT) method builds upon prompt engineering. In CoT, the input prompt not only includes the task’s description but also provides thoughts or intermediate steps of thinking. This addition significantly enhances the LLM’s problem-solving abilities without requiring model updates or modifications. To compare the capabilities of LLMs to existing paradigms like Chain-of-Thought and Tree of Thoughts (ToT), a new framework called Graph of Thoughts (GoT) has been introduced.
The Graph of Thoughts (GoT) Framework
The Graph of Thoughts (GoT) represents data as an arbitrary graph, allowing LLMs to handle and generate data in a more flexible manner. In this graph, individual pieces of information, or LLM thoughts, are represented as vertices, and the connections and dependencies between them are represented as edges. This framework enables the combination of different LLM ideas to produce more powerful and effective results. By allowing these thoughts to be interconnected within the graph, complex networks of thoughts can be captured. This allows for the integration of various ideas into a cohesive answer, simplifying intricate thought networks and improving ideas through feedback loops.
GoT has demonstrated superior performance compared to existing methods across multiple tasks. For example, it outperforms ToT in a sorting test by increasing sorting quality by 62% while reducing computing expenses by over 31%. This highlights GoT’s ability to balance task accuracy with resource efficiency. Another notable advantage of GoT is its extensibility. The framework is easily adaptable to new idea transformations, allowing for creative prompting schemes. This adaptability is crucial for navigating the evolving landscape of LLM research and applications.
The GoT framework significantly advances the alignment of LLM reasoning with human thinking processes and brain systems. Both human and brain thought processes involve complex networks of thoughts that interact, branch out, and influence one another. GoT bridges the gap between traditional linear techniques and these sophisticated, network-like mental processes, enhancing the capabilities of LLMs and their ability to handle challenging problems.
For more information, please refer to the research paper and GitHub repository for this project. Credit goes to the researchers involved in this project. Also, make sure to join our 29k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter where we share the latest AI research news, cool AI projects, and more.
Tanya Malhotra is a final year undergrad from the University of Petroleum & Energy Studies, Dehradun, pursuing BTech in Computer Science Engineering with a specialization in Artificial Intelligence and Machine Learning.
She is a Data Science enthusiast with good analytical and critical thinking, along