Recent advances in artificial intelligence have led to significant improvements in large language models (LLMs) like GPT-3 and GPT-4. These models have shown great promise in problem-solving and understanding natural language. However, when it comes to applications that require high accuracy and reliability, such as in healthcare and biology, there is still a major challenge to overcome – hallucination.
Detecting and measuring the confidence level of hallucinations in LLM output is a difficult task. It is especially challenging after using reinforcement learning with human input, as the confidence score from the LLM can be unreliable. Traditional techniques for evaluating confidence are expensive and can be biased. There are two main methods for assessing confidence in LLM responses.
The first method involves prompting the LLM in various ways to generate multiple responses, which are then used to determine the reliability of the answer. However, these techniques are not always quantitative and can be influenced by model bias. The second method involves using external sources of data, such as human reviewers or large labeled datasets, to assess the confidence. But these methods require extensive manual annotation work.
To address these challenges, researchers from Microsoft have developed a flexible framework that combines data from both the LLM response and external supervision sources. They were inspired by programmatic supervision and Pareto optimization research. Their approach involves using external sources of supervision independent of the LLM to reduce bias. They also consider LLM errors as noisy perturbations on the gold labels, which improves calibration.
The researchers propose a method called Pareto Optimum Learning assessed risk (POLAR) score to measure the likelihood of LLM mistakes. They conducted experiments on four different natural language processing tasks and found that the POLAR score was strongly correlated with the LLM error rate. By using dynamic prompting strategies based on the POLAR score, they were able to improve LLM performance in high-risk situations.
The key innovation of their approach is the integration of Pareto optimum self-supervision, which allows for better calibration of the LLM without the need for human-labeled training data. This framework is particularly useful in fields where annotation is expensive.
In conclusion, the researchers have proposed a novel method for addressing the challenge of hallucination in LLMs. Their approach combines data from the LLM response and external supervision sources to improve calibration and reliability. By using the POLAR score, they were able to enhance LLM performance in high-risk situations. This framework has the potential to advance the field of artificial intelligence and improve the accuracy and dependability of LLM applications.
Check out the paper for more details. Don’t forget to join our ML SubReddit, Discord Channel, Twitter, and Email Newsletter for the latest updates on AI research. If you have any questions or feedback, feel free to reach out to us at Asif@marktechpost.com.
Featured Tools:
– Check Out 100’s AI Tools in AI Tools Club