Language models (LMs) have had a significant impact on natural language processing applications in various industries. They are widely used in fields like healthcare, software development, and finance. One of the most successful applications of transformer-based models is their use in writing software code and helping authors improve their writing style and storyline.
However, LMs are not just limited to these applications. They are increasingly being used in open-ended contexts, such as chatbots and dialogue assistants, to answer subjective questions. These questions can range from asking the AI if AI will take over the world in the future to whether legalizing euthanasia is a good idea. The responses from LMs to such subjective queries can shape societal views and have a significant impact.
It is challenging to predict how LMs will respond to subjective questions as designers and developers of these models come from diverse backgrounds with different viewpoints. Additionally, there is no “correct” response to judge an LM. Any viewpoint expressed by the model can greatly influence user satisfaction and opinion formation.
To evaluate LMs in open-ended tasks, a team of postdoctoral researchers from Stanford University and Columbia University has developed a quantitative framework. They used public opinion surveys and responses from different demographic groups to study the range of opinions generated by LMs and how they align with human populations.
The researchers created a dataset called OpinionQA by using expert-chosen public opinion surveys and multiple-choice questions. These surveys collected opinions from various demographic groups in the US on topics like abortion and gun violence. The dataset allowed the researchers to quantify the alignment of LM responses with human opinions.
The team assessed nine LMs from AI21 Labs and OpenAI using the OpinionQA dataset. They looked at representativeness, steerability, and consistency. Representativeness refers to how well the default LM beliefs match those of the US population or a specific segment. It was found that contemporary LMs have significant divergences from demographic groupings on topics like climate change. The researchers also observed that LMs did not adequately represent the viewpoints of certain groups, such as those over 65 and widows.
Steerability refers to whether an LM follows the opinion distribution of a group when prompted. Most LMs tend to align more with a group when encouraged to do so.
Consistency was another aspect examined by the researchers. They investigated if LMs aligned consistently with different groups across various topics. While some LMs aligned well with specific groups on certain issues, the alignment was not consistent across all topics.
In summary, the researchers from Stanford and Columbia University have developed a framework to analyze LM opinions using public opinion surveys. Their work led to the creation of the OpinionQA dataset, which revealed misalignments between LMs and human opinions. The framework can be extended to datasets from different regions. The researchers hope that their work will contribute to the evaluation of LMs in open-ended tasks and the development of unbiased and stereotype-free LMs.
For more information on the OpinionQA dataset, you can visit the research paper and GitHub page.