An individual’s personality is made up of unique qualities and ways of thinking that shape our social interactions and preferences. When it comes to AI, Language Models (LLMs) have the ability to convincingly imitate human-like personalities in their outputs, showcasing a synthetic personality. However, recent research has shown that LLMs can also produce violent language and use deceptive and manipulative language, making their conversations and knowledge extraction unreliable.

Understanding the traits of the language generated by LLMs is crucial as they become the dominant interface for human-computer interaction. Researchers have been studying methods to minimize the negative impact of LLMs’ personality traits, such as using few-shot prompting. However, there is currently no scientific and systematic way to quantify LLMs’ personality.

To address this, researchers from Google DeepMind, the University of Cambridge, Google Research, Keio University, and the University of California, Berkeley have proposed psychometric approaches to characterize and shape LLM-based personality syntheses. They have created a methodology that uses existing psychometric tests to establish the validity of characterizing personalities in LLM-generated literature. They have also developed a method to mimic population variance in LLM responses through controlled prompting, allowing them to test the statistical correlations between personality and its external correlates.

The researchers tested their approach on LLMs of different sizes and training methods in two natural interaction settings: MCQA (machine comprehension question answering) and long-form text generation. The findings showed that LLMs can reliably simulate personality in their outputs, especially for larger models that have been fine-tuned with instructions. Additionally, personality in LLM outputs can be shaped along desired dimensions to mimic specific personality profiles.

Overall, this research provides valuable insights into understanding and engineering personality in LLMs. By developing more rigorous and verified methods, we can ensure that LLM-generated personalities are safe, appropriate, and effective.

Source link