Positionality refers to the perspectives of researchers based on their own experiences, identity, culture, and background. These perspectives influence the design decisions made when creating natural language processing (NLP) datasets and models. Design bias in these datasets and models can result from latent design choices and the researcher’s positionality, leading to discrepancies in performance for different populations. This can perpetuate systemic inequities if one group’s standards are enforced on the rest of the world.
One challenge in addressing design bias is the wide range of design decisions involved in building datasets and models, with only some of these decisions being recorded. Additionally, many commonly used models are not readily accessible outside of application programming interfaces (APIs), making it difficult to directly identify design biases.
To tackle these issues, researchers from the University of Washington, Carnegie Mellon University, and Allen Institute for AI have developed NLPositionality. This approach involves recruiting a diverse community of volunteers from different cultural and linguistic backgrounds to annotate dataset samples. The researchers then compare different identities and contexts to assess biases in the design based on the alignment with the original dataset labels or model predictions.
NLPositionality offers several advantages over other methods such as paid crowdsourcing or in-lab experiments. Unlike other platforms, LabintheWild, which is used in NLPositionality, has a more diverse participant population. Instead of relying on monetary remuneration, this method leverages participants’ intrinsic motivation to expand their self-awareness, resulting in improved data quality. This platform also allows for ongoing collection of new annotations, capturing more recent observations of design biases over extended periods. Importantly, NLPositionality does not require pre-existing labels or post hoc predictions to be applied to datasets or models.
The researchers applied NLPositionality to examine design biases in two NLP tasks: social acceptability and hate speech detection. They focused on task-specific and task-general large language models like GPT-4, as well as the associated datasets and supervised models. As of May 25, 2023, they have collected annotations from 1,096 annotators in 87 countries, generating a total of 16,299 annotations. These annotations revealed that datasets and models designed for white, college-educated millennials from English-speaking countries, a subset of “WEIRD” populations (Western, Educated, Industrialized, Rich, Democratic), performed best. This highlights the need for more diverse models and datasets to ensure the inclusivity and fairness of NLP research.
In conclusion, NLPositionality provides a method for assessing positionality and design biases in NLP datasets and models. By involving a diverse range of participants and avoiding reliance on pre-existing labels, NLPositionality offers a fresh perspective on addressing design biases. Its application to social acceptability and hate speech detection tasks underscored the importance of inclusive and diverse research in NLP.