Are AI Text Detectors Mislabeling Essays Written by Non-Native English Speakers?
The Reliability and Consequences of AI Text Detectors
In a recently published opinion paper in the journal Patterns, researchers have highlighted the pitfalls of using computer programs to identify whether a text was written by artificial intelligence (AI). Surprisingly, these detectors often mistakenly label essays written by non-native English speakers as AI-generated content. The researchers warn against relying on these detectors due to their unreliability, as it can have adverse effects on individuals such as students and job applicants.
Concerns in Education
The growing use of AI tools, such as OpenAI’s ChatGPT chatbot, in student assignments has raised concerns among educators in the U.S. To address this, many educators have started employing GPT detectors to assess whether the text was generated by AI. However, the reliability and effectiveness of these detectors have not been thoroughly tested.
Testing GPT Detectors
In a study conducted by senior author James Zou and his team from Stanford University, seven popular GPT detectors were put to the test. They analyzed 91 English essays written by non-native English speakers who had taken the Test of English as a Foreign Language (TOEFL) proficiency test. Shockingly, the detectors incorrectly identified over 50% of these essays as AI-generated. One detector even falsely flagged almost 98% of the essays as written by AI. In contrast, the detectors accurately classified over 90% of essays written by American eighth-grade students as human-generated.
The Role of Word Choice
The detectors function by evaluating text perplexity, which measures the unexpectedness of word choices in an essay. Zou explains that using common English words leads to a low perplexity score, increasing the likelihood of being flagged as AI-generated. On the other hand, using complex and sophisticated vocabulary makes the text more likely to be classified as human-written. This is because large language models like ChatGPT are trained to generate text with low perplexity to simulate natural human language.
Vulnerability of Non-Native English Writers
Unfortunately, non-native English writers often use simpler word choices, making them more susceptible to being falsely identified as using AI. To illustrate this, Zou’s team inserted the human-written TOEFL essays into ChatGPT and had it edit the text using more complex vocabulary. Surprisingly, the GPT detectors then labeled these AI-edited essays as human-written.
The Need for Caution and Improvement
Given the biases and ease with which GPT detectors can be tricked, Zou advises against using them in classrooms. Their usage can extend beyond education, as search engines such as Google also devalue AI-generated content, inadvertently silencing non-native English writers. While AI tools can benefit student learning, it is essential to enhance and evaluate GPT detectors before implementing them. Zou suggests that training these algorithms with a wider range of writing styles could be a way to improve their reliability.