AI text detectors can reliably tell AI-written text from human writing.
Tests of detection tools found poor reliability, frequent mislabelling of writing by non-native English speakers, and easy evasion through paraphrasing. OpenAI withdrew its own classifier for low accuracy. A claim of reliability needs a named detector, version, population and error rate.
Current
Cases on this claim
Supporting
1
Disputing
0
1 case from 1 contributor. These numbers count cases people have written. They are not votes, and Koinovera does not decide whether the claim is true.
Original case
The case IKECHUKWU put this claim forward with on 6 Oct 2026. It is the publisher's case, open to the same scrutiny as every other.
+Supports
The study claims that GPT detectors are not sufficiently reliable or fair, particularly because they can misclassify human writing as AI-generated and can be easily bypassed. This is because over half of non-native English writing samples were incorrectly classified as AI-generated, while better prompt design could help AI-generated text evade detection.[1]
Passage from the sourceIntroduction
Generative language models based on
GPT, such as ChatGPT, have gained significant attention in recent times. Within a
mere 2 months of its launch, ChatGPT
amassed over 100 million monthly active
users, marking its place as one of the fastest-growing consumer internet applications in history.1 Despite their immense
potential for enhancing productivity and
fostering creativity, these powerful models
also pose risks, such as the proliferation of
AI-generated content masquerading as
human written, which may lead to the
spread of fake content and exam cheating.
Educators, in particular, are increasingly
concerned about determining when and
where students have used AI and AI writing
tools in their work. However, multiple
studies have demonstrated the difficulty
humans face in detecting AI-generated
content with the naked eye,2 thus creating
an urgent and pressing demand for effective detection methods. While several
GPT detectors have been developed and
implemented to mitigate the risks associated with AI-generated content, their accuracy, reliability, and effectiveness remain
uncertain due to limited evaluation.3 This
knowledge gap is especially worrisome
given the potentially harmful consequences ofmistakenly flagging an innocent
student’s work as AI generated.4
Given the transformative impact of
generative language models and the potential risks associated with their misuse,
developing trustworthy and accurate
detection methods is crucial. In our recent
preprint,5,6 we exposed an alarming bias
in GPT detectors against non-native English speakers: over half of the non-native
English writing samples were misclassified as AI generated, while the accuracy
for native samples remained near perfect.
Our analysis further revealed a trend
where more literary language was classified as more ‘‘human’’: enhancement
of word choice in non-native English
writing samples reduced misclassification, while simplifying native writing samples increased it, suggesting that GPT
detectors are inadvertently penalizing
individuals with limited linguistic proficiency. On the other hand, we found that
GPT detectors be easily bypassed by better ChatGPT prompt design. This raises a
pivotal question: if AI-generated content
can easily evade detection while human
text is frequently misclassified, how effective are these detectors truly?
Our findings emphasize the need for
increased focus on the fairness and robustness of GPT detectors, as overlooking
their biases may lead to unintended
consequences, such as themarginalization
of non-native speakers in evaluative or
educational settings. This paper is among
the first to systematically examine the
biases present in GPT detectors and advocates for further research into addressing
these biases and refining the current detection methods to ensure a more equitable
and secure digital landscape for all users.
Specific 2023 detectors and text samples; newer versions require fresh evaluation.
The study claims that AI-generated text detection tools are neither accurate nor reliable enough for academic use. This is because the 12 publicly available tools and two commercial systems tested often classified AI-generated text as human-written, and their performance became significantly worse when the text was modified or obfuscated.[2]
Passage from the sourceAbstract
Recent advances in generative pre-trained transformer large language models have emphasised the potential risks of unfair use of artificial intelligence (AI) generated content in an academic environment and intensified efforts in searching for solutions to detect such content. The paper examines the general functionality of detection tools for AI-generated text and evaluates them based on accuracy and error type analysis. Specifically, the study seeks to answer research questions about whether existing detection tools can reliably differentiate between human-written text and ChatGPT-generated text, and whether machine translation and content obfuscation techniques affect the detection of AI-generated text. The research covers 12 publicly available tools and two commercial systems (Turnitin and PlagiarismCheck) that are widely used in the academic setting. The researchers conclude that the available detection tools are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text. Furthermore, content obfuscation techniques significantly worsen the performance of tools. The study makes several significant contributions. First, it summarises up-to-date similar scientific and non-scientific efforts in the field. Second, it presents the result of one of the most comprehensive tests conducted so far, based on a rigorous research methodology, an original document set, and a broad coverage of tools. Third, it discusses the implications and drawbacks of using detection tools for AI-generated text in academic settings.
Summary of the cases and their sourcesAI analysisNot generated yet
AI summary
Written by AI from the published cases and the sources they cite. It describes what each side argues and what the readable sources show, and is not a ruling from Koinovera. The generation date does not guarantee that the claim wording or sources are unchanged. It only changes when someone asks for an update.
No generated summary yet. Koinovera can summarise the usable evidence attached to this claim and cite the passages used.
Every argument made about this claim, each with its own reasoning and sources.
Strongest orders cases by how readers have rated their reasoning, weighted by the standing of those readers and lowered when a case rests on poorly rated sources. Cases nobody has rated sit in the middle, newest first. It never counts how many cases or sources there are.