AI Detectors Are Broken — Here's the Evidence

Turnitin, the plagiarism-checking company used by more than 16,000 institutions worldwide, built an AI-writing detector on top of its existing similarity checker in 2023 (The Markup). In its own blog post, Turnitin's chief product officer Annie Chechitelli wrote that the company's "efforts have primarily been on ensuring a high accuracy rate accompanied by a less than 1% false positive rate" (Turnitin, "Understanding false positives within our AI writing detection capabilities," Mar. 16, 2023). That under-1-percent figure is the number most schools were told to trust when they turned the feature on.

Even Turnitin's own post immediately qualifies the number: "there is still a small risk of false positives," and the company tells instructors the false-positive rate "is not zero" (Turnitin blog). To its credit, Turnitin later published its own follow-up test on nearly 2,000 essays from English Language Learners and reported a 0.014 false-positive rate for that group versus 0.013 for native English writers — a difference the company calls not statistically significant (Turnitin, AI writing solutions page). That is a genuine data point in Turnitin's favor and worth citing precisely because it complicates a clean "all detectors are biased" narrative — but as later modules show, other widely used detectors were never tested this way, and Turnitin's own 1-percent figure still produces hundreds of wrongly flagged students at university scale.

OpenAI launched a free AI Text Classifier in January 2023, then discontinued it on July 20, 2023. Its own retirement notice, still live on OpenAI's site, states the reason plainly: "the AI classifier is no longer available due to its low rate of accuracy" (OpenAI). The same page discloses the numbers behind that decision: on OpenAI's internal "challenge set" of English texts, the classifier correctly identified AI-written text only 26 percent of the time, while incorrectly labeling human-written text as AI-generated 9 percent of the time. OpenAI also warned the tool was "very unreliable" on text shorter than 1,000 characters and performed "significantly worse" on non-English text.

OpenAI was not alone. Around the same time, the education platforms Quill.org and CommonLit both discontinued their AI Writing Check tools, telling reporters that generative AI had become too sophisticated to reliably detect (The Markup). Three companies with direct access to the underlying language models — including the one that built ChatGPT — each concluded their own detectors were not accurate enough to keep running.

The most comprehensive independent test to date comes from a team of eight researchers across seven universities, led by Debora Weber-Wulff, published in the International Journal for Educational Integrity. They tested 12 free detection tools plus two commercial systems, Turnitin and PlagiarismCheck, against original and machine-translated or paraphrased text. Their conclusion: "the available detection tools are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text," and obfuscation techniques such as paraphrasing "significantly worsen the performance of tools" (Weber-Wulff et al., 2023, Int J Educ Integr 19, 26; preprint at arXiv:2306.15666).

A smaller but more precisely measured test came from SURF, the Dutch higher-education IT cooperative. Researcher Vivian van Oijen ran seven detectors — including GPTZero and OpenAI's classifier — against AI-generated text and found an overall accuracy of just 27.9 percent, with the single best-performing tool topping out at 50 percent, "no better than a coin toss." The same tools scored 83 percent accuracy on human-written text, meaning they defaulted toward calling almost everything human (van Oijen, "AI-generated text detectors: Do they work?" SURF Communities, 2023). Separately, a University of Maryland team showed that a simple paraphrasing attack can knock out most detectors, including ones based on cryptographic watermarking, without materially damaging the text's quality (Sadasivan et al., "Can AI-Generated Text be Reliably Detected?" arXiv:2303.11156). Given how widely these figures diverge — 27.9 percent in one lab, near-random in another, high-90s in Turnitin's own controlled tests — the popularly cited "detectors run 60-80 percent accurate in the real world" is best treated as a reported range repeated across secondary press coverage rather than a number traceable to one named primary study.

In April 2023, a screenshot went viral showing GPTZero declaring a section of the U.S. Constitution "likely to be written entirely by AI." Ars Technica confirmed the result and ran its own tests: ZeroGPT flagged the same text as "AI/GPT Generated," and a passage from the Book of Genesis came back 88.2 percent AI-generated according to ZeroGPT (Ars Technica, "Why AI writing detectors don't work," Jul. 14, 2023). GPTZero's creator, Edward Tian, told the outlet the cause was that founding-era legal language is so heavily represented in training data that language models learn to reproduce its patterns, which the detector then reads as a signature of AI generation. This was not a hypothetical failure mode: The Markup documented a Johns Hopkins student whose Turnitin report flagged more than 90 percent of a paper as AI-written, despite the student producing drafts, notes, and a full Google Docs revision history proving otherwise (The Markup).

You do not need a lab to see how these error rates translate into real accusations — the arithmetic is one multiplication. Take a mid-size university with 20,000 students, each submitting one paper through an AI detector during a given assignment cycle.

• Using Turnitin's own published false-positive rate (under 1%, treat as 1% for a conservative estimate): 20,000 students × 0.01 = 200 students wrongly flagged on that one assignment, even under the vendor's best-case number.
• Using OpenAI's own disclosed false-positive rate for its now-retired classifier (9%): 20,000 students × 0.09 = 1,800 students wrongly flagged on the same assignment.

Both inputs come directly from the companies that built the tools, not from critics. The gap between 200 and 1,800 wrongly accused students, from the same student population and the same single assignment, shows how much the final headcount depends on which detector — and whose disclosed number — a school happens to trust.

Stanford researchers Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou tested seven widely used GPT detectors on 91 TOEFL essays written by non-native English speakers and a comparison set of essays written by U.S.-born eighth-graders. The detectors were "near-perfect" on the eighth-grade essays but classified 61.22 percent of the TOEFL essays as AI-generated. All seven detectors unanimously flagged 18 of the 91 TOEFL essays, 19 percent, as AI-written, and 89 of the 91, 97 percent, were flagged by at least one detector (Stanford HAI, "AI-Detectors Biased Against Non-Native English Writers," May 15, 2023; paper: Liang et al., "GPT detectors are biased against non-native English writers," Patterns 4(7), 2023, arXiv:2304.02819).

The detectors mostly score text using "perplexity," a measure of how predictable the word choices are compared to a language model's training data. Non-native writers, on average, use a narrower vocabulary and simpler sentence structures than native speakers — the same statistical fingerprint that low-perplexity, formulaic AI text produces. Zou summarized the risk directly: "These numbers pose serious questions about the objectivity of AI detectors and raise the potential that foreign-born students and workers might be unfairly accused of or, worse, penalized for cheating" (Stanford HAI). Combine that bias with the false-positive math from Module 4, and the population most likely to be wrongly accused by an AI detector is disproportionately made up of the students least equipped to fight the accusation.