How Accurate Are AI Detectors? What the Research Says for Students and Teachers

How accurate are AI detectors?

The short answer is that AI text detectors can be right much of the time under test conditions, but they make errors that matter when a student's grade or record is at stake. Accuracy depends heavily on the tool, the length of the text, the language, whether the AI text was edited, and the threshold the tool uses.

The clearest early warning came from OpenAI, the company behind ChatGPT. Its January 2023 classifier correctly identified 26% of AI-written text as 'likely AI-written' and wrongly labelled human-written text as AI-written 9% of the time on its English challenge set ^[1]. OpenAI made it unavailable from 20 July 2023 'due to its low rate of accuracy' ^[1]. In its current help page for educators, OpenAI answers the question 'Do AI detectors work?' with 'In short, not in our experience' ^[2].

Independent research has reached similar conclusions for many tools. A 2023 study of 12 publicly available tools and two commercial systems, Turnitin and PlagiarismCheck, concluded that the tools were 'neither accurate nor reliable' and tended to classify text as human-written rather than detect AI text ^[4]. Later work is more mixed: a September 2025 working paper found that commercial detectors outperformed an open-source one, and that one product, Pangram, had near-zero error rates in its tests ^[10]. Results from one study, on one data set, do not transfer automatically to every classroom.

How do AI detectors work?

Most detectors are classifiers: software trained to sort text into 'human' or 'AI' and to give a score. Turnitin explains the idea in terms of word probability. Large language models (LLMs), the systems behind chatbots such as ChatGPT, tend to choose the next word in a 'consistent and highly probable fashion', while human writing tends to be 'inconsistent and idiosyncratic' ^[5]. Its classifiers are trained to detect those differences ^[5].

A related measure is perplexity, which roughly describes how surprising a text is to a language model. Low-perplexity text is predictable. Liang and colleagues showed that this signal can mislead: essays by non-native English writers had significantly lower perplexity, which helps explain why detectors flagged them ^[3]. Turnitin, for its part, says its own model is not explicitly programmed to evaluate perplexity and instead learns statistical patterns from its training data ^[5].

What does a detector score mean?

Turnitin assigns each sentence a score between 0 and 1, labels sentences above a threshold as AI, and reports the share of evaluated sentences above that threshold as the document score ^[5]. It skips lists, headers, references, tables, formulas and code, so the percentage is not necessarily a share of the whole submission ^[5]. A score is a probability-based estimate, not a record of how the text was written.

Is watermarking different?

Yes. A watermark is a hidden statistical pattern added while the text is generated. Google DeepMind's SynthID-Text changes only the sampling step and can be detected without running the original model; a live experiment covered nearly 20 million Gemini responses ^[12]. Its authors say watermarks are not a complete solution: they only work if the company generating the text applies them, open-source models are hard to cover, and watermarks can be weakened by paraphrasing or attacked by stealing, spoofing and scrubbing ^[12].

What does the research say about false positives and bias?

A false positive is human writing wrongly labelled as AI. In education it is the more serious error, because it can lead to an accusation against an honest student. A false negative is AI text that passes as human.

Non-native English writers

In a study by Liang and colleagues, seven widely used detectors were tested on 91 human-written TOEFL essays from a Chinese educational forum and 88 essays by US eighth-grade students ^[3]. The detectors were nearly perfect on the US essays but misclassified more than half of the TOEFL essays, with an average false positive rate of 61.22% ^[3]. All seven detectors flagged 18 of the 91 TOEFL essays (19.78%), and 89 of the 91 (97.80%) were flagged by at least one detector ^[3]. When ChatGPT was used to enrich the vocabulary of those essays, the average false positive rate dropped to 11.77% ^[3]. Asking ChatGPT to simplify the US essays 'as if written by a non-native speaker' raised it from 5.19% to 56.65% ^[3].

Turnitin, which was not part of that study, disputes that the result applies to its product. It notes that the TOEFL essays were all under 150 words and that it does not score such short documents ^[6]. In its own 2023 test, for documents of at least 300 words, the difference in false positive rate between second-language and first-language English writers was small and not statistically significant ^[6]. That evaluation was run by the vendor itself.

Why do small error rates add up?

Vanderbilt University did the arithmetic when it switched off Turnitin's AI detector in August 2023. Turnitin had claimed a 1% false positive rate at launch; Vanderbilt had submitted about 75,000 papers in 2022, so around 750 could have been wrongly flagged ^[7]. The same reasoning applies to any detector: a rate that sounds small can still affect many people when millions of documents are checked.

Can AI detectors be fooled by editing or humanizer tools?

Often, yes, though newer tools claim to resist it. In the 2023 test of 14 tools, average accuracy (binary scoring) was 94% for human-written text but only 30% for AI text that had been manually edited and 15% for AI text run through a machine paraphrasing tool ^[4]. Content obfuscation, meaning deliberate changes to hide the AI origin, significantly worsened the tools' performance ^[4].

The RAID benchmark, with more than 6 million generations from 11 models across 8 domains, 11 adversarial attacks and 4 decoding strategies, found that the 12 detectors it evaluated were 'easily fooled' by attacks, changes in sampling settings, repetition penalties and unseen models ^[8]. Its authors note that many detectors claim 99% accuracy or more but are rarely tested on shared, challenging data ^[8]. Sadasivan and colleagues showed that repeatedly paraphrasing AI text with another model can significantly cut detection rates while only slightly degrading quality in many cases ^[9]. They also show that, as AI text becomes closer to human text, the performance of even the best possible detector declines ^[9].

Vendors have responded. Turnitin says it can identify AI text modified by paraphraser or 'bypasser' tools, also called humanizers, but only for English submissions ^[5]. The 2025 NBER working paper reports that Pangram's low error rates held up against humanizer tools and very short passages in its tests ^[10]. These are encouraging results, but they describe performance at one point in time against the tools available then. The authors also stress that decision-makers weigh the two kinds of error differently and propose setting an explicit tolerance, or 'policy cap', for them ^[10].

Can teachers spot AI writing better than software?

Some can. In a 2025 study, annotators read 300 non-fiction English articles and labelled each as human or AI-written ^[11]. People who frequently used LLMs for writing were very good at it: the majority vote of five such 'expert' annotators misclassified only 1 of the 300 articles, outperforming most commercial and open-source detectors the authors evaluated, even when paraphrasing and humanization were used ^[11]. The experts relied on 'AI vocabulary' but also on formality, originality and clarity ^[11].

That finding comes with limits. It used a panel of five experienced readers and a majority vote, not a single teacher grading quickly, and it covered English non-fiction articles rather than student coursework ^[11]. It suggests that informed human judgement plus evidence beats a single number, not that intuition alone should decide a case.

How should teachers use AI detector results fairly?

The tool makers themselves warn against treating a score as proof. Turnitin states that it 'does not make a determination of misconduct' and that the AI percentage 'should not be used as the sole basis for action or a definitive grading measure' ^[5]. OpenAI says ChatGPT cannot tell whether a text is AI-generated and that its answers to questions like 'did you write this?' are random ^[2].

1. Treat a flag as a reason to look closer, not a verdict. Turnitin describes its score as data to inform a decision under the institution's own policy ^[5].
2. Check the conditions. Turnitin does not process documents with fewer than 300 words in evaluated sentences, and it does not display scores below 20% in its report ^[5]. Short or list-heavy work gives weak signals.
3. Remember the error range. Turnitin says that when it reports 50% likely AI text, the document could contain as much as 65% AI writing ^[5]; the tool is tuned to miss some AI text rather than raise false flags, so the number is an estimate, not a measurement.
4. Gather other evidence. Drafts, notes, version history and a conversation with the student about the work are more informative than a single score. OpenAI says some teachers have found it useful to encourage students to share specific ChatGPT conversations ^[2].
5. Consider language background. OpenAI reported indications that detectors could disproportionately affect English learners and students whose writing is formulaic or concise ^[2], and Liang and colleagues found far higher false positive rates for non-native essays ^[3].
6. Set clear rules in advance. Students need to know which AI uses are allowed, such as grammar help, and which are not; the NBER authors note that encouraged uses can be hard to separate from others ^[10].

The cost of skipping these steps is real. ABC News reported in October 2025 that Australian Catholic University registered nearly 6,000 alleged misconduct cases in 2024, about 90% of them related to AI, according to internal documents; the university called those figures 'substantially overstated' ^[13]. ACU stopped using Turnitin's AI tool in March 2025, and its deputy vice-chancellor said cases where that tool was the only evidence were dismissed immediately ^[13].

What can a student do if a detector wrongly flags their work?

This is general information, not legal advice; each school has its own procedure. A detector flag is not, by the tool makers' own account, proof of cheating ^[2][5].

• Ask what evidence exists. Find out which tool was used, what score it gave and whether anything else supports the concern. Turnitin's indicator is visible only to instructors and administrators, so students may need to ask for it ^[5].
• Show your process. Drafts, outlines, handwritten notes, browser history of sources and document version history all show how the work developed.
• Offer to explain the work. Being able to discuss your argument and sources is strong evidence of authorship.
• Use the formal appeal route and keep copies of every message.
• Do not ask a chatbot to judge. OpenAI says ChatGPT's answers about whether it wrote a text have no basis in fact ^[2].

For students in Iran and other countries who write in English as a second language, the bias research is directly relevant ^[3]. Turnitin's AI detection supports long-form English, Spanish, Japanese, Modern Arabic, Dutch, German and Turkish; Persian is not on that list, and submissions in unsupported languages are not processed ^[5].

Will AI detection get more reliable?

As of 3 November 2026, the picture is a race between generators and detectors. Turnitin updated its English detection model architecture in October 2026 and lists models including GPT-5.4 and Claude Opus 4.6 among those it can detect ^[5]. Some 2025 research shows strong results for the best commercial tools ^[10], while benchmark studies show that untested conditions still fool many detectors ^[8]. Theoretical work suggests detection becomes harder as AI writing becomes more human-like ^[9].

Watermarking may help for text from providers that adopt it, but its authors describe it as complementary to other methods rather than a full answer ^[12]. Guidance from both OpenAI and Turnitin points schools towards clear AI-use policies, records of how students worked, and decisions made by educators rather than by a score ^[2][5].