AI Detector Accuracy Tracker: Do AI Detectors Actually Work? (Q3 2026 Reading)

Line GPTZero and the two best open-weight detectors up against 861 sentences that provably predate large language models, and 23.3% of those sentences get flagged as AI by at least one of them. Only 2.7% get flagged by all three. That gap is the finding. The detectors are not merely wrong at a rate — they are wrong about different sentences, which means "AI detector accuracy" is not a number you can look up. It is a spread, and where you land in it depends on which tool your professor, editor, or client happens to have open.

Finding those two open-weight detectors took a panel of six. On August 19, 2026 we pushed the frozen corpus through six AI text detectors whose model weights are published, so anyone can reproduce the run. Before measuring false positives we checked each one could tell AI text from human text at all. Three of the six failed. Two scored statistically indistinguishable from a coin flip; the 2019-vintage RoBERTa OpenAI detector came back inverted, scoring modern human sentences as more machine-like than modern AI sentences. All three are free downloads that anyone can wrap in a web form and sell as an "AI checker" tomorrow.

Three survived. The two best flagged 9.9% and 10.0% of the human sentences; the third, MELD, flagged 20.7% and is tracked separately below. Three months earlier, GPTZero's commercial API flagged 13.8% of the identical corpus. So the headline number drifted slightly in the right direction — and is still roughly one real person's sentence in ten.

What This Tracker Measures (and Why It's the Reverse of Every Vendor Benchmark)

Every published detector benchmark we could find runs in one direction: feed the tool AI-generated text, count how much it catches. That produces the numbers on the marketing pages — 98%, 99%, 99.1%. It tells you almost nothing about the question that actually ruins people's weeks, which is how often does this thing accuse a real writer?

So we run it backwards. The input is a frozen corpus of 861 sentences that we can prove no language model wrote, because they predate the models: English Wikipedia articles, PubMed abstracts published 2015-2019, Wikipedia coverage of 2018-2019 news events, and ESL learner writing posted to Reddit before 2024. Every sentence is 20-50 words, mean 27.2 — roughly what people actually paste into a detector. Nothing in it is customer content, so the corpus stays frozen and publishable, and every future reading uses the exact same sentences. That's the whole point of a tracker: change the detector, never the input.

A sentence counts as a false positive when a detector scores it above 0.5 on its own AI-probability scale. Where a detector returns a categorical verdict instead — GPTZero — we count "ai" and "mixed" as flags. Full methodology sits under the sources at the bottom.

Readings So Far

The Q3 2026 Reading: Three of Six Detectors Failed Before We Could Even Score Them

Here is the part that surprised us. Before measuring how often a detector flags human writing, you have to check that it can distinguish AI writing at all — otherwise a great-looking false-positive rate just means the tool says "human" to everything. So each detector got a validation pass first: 373 AI-written sentences pulled from our own frozen test set, split and sampled to match the human sentences' length distribution (mean 26.8 words vs 27.2), then scored for AUROC.

Three of the six landed at or below chance.

TMR flags 90.2% of verified human sentences — 97.3% of the news ones. Fakespot's v1 checkpoint flags 63.9%. The 2019-vintage RoBERTa OpenAI detector is worse than useless: at AUROC 0.375 it scores modern human sentences as more machine-like than modern AI sentences, because it learned GPT-2's sampling quirks and 2026 models don't produce them. All three of these are free downloads. Any of them can be wrapped in a web form and marketed as a "99% accurate AI checker" by Friday, and you would have no way to tell from the outside.

That is the practical warning buried in this reading. When a free online AI checker returns a scary percentage, the failure mode you should suspect first isn't a sophisticated model catching you out. It's an obsolete or miscalibrated classifier that says "AI" to almost everything.

The Detectors Don't Agree on Which Sentences Are AI

Individual false-positive rates undersell the problem, because they hide whether the detectors are wrong about the same sentences. They are not.

Of GPTZero's 119 false positives from May, Vanguard cleared 88 of them. Of Vanguard's 85 flags in August, GPTZero had cleared 54. The two tools are wrong about largely different sentences written by largely different people.

Stack that up and the number that matters gets worse, not better:

• 23.3% of the 861 verified-human sentences were flagged by at least one of GPTZero, Vanguard and Gradient (201 sentences).
• 2.7% were flagged by all three (23 sentences).
• Add MELD, which also passed validation, and "flagged by at least one of four" reaches 38.7%.

This is a smaller-scale reproduction of the finding that made Liang et al. (2023) famous — 97.8% of TOEFL essays by non-native writers were flagged by at least one of seven detectors — except ours runs on ordinary sentence-level prose from four different corpora, three years later, with a detector generation that didn't exist when they wrote it. The multi-detector union effect has not gone away. If somebody runs your work past enough checkers, one of them will eventually accuse you.

Which Human Writing Gets Falsely Flagged

Break the false positives down by corpus and the same ranking appears in all three of the detectors compared below, across two generations and both the commercial and open-weight families.

News prose is the worst-hit category for all three of those detectors, in both readings. Dense with dates, named entities, figures and formal constructions, written to a house style that strips idiosyncrasy — that is a very good description of low perplexity and low burstiness, which is exactly what the classifiers are trained to punish. ESL writing sits second, consistent with the entire published literature on detector bias against non-native English writers, and with the ESL sub-corpus in our own 861-sentence GPTZero study.

The corollary is uncomfortable: the writing most likely to get falsely accused is disciplined, factual, edited writing. Sloppiness is a defense.

The Failure Mode Isn't Uncertainty — It's Confidence

The score distribution says something the headline rate hides. Most of the time the detectors are confidently, correctly quiet: 73.5% of the human sentences score under 0.1 on Vanguard. But the errors aren't clustered in the murky middle where you'd want them. Seventeen verified human sentences score above 0.9 — the detector is more than 90% certain a real person's sentence was machine-written.

An "80% AI" score on a page you wrote yourself is not a rare glitch. On this corpus it happens to roughly 3% of human sentences on Vanguard and 2% on Gradient. Scale that to a 40-sentence essay and the chance that at least one of your sentences draws a high-confidence AI score is not small.

What This Reading Cannot Tell You

Four limits, stated up front rather than buried, because a tracker that hides its caveats is just marketing with a chart.

Sentences are harder than paragraphs. Vanguard scored AUROC 0.854 here. The same model scored 0.986 in our July paragraph-level validation. Single sentences carry less signal, so both the discrimination and the false-positive numbers on this page describe the short-text case: chat replies, social posts, abstracts, ad copy, and any tool that scores a document sentence by sentence. A 900-word essay judged as one blob is an easier problem for the detector.

Open-weight detectors are not stand-ins for the commercial ones. Vanguard is not a proxy for Turnitin, and this tracker never treats it as one. Published comparisons put the rank correlation between open detectors and GPTZero at roughly 0.11 to 0.21, and our own κ of 0.21 reproduces that independently. That is precisely why every reading in the table above is reported per detector rather than averaged into a single "detector accuracy" figure. Averaging tools that disagree this much produces a number that describes nothing.

The AI side of the run is for validation only. The 373 AI-written sentences exist to check that a detector can discriminate at all. No false-positive rate on this page is computed from them. If we had skipped that step, TMR would look like a 90%-recall detector instead of a broken one.

GPTZero's row is cited, not re-measured. Our May 2026 GPTZero API key is no longer provisioned and we did not buy a new one for this reading, so the Q2 number comes from the archived run rather than a fresh call. Vendors update models silently; GPTZero's current production behavior may differ from 2026-05-11-base. Treat the Q2 row as a dated measurement, which is what it is.

How We Got Here: Three Years of AI Detector Accuracy

Two readings do not make a trend on their own. They land differently against the three years behind them, because the striking thing about this category is how little the measured numbers have moved while the marketing numbers stayed pinned at 98-99%.

In January 2023 OpenAI shipped its own detector, built by the lab that made the text generator. The launch disclosed a 26% true-positive rate and a 9% false-positive rate, and described the tool as having "many limitations" that "should not be used as a primary decision-making tool." Six months later OpenAI withdrew it, leaving a note that is still on that page: "the AI classifier is no longer available due to its low rate of accuracy." No replacement has shipped since.

The independent audits arrived the same year and have never been overturned. Weber-Wulff et al. tested 14 commercial tools across English, Spanish, and German and concluded that "none of the tools tested was accurate or reliable." Liang et al., in Patterns, ran TOEFL essays by non-native English speakers through seven detectors: a 61.3% average false-positive rate, and 97.8% of the essays flagged by at least one detector. Meanwhile Turnitin had switched AI detection on across its platform with no institutional opt-out, and by August 2023 the Washington Post was reporting on students trying to prove a negative.

Then the customers started leaving. Vanderbilt disabled Turnitin's AI detector in August 2023 and published its reasoning. Yale, the University of Waterloo, Washington State, and a long tail of others followed through 2024 and 2025, most citing bias and false positives. We track that landscape separately in our review of university AI detection policies.

Our Q2 and Q3 readings are the newest data points on that line, and they land where the 2023 audits said they would: inside the double-digit band, at the lower end of it, three model generations later.

Vendor Claims vs Independent Measurement

Set what the vendors advertise against what anyone outside the vendor has actually measured. We only fill the measured column where a citable measurement exists — ours or someone else's. Blank means blank, not zero.

Two things stand out. The gap between what vendors advertise and what anyone outside the vendor measures has never closed — it is still tens of percentage points wide on every row with an independent number. And the only entry that ever revised its claim downward is the one that shut itself down, which is also the only one that was not selling a subscription. Full 861-sentence GPTZero breakdown, including the top-10 falsely flagged sentences.

Why the Numbers Won't Move Much

The plateau is not laziness. The detection vendors have raised real money and shipped real updates. The problem is structural, and it sits in what the classifiers can measure.

Almost every commercial AI detector scores text on two features: perplexity (how surprising each word is, given the words around it) and burstiness (how much that surprise varies across a document). LLM output tends to be low and flat on both. So is a lot of human writing — formal academic prose, technical documentation, house-style journalism, and especially non-native English. The classifier cannot separate them because at that level of representation there is nothing to separate. That is the mechanism behind every row of the per-corpus table above, and we unpack it further in our review of AI detection false positives.

More training data does not fix it. As frontier models got better at fluent, varied prose, the gap between machine and human perplexity narrowed rather than widened — the detectors are reading a smaller signal each year, not a larger one. This is exactly the reason OpenAI gave when it pulled its own classifier, and the reason the labs pivoted to provenance instead. Google DeepMind's SynthID text watermarking, published in Nature in 2024, marks text at generation time instead of trying to classify it afterward. Anthropic has never shipped a detector at all. The three labs producing most of the world's LLM text have concluded that downstream classification is the wrong approach, while detection vendors continue to advertise 98-99%. Both positions cannot be correct.

Watermarking would genuinely fix this, by turning a classification problem into a verification problem. It requires OpenAI, Anthropic, and Google to adopt a shared standard at the same time, because a single-vendor watermark is stripped by routing through any other model. As of August 2026 no such standard has shipped, and the realistic timeline is years.

What To Do If a Detector Flags You

Meanwhile the scores keep appearing on student papers, hiring screens, freelance assignments, and admissions essays — read by people who mostly do not know what κ 0.21 means. That asymmetry is what makes the category harmful regardless of where the headline accuracy sits.

What actually helps, roughly in order of effort:

• Collect process evidence before you argue. Version history, drafts, notes, research trail. The fact pattern winning appeals right now is "low-perplexity writing, vendor-disclaimed score, no corroborating evidence" — and the corroborating evidence is the part you control. Our falsely accused of using AI guide is the step-by-step version.
• Ask which tool, at what threshold. A score is meaningless without both. If the answer is a free web checker, the tables above are your argument: three of six downloadable detectors in our panel could not beat a coin flip, and the worst flagged 90.2% of verified human sentences.
• Disclose where policy permits. Most 2026 policies allow AI assistance with disclosure, and disclosure removes false-positive risk entirely because the question stops being adversarial.
• Vary the prose that gets flagged. Mix sentence lengths, cut formulaic transitions, add specifics a model wouldn't reach for. Those edits break the low-and-flat signal and generally produce better writing anyway. An AI humanizer like ToHuman does the same thing at scale by reintroducing perplexity and burstiness variation — it routes around the classifier rather than fixing it, which is the honest description of what any tool in this category does. ToHuman's free plan covers 2,500 humanized words a month, no card required, and the homepage demo takes up to 700 characters with no signup.

If you write in non-native English, the bias is structural and documented across the entire literature, and the ESL-specific appeal steps sit inside our falsely accused guide. If your flag came from Turnitin specifically, what Turnitin's AI detection actually does in 2026 covers the 20% floor and the vendor's own disclaimers.

The Next Reading: Q4 2026

The same 861 sentences go through the panel again in November 2026. The corpus never changes; only the detectors do. Three things we want to answer next quarter: whether a re-provisioned commercial API still lands near 13.8%, whether the κ between commercial and open-weight detectors moves off 0.21, and whether news prose stays the worst-hit corpus for a fourth consecutive detector generation.

If the false-positive rate ever falls to the low single digits on this corpus, we will publish that with the same prominence as everything above. So far, across three detector generations and two vendor families, it has not come close.

Frequently Asked Questions

Do AI detectors work yet?

Partially. In our Q3 2026 reading, three of six downloadable AI detectors could not separate AI text from human text better than a coin flip (AUROC 0.375 to 0.469). Three passed validation, and the two best still flagged 9.9% and 10.0% of 861 verified human-written sentences as AI; the third flagged 20.7%. They work well enough to be believed and badly enough to be dangerous, which is the worst combination for the person on the receiving end of a score.

Are AI detectors actually accurate?

It depends which number you measure. On AI-generated text, the better detectors separate reasonably well — the strongest in our Q3 2026 panel scored AUROC 0.858. On human writing, the two detectors with the lowest false-positive rates still flagged roughly one sentence in ten, rising to 21.6% on news prose. Vendor pages advertise 98% to 99% accuracy from internal test sets that independent audits have never replicated. Weber-Wulff et al. tested 14 tools in 2023 and concluded that none of them was accurate or reliable enough for academic use.

Which AI detector is the most accurate?

That is the wrong question, and our own data shows why. On the identical 861 sentences, GPTZero and Vanguard, the open-weight detector with the lowest false-positive rate, agreed at Cohen's kappa 0.213 — "fair" agreement, barely above chance. They are not wrong about the same sentences. 23.3% of the corpus was flagged by at least one of GPTZero, Vanguard and Gradient, and only 2.7% by all three. Choosing the most accurate detector does not protect you, because your exposure depends on which tool the person judging your work happens to open.

Is 40% AI detection bad?

It is not proof of anything. A 40% score means the tool flagged part of the text, not that 40% of it was machine-written. On our frozen corpus of prose we can prove predates large language models, 23.3% of sentences were flagged by at least one of the three detectors in this comparison, and 36.9% of the news sentences were. Scores in that range occur on writing no machine touched. Treat a percentage as a reason to look at process evidence, never as a verdict.

My AI detector thinks my writing is AI. What should I do?

Do not panic-rewrite. Collect process evidence first — version history, drafts, notes, research trail — because that is what appeals are actually won on. Then ask which tool produced the score, at what threshold, and whether your institution's own policy allows a detector score to stand as sole evidence. Most policies now say it cannot. ToHuman's guide to being falsely accused of using AI is the step-by-step appeals playbook.

Can Turnitin actually detect AI writing?

Turnitin ships AI writing detection and has claimed 98% accuracy on its own testing, but the same product withholds a percentage for scores below 20%, which its documentation says are less reliable, and that same documentation says the score should not be the sole basis for adverse action against a student. Vanderbilt disabled the feature in August 2023 and the University of Waterloo discontinued it, both citing false positives. Turnitin has no public API, so this tracker does not measure it — we cite its documentation and institutional decisions rather than inventing a number.

Sources

1. OpenAI — New AI classifier for indicating AI-written text (Jan 2023, with July 2023 discontinuation note)
2. Liang et al., "GPT detectors are biased against non-native English writers," Patterns (2023)
3. Weber-Wulff et al., "Testing of detection tools for AI-generated text," International Journal for Educational Integrity (2023)
4. Washington Post — He used AI to cheat. Now he's accused of cheating differently. (Aug 2023)
5. Vanderbilt University — Why we're disabling Turnitin's AI detector (Aug 2023)
6. Yale Poorvu Center — AI Guidance for Teaching
7. University of Waterloo — Waterloo discontinuing the use of AI detection tool Turnitin.com (September 2025)
8. NBC News — College students turn to AI to beat AI cheating detectors
9. The Serials Librarian — AI Detection Unfairly Accuses Scholars of AI Plagiarism (2024)
10. Google DeepMind — Scalable watermarking for identifying large language model outputs (SynthID Text), Nature (2024)
11. Turnitin — AI Writing Detection product documentation
12. ToHuman primary research — 861 human-written sentences through GPTZero, May 2026 (13.8% false-positive rate) — the Q2 2026 reading and the parent methodology for this tracker
13. Vanguard AI text detector — model weights and card (Hugging Face)
14. Gradient AI text detector — model weights and card (Hugging Face)
15. RoBERTa base OpenAI detector (2019) — model weights and card (Hugging Face)
16. Fakespot AI text detection v1 — model weights and card (Hugging Face)

Methodology: The corpus is 861 sentences of 20-50 words (mean 27.2) drawn from English Wikipedia, PubMed abstracts published 2015-2019, Wikipedia coverage of 2018-2019 news events, and ESL learner writing posted to Reddit before 2024 — all of it verifiably predating the models being tested, none of it customer content. It is frozen: every reading uses the identical sentences. The Q2 2026 reading submitted that corpus to GPTZero's v2/predict/text endpoint at model version 2026-05-11-base on May 20, 2026, counting "ai" and "mixed" verdicts as flags; that key is no longer provisioned, so the Q2 row is cited from the archived run rather than re-measured. The Q3 2026 reading ran on August 19, 2026 against six open-weight detectors executed locally on MPS, scoring a sentence as flagged at AI-probability > 0.5; raw output is bench/results/tracker_q3_2026.json. Each detector was first validated for discrimination using 373 AI-written sentences from our frozen internal test set, length-matched to the human corpus (mean 26.8 words); those AI sentences are used only for AUROC and never enter a false-positive rate. Cohen's κ is computed on binary flag agreement over the same 861 sentences. Vendor accuracy claims are quoted from each vendor's current documentation and are self-reported. Last updated August 20, 2026.