AI Detector — Blog

What zero-shot AI-text detection actually means

2026-08-12T00:00:00Z

If you have ever pasted a paragraph into an "AI detector" and gotten a confidence score, you have probably wondered what the number actually means. The honest answer starts with how the detector was built. Most commercial detectors are classifiers: they were trained on a large set of examples labelled "AI" or "human", and they learn the surface patterns that separate the two. That approach has a serious weakness — it is only as good as the models it was trained on.

The training-set problem

A supervised AI detector is trained on text from a fixed set of generators, plus human-written text for comparison. When a new model ships, or when a prompt style changes, the detector's picture of "AI text" goes stale. It may over-flag a human writer who happens to resemble the training set, or under-flag a rewritten AI draft. This is why detectors are frequently re-trained and why their scores shift version to version.

What zero-shot means here

A zero-shot detector does not rely on a curated training set of labelleds. Instead it measures intrinsic properties of the text itself. The core idea: a language model that generates text tends to be more predictable about its own output than a human writing the same words. A zero-shot detector runs one or more reference models over the text and compares the actual word choices to what those models would have predicted.

Perplexity / log probability. How surprised is a language model by this text? Very low surprise (a predictable sequence) is a weak signal of machine generation.
Entropy / logrank. How uncertain was the generation, and how often did the most likely token actually get chosen? Machine text is often more consistently "peak-y" than human text.
DetectGPT and friends. Instead of a single score, perturb the text (mask and re-generate words) and measure whether the original sits at a local probability maximum.

Why combine several signals

No single statistic is reliable. A short, formulaic passage — a legal boilerplate, a product spec, a textbook definition — is highly predictable whether a human or a model wrote it. A deliberately “bursty” human writer can look machine-like to a perplexity baseline. That is why this tool runs a family of zero-shot signals and reports them side by side, then fuses them into a single calibrated verdict rather than pretending one number is definitive.

Signal	What it measures	Overlap with humans
Perplexity / logprob	Mean model surprise over the text	High on formulaic prose
Entropy / logrank	Spikiness of the token choices	High on monotone writers
DetectGPT	Local peak of the probability surface	Lower — needs enough text
Fused verdict	Weighted combination + uncertainty	Reported as a range, never a fact

What zero-shot still cannot do

The limits matter as much as the method. A zero-shot detector measures statistical properties, not intent or authorship. It cannot tell you who wrote something, it cannot prove a human wrote it, and it has no reliable signal on very short text. A confident zero-shot score is a strong clue worth investigating, never a certificate.

💡 This site runs a zero-shot multi-detector engine: likelihood, entropy, logrank and perplexity baselines alongside DetectGPT, Fast-DetectGPT, Binoculars and DetectLLM, fused into an honest per-detector breakdown. Paste text on the homepage to try it.

DetectGPT, Fast-DetectGPT, Binoculars and DetectLLM: beyond the perplexity baseline

2026-08-08T00:00:00Z

The simplest way to guess whether text is machine-written is to ask a language model how predictable it is. Low perplexity, the story goes, means the text is flat and probable, therefore AI. That baseline is cheap, but it is famously brittle. The newer zero-shot detectors are refinements of the same instinct — measure the text against a model's expectations — with cleverer statistics.

The baseline: perplexity and log probability

Perplexity is the average "surprise" of a model over a sequence. If the model is confident, perplexity drops; if the text is twisty and human, it rises. The obvious failure mode is that humans write boring, predictable text all the time. Legal clauses, instructions, and form letters are low-perplexity by nature. The reverse is true too: human writers are often more varied, which can look "machine-like" to a baseline that rewards variety or punishes it inconsistently.

DetectGPT: the peak test

DetectGPT (Mitchell et al., 2023) asks a sharper question: is this text a local peak of the model's probability surface? The method masks a few words, re-generates alternatives with a smaller model, and checks whether the original has a systematically higher probability than its own perturbations. Machine text tends to sit on a local maximum; human text does not. The cost: it needs multiple forward passes, so it is slow and needs fairly long passages.

Fast-DetectGPT: the same idea, faster

Fast-DetectGPT (Bao et al., 2023) keeps the peak intuition but removes the expensive perturbation step. Instead of re-generating candidates, it uses the model's own conditional probability distribution as the reference, and compares the observed token probability to the distribution's mean. It reaches DetectGPT-level accuracy at a fraction of the sampling cost, which is what makes an interactive detector practical.

Binoculars: two-model agreement

Binoculars (Hans et al., 2024) uses two models: a "scorer" and a "perplexity" model. It measures the ratio of the scorer's log probability to the perplexity model's log probability. The clever detail is that the two models disagree systematically on machine-generated text but not on human text, which makes the score more discriminative than a single model. It is a zero-shot method with strong reported numbers, though it is calibrated to specific model pairs.

DetectLLM: complementary views

DetectLLM (Yang et al., 2024) combines a log-rank metric (where the chosen token ranks in the model's distribution) with an entropy metric, plus a context-aware comparison between the base text and a re-generated version. By running two complementary measures and combining them, it hedges against the failure modes of any single statistic.

How they stack up

Method	Core idea	Speed	Main weakness
Perplexity baseline	Mean model surprise	Fast	Fooled by any predictable prose
DetectGPT	Local probability peak	Slow (sampling)	Long text, costly
Fast-DetectGPT	Peak vs. distribution mean	Fast	Needs a good reference model
Binoculars	Two-model score ratio	Fast	Sensitive to model pairing
DetectLLM	Log-rank + entropy fusion	Medium	Calibration matters

Why run all of them

No single method wins everywhere. A passage can be confident under one metric and ambiguous under another, and the disagreements are often where the interesting cases live. Reporting a fused verdict with the per-detector breakdown — rather than one black-box percentage — lets you see why a text was scored the way it was, and to treat agreement across independent methods as stronger evidence than any single score.

💡 None of these methods certifies authorship. They estimate a statistical property, and like every detector they produce false positives and false negatives. Use them as a prompt to look closer, not as a verdict.

Thresholds, calibration, and the false-positive problem

2026-08-04T00:00:00Z

A detector does not output truth. It outputs a score, and someone has to decide what score means "probably AI." That decision — the threshold — is where most real-world problems come from. It is not a technical curiosity; it is a policy trade-off that decides who gets falsely accused.

The two ways to be wrong

Every binary decision has two error modes. A false positive is human text flagged as AI. A false negative is AI text that slips through. You can reduce one only by increasing the other. There is no threshold that makes both zero, because the two score distributions overlap.

You cannot make a detector perfectly accurate. You can only choose which mistakes it makes, and who pays for them.

Where the threshold lands

Set the threshold high and you stop bothering honest writers, but you also miss a lot of genuinely generated content. Set it low and you catch more, at the price of accusing people who wrote every word themselves. The "right" setting depends entirely on the cost of each mistake. A teacher deciding whether to refer a student to a conduct panel has a very different tolerance than a publisher triaging a large inflow of submissions.

The deeper issue: calibration

A well-calibrated score is one where a 0.8 means "about 80% of the time this is machine-generated." Most detectors are not calibrated. Their raw scores are arbitrary — a 0.8 from one detector is not comparable to a 0.8 from another, and may not correspond to any real probability at all. Plotting the false-positive rate against the true-positive rate (the ROC curve) shows how well a detector orders texts, but says nothing about whether a particular number is trustworthy on its own.

What honest detection looks like

Report calibrated probabilities where possible, and make it clear when they are not calibrated.
Show the per-detector breakdown, so a single number is never treated as a verdict.
Surface uncertainty explicitly — a wide spread between detectors means the text is genuinely ambiguous.
Tell users when the evidence is too thin to judge (too short, too formulaic, too many edits).

💡 This tool offers operating presets — a balanced mode and a low-false-positive mode — so you can choose where the dial sits for your own use case, and it reports the spread across detectors rather than hiding it.

The only honest practice

Because no threshold is universally right, the responsible use of a detector is to treat it as one piece of evidence in a larger review, never as a verdict machine. Calibration and transparency are what let you use the number sensibly. A detector that promises certainty avoids the hard part — and is the one you should be most careful trusting.

Why no AI detector can certify that a human wrote something

2026-07-28T00:00:00Z

The most common misuse of an AI detector is treating a score as a verdict: "this text is 97% AI, therefore the student cheated." That inference is not supported by what a detector actually measures. A detector estimates how much a passage resembles text a language model would produce. That is not the same as who wrote it, and it never can be.

Detection is a fact about the text, not the person

A person can produce text that looks machine-like. They can draft carefully, edit with a model's help, imitate a flat style, or write a very formulaic passage by hand. Conversely, a person can lightly edit AI-generated text and make it look more human — or a model can deliberately produce "bursty" text that reads as human. The same words, and the same detector score, can arise from completely different authorship stories.

The two directions of failure

False accusation. A careful human writer who drafts in a predictable style, or who writes about a well-defined topic, can be flagged. This is the harm with the highest cost, because it targets people who did nothing wrong.
False assurance. Heavily edited or humanized AI text passes. A clean score does not mean a human wrote it — it usually means the text was edited enough to become statistically ambiguous.

The editing reality

The strongest limit is that detection is not robust to editing. Change enough words, reorder sentences, or rewrite with a different model, and the statistical fingerprint shifts. A detector can reasonably flag raw generator output, but it degrades quickly as text is adapted. So a clean score on edited text is not evidence of human authorship — it is often just evidence that someone edited it.

What detection is good for

Detection is still useful, if you use it for what it is. It is a screening signal that can flag text worth a closer look. It can surface a suspiciously uniform paragraph in a large corpus. It can tell you that a passage is statistically consistent with a model's output. Those are genuine, practical uses.

A detector gives you a reason to ask a question. It never gives you the right to skip the conversation.

The honest rule

Never use a detector score as the sole basis for an accusation, an academic penalty, a hiring decision, or a published claim about authorship. Use it as a prompt to investigate, and lean on the human evidence — drafts, process, interviews, authentic — when a claim matters. Any tool that tells you otherwise is selling you a certainty it does not have.

What the EU AI Act Article 50 means for AI-text detection

2026-07-20T00:00:00Z

The EU AI Act is the first comprehensive attempt to regulate AI, and its Article 50 is the transparency obligation that touches most readers. It requires certain AI-generated content to be disclosed and, in some cases, to be machine-readable. It is a disclosure duty — not a detection duty.

What Article 50 requires

Providers must mark synthetic audio, image and video so it can be detected as artificially generated or manipulated (Article 50, paragraph 4).
Providers of text-generating systems must ensure output is marked in a machine-readable format and detectable as AI-generated — "where technically feasible", and with explicit carve-outs (Article 50, paragraph 2).
Deployers of synthetic media must disclose that content was artificially generated or manipulated (Article 50, paragraph 3).
Text transparency carries important exceptions, notably for "assistive" use where the text is substantially edited or where disclosing would be disproportionate.

The technical caveat that matters

Note the qualification in Article 50: marking text is required "taking into account the limitations of the state of the art" and only where technically feasible. That caveat exists because of the very limits this site documents — a text watermark can be removed by rewriting, and no detector reliably certifies authorship. The regulation does not pretend otherwise.

Transparency is a duty to disclose, not to detect

The EU AI Act mostly obliges the producer and deployer of AI content to be upfront. It does not impose a duty on a third party to verify authorship, and it certainly does not make a detector's score a legal finding of fact. If you publish AI-generated content, Article 50 asks you to disclose; it does not give a detector the power to prove a human wrote something.

Where detection belongs

Detection plays a supporting role. It can identify text that ought to have been disclosed but was not, and it can flag synthetic media where a watermark has been stripped. It is a compliance aid and a screening tool. It is not a substitute for disclosure, and it is not proof of anything on its own.

💡 This tool reports detection signals and honest uncertainty so you can reason about provenance. It does not certify that text is or is not AI-generated, and it does not make legal determinations. For disclosure obligations, the responsibility to be transparent rests with the person publishing the content.

The practical takeaway

If you work in a regulated context, treat Article 50 as a process you must follow — disclose where required — and treat detection as an internal check. The two are complementary. Confusing them, either by expecting detection to prove authorship or by assuming disclosure makes detection unnecessary, is what causes most of the trouble.

How AI-text detectors are actually evaluated: AUROC, AUPRC, and the benchmark problem

2026-07-14T00:00:00Z

Detector marketing is full of headline accuracy figures. The problem is that accuracy is nearly meaningless without a benchmark behind it. Two detectors can both claim "95%" and be entirely different tools. To read a claim honestly you have to ask a different question: evaluated on what, and measured with which metric?

Why "accuracy" is the wrong headline

A detector's job is not to be right on average; it is to separate two overlapping distributions. If 99% of your corpus is human-written and the detector never flags anything, it is 99% accurate and completely useless. Accuracy is sensitive to class balance, which is exactly why it is a poor headline metric.

AUROC: ranking quality

AUROC (Area Under the Receiver Operating Characteristic curve) measures how well a detector ranks texts. It asks: if I pick one human text and one machine text at random, how often does the detector score the machine text higher? A perfect detector scores 1.0; a coin flip scores 0.5. AUROC is threshold-independent, which makes it a good summary of ordering ability — but it tells you nothing about what score to trust, or how many false positives you will get at a chosen cutoff.

AUPRC: how much you'll believe a positive

AUPRC (Area Under the Precision-Recall curve) is what you care about when you will act on a positive result. It combines precision (of the flagged texts, how many really are AI) with recall (of all AI texts, how many got flagged). AUPRC is far more sensitive to class imbalance than AUROC, and it is the metric that actually reflects the false-positive problem. When a vendor quotes a high AUPRC, they are saying "when I flag something, you can usually trust the flag" — which is the claim most buyers actually want.

The benchmark problem

Results depend entirely on the dataset. A recent effort standardised this with the HWT (Human-Written vs Machine-Generated) benchmark and the related MGT (Machine-Generated Text) collections, which gather text from many models in many styles and pair it with matching human writing. Detectors that score well on a narrow, old dataset can collapse on a newer model or a different genre. Generalisation is the hard part, and a single benchmark cannot capture it.

Metric	What it answers	Watch out for
Accuracy	% correct overall	Useless when classes are imbalanced
AUROC	Can it tell machine text from human text, in order?	Not about any single cutoff
AUPRC	When it flags text, can I trust the flag?	Very sensitive to dataset balance
Per-model AUROC	How it performs per generator	Averages hide big variation

How to read a vendor claim

Ask for AUROC and AUPRC on a named, current benchmark, not a one-line accuracy figure.
Ask for per-model results — a detector that is great on one generator and blind to another is not a general detector.
Ask for the false-positive rate at the operating threshold, because that is what decides who gets accused.
Ask whether the detector is zero-shot or trained on the evaluation models — a model that saw the generator in training will over-perform.

💡 This tool publishes a fused verdict with per-detector scores so you can see how the methods agree, and it surfaces uncertainty rather than a single black-box percentage. No benchmark number is a promise of perfection.

The state of AI-text detection in 2026

2026-08-18T00:00:00Z

AI-text detection has moved from a curiosity to a daily responsibility for teachers, editors, publishers, and reviewers. This is a 2026 snapshot: what has genuinely improved, what has not, and how to use a detector without getting burned.

What has improved

The practical gains have been in efficiency and interpretability rather than magic. Fast-DetectGPT made the probability-peak idea cheap enough to run interactively. Binoculars showed that comparing two models' log-probabilities is more discriminative than a single score. And multi-detector engines like the one behind this site make it possible to see agreement across methods instead of one opaque number.

What has not changed

Detection is still a statistical estimate, not a proof of authorship. That limit does not move.
Editing and rewriting still defeat detectors. Heavily edited AI text becomes statistically ambiguous, which means clean scores are not evidence of humans.
False positives still happen, and they still land on real people who did nothing wrong.
Vendors still overclaim. Prefer tools that show their work over tools that quote a single confidence.

How regulation changed the conversation

Transparency rules — most notably the EU AI Act's Article 50 — have shifted responsibility toward disclosure by producers and deployers rather than detection by third parties. That is a meaningful change: it puts the onus on the person generating content to be upfront, and it makes detection a compliance aid rather than the front line. Text watermarking, where it is technically feasible, is the preferred mechanism; where it is not, the same caveats about rewritability apply.

The practical playbook

Use a detector as a screening signal. A strong result is a prompt to look closer, not a verdict.
Review the per-detector breakdown. When independent methods agree, the signal is stronger; when they disagree, the text is genuinely ambiguous.
Never act on a score alone when the consequences matter — especially an accusation. Get drafts, process, or an authentic sample.
Prefer calibrated probabilities and stated thresholds over raw confidence.
Disclose when you publish AI-generated content, and treat detection as a check, not a substitute.

Where this is heading

Expect better fusion of existing signals, more attention to calibration, and a closer tie between detection and disclosure obligations. Do not expect a detector that proves authorship — that goal is at odds with the fundamental limits of the methods. The realistic future is more honest, more transparent detection that knows its own boundaries.

💡 The honest rule: use detection to decide whether to look closer, not to decide whether someone is guilty. That single habit avoids the vast majority of harm that AI-text detection can cause.