🧪 DetectGPT, Fast-DetectGPT, Binoculars and DetectLLM: beyond the perplexity baseline

The simplest way to guess whether text is machine-written is to ask a language model how predictable it is. Low perplexity, the story goes, means the text is flat and probable, therefore AI. That baseline is cheap, but it is famously brittle. The newer zero-shot detectors are refinements of the same instinct — measure the text against a model's expectations — with cleverer statistics.

The baseline: perplexity and log probability

Perplexity is the average "surprise" of a model over a sequence. If the model is confident, perplexity drops; if the text is twisty and human, it rises. The obvious failure mode is that humans write boring, predictable text all the time. Legal clauses, instructions, and form letters are low-perplexity by nature. The reverse is true too: human writers are often more varied, which can look "machine-like" to a baseline that rewards variety or punishes it inconsistently.

DetectGPT: the peak test

DetectGPT (Mitchell et al., 2023) asks a sharper question: is this text a local peak of the model's probability surface? The method masks a few words, re-generates alternatives with a smaller model, and checks whether the original has a systematically higher probability than its own perturbations. Machine text tends to sit on a local maximum; human text does not. The cost: it needs multiple forward passes, so it is slow and needs fairly long passages.

Fast-DetectGPT: the same idea, faster

Fast-DetectGPT (Bao et al., 2023) keeps the peak intuition but removes the expensive perturbation step. Instead of re-generating candidates, it uses the model's own conditional probability distribution as the reference, and compares the observed token probability to the distribution's mean. It reaches DetectGPT-level accuracy at a fraction of the sampling cost, which is what makes an interactive detector practical.

Binoculars: two-model agreement

Binoculars (Hans et al., 2024) uses two models: a "scorer" and a "perplexity" model. It measures the ratio of the scorer's log probability to the perplexity model's log probability. The clever detail is that the two models disagree systematically on machine-generated text but not on human text, which makes the score more discriminative than a single model. It is a zero-shot method with strong reported numbers, though it is calibrated to specific model pairs.

DetectLLM: complementary views

DetectLLM (Yang et al., 2024) combines a log-rank metric (where the chosen token ranks in the model's distribution) with an entropy metric, plus a context-aware comparison between the base text and a re-generated version. By running two complementary measures and combining them, it hedges against the failure modes of any single statistic.

How they stack up

Why run all of them

No single method wins everywhere. A passage can be confident under one metric and ambiguous under another, and the disagreements are often where the interesting cases live. Reporting a fused verdict with the per-detector breakdown — rather than one black-box percentage — lets you see why a text was scored the way it was, and to treat agreement across independent methods as stronger evidence than any single score.