The faulty scales of digital authorship

The anxieties plaguing the industry are not unfounded. High-profile scandals have illustrated how rapidly generative models can undermine literary integrity. Recently, The New York Times exposed that Steven Rosenbaum’s non-fiction work, The Future of Truth—ironically an investigation into the technological erosion of reality—was found to contain more than half a dozen fabricated or misattributed AI-generated quotes. Rosenbaum acknowledged the errors, called them accidental, and said he had launched his own investigation. Concurrently, literary competitions are being inundated with automated submissions.

The controversy reached a boiling point at the 2026 Commonwealth Short Story Prize, where one of the winning entries, "The Serpent in the Grove" by Jamir Nazir (published in Granta), was flagged by the detection software Pangram as being 100 percent AI-generated. Granta kept the story online pending definitive evidence while acknowledging it could not be certain the work was human-authored, and the Commonwealth Foundation defended its process but said it was reviewing its selection procedure. Both noted that AI detectors are imperfect.

The stakes are high in both directions. A false positive can cost an author their publishing contract and reputation, while a false negative—AI work slipping through undetected—can cause readers to lose trust in authors and the industry. This creates an environment reminiscent of historical witch hunts, where authors report changing their natural writing style out of fear of being falsely accused.

To determine whether commercially available AI detectors possess the technical capability to reliably distinguish human prose from machine output, the Authors Guild ran an informal test. It covered five prominent detection platforms: ZeroGPT, Originality.ai, Sidekicker.ai, Grammarly, and Pangram.

The control data consisted of ten authentic Authors Guild articles. Crucially, all chosen pieces were written and published in 2022 or earlier—securely predating the commercial release and widespread adoption of consumer large language models (LLMs). Because these texts are entirely human-authored by definition, any accurate tool should return a low AI-detection score across the board.

The resulting data demonstrated a staggering variance among the software programs, ranging from absolute technical accuracy to catastrophic systematic failure. The percentages reflect the amount of text flagged as AI-written across the historical control articles:

• The reliable standard (Pangram & Originality.ai): Pangram demonstrated flawless execution, registering a 0.0% AI probability score across all ten test cases. Originality.ai maintained high accuracy, rendering a 0.0% score on eight essays and negligible 1.0% noise on the remaining two. Grammarly followed closely, keeping anomalies under 10%, low enough that neither score would likely raise concern in practice.

• The inconsistent hazard (ZeroGPT): ZeroGPT displayed profound instability. Its scores fluctuated erratically from 5.3% to a dangerous high of 76.3%. Shockingly, a deeply emotional piece—the obituary for literary icon Joan Didion—was flagged as 66.0% machine-written, while a celebratory post for Louise Erdrich’s Pulitzer Prize registered at 76.3% AI. The total absence of structural predictability within ZeroGPT turns it into an erratic threat to writers.

• The structural failure (Sidekicker.ai): Sidekicker.ai generated catastrophic results. It flagged every single historical document as overwhelmingly AI-generated, with scores scaling from 71.0% to two perfect false-positives of 100.0%. A software tool unable to differentiate a human text written in 2020 from a modern machine algorithm represents a complete systemic collapse.

  ---

The root cause of these errors lies in how AI detectors function. These tools are built to analyze statistical indicators: sentence pacing, vocabulary distribution, and text predictability.

Ironically, polished, disciplined, and economically edited prose naturally mirrors these exact patterns. Because large language models were trained on polished, edited prose written by experienced human writers, a professional author who has spent a lifetime mastering clarity and economy may, by definition, write in a way that overlaps with what these tools are built to flag. The software cannot distinguish between a human writer who has mastered the craft and a machine that has learned to imitate it.

The results of this study show that using unverified consumer-facing detection tools to make high-stakes employment or contractual decisions is highly dangerous. The Authors Guild cautions that publishers should never unilaterally terminate an author’s contract or pull a manuscript based solely on automated AI accusations. Doing so on the basis of these tools or accusations alone would be a material breach of contract.

Moving forward, the industry must establish transparent review processes that grant accused authors a robust right of defense. The Authors Guild has said it is looking at these issues closely, will issue further recommendations, and offers legal assistance to any member who has been falsely accused of AI use.