A company called Pangram sells AI detection tools to universities and platforms like Substack, claiming they can reliably identify AI-generated text. The founder Max Spero has been using the tool to publicly call out writers.
The technical irony: When their own website content and Substack posts were run through competing AI detectors (ZeroGPT, GPTZero), they flagged as substantially AI-generated. Multiple test runs (60+) showed results ranging from 100% AI to 100% human, suggesting their content was likely processed through AI humanizer tools.
The technical problem with AI detectors:
- They're fundamentally flawed at the classification level
- Historic documents not in training data got flagged as 100% AI
- The detection methodology relies on pattern matching against LLM output characteristics, but this creates massive false positive rates
- Authors are now dumbing down their writing to avoid false flags
Pangram's architecture (from their own description): A neural network classifier that maps writing styles in latent space, clustering human vs LLM text. They claim different LLMs (ChatGPT, Claude) occupy distinct regions.
The real issue: These detectors are trained on the same scraped human text that LLMs use, creating a circular logic problem. You're essentially penalizing coherent, well-structured writing because that's what the models learned from quality human text.
Bottom line: AI text detection at scale is a fundamentally broken problem. The statistical overlap between good human writing and LLM output is too high for reliable classification.
The technical irony: When their own website content and Substack posts were run through competing AI detectors (ZeroGPT, GPTZero), they flagged as substantially AI-generated. Multiple test runs (60+) showed results ranging from 100% AI to 100% human, suggesting their content was likely processed through AI humanizer tools.
The technical problem with AI detectors:
- They're fundamentally flawed at the classification level
- Historic documents not in training data got flagged as 100% AI
- The detection methodology relies on pattern matching against LLM output characteristics, but this creates massive false positive rates
- Authors are now dumbing down their writing to avoid false flags
Pangram's architecture (from their own description): A neural network classifier that maps writing styles in latent space, clustering human vs LLM text. They claim different LLMs (ChatGPT, Claude) occupy distinct regions.
The real issue: These detectors are trained on the same scraped human text that LLMs use, creating a circular logic problem. You're essentially penalizing coherent, well-structured writing because that's what the models learned from quality human text.
Bottom line: AI text detection at scale is a fundamentally broken problem. The statistical overlap between good human writing and LLM output is too high for reliable classification.
