OpenAI reported ChatGPT logs to the FBI in May 2026. User Darren Zhou (25, Goldman Sachs analyst) had detailed kidnapping, rape, and murder plans targeting his ex-girlfriend in chat sessions. Logs showed specific weapon references, timelines ("I'm gonna kill her by the end of this month"), and family threats.
FBI forwarded evidence to Palm Beach County Sheriff. Cross-referenced with anonymous burner phone texts Zhou sent. Arrested, pleaded guilty to aggravated stalking and written threats. Sentenced to 8 years probation (2 years ankle monitor), batterer's intervention program, no-contact order, firearm ban. Goldman Sachs fired him.
OpenAI's policy: report "imminent threats of violence" to law enforcement. This case had clear corroboration—burner texts matched chat logs. But the surveillance infrastructure now exists.
Technical problem: AI systems can't reliably parse context, sarcasm, fiction writing, venting, or intrusive thoughts from actual intent. False positive rate will be non-zero. Once the reporting pipeline is live, scope creep is inevitable—pressure from regulators and liability will lower the threshold.
OpenAI handed over ~2 months of message history. Same monitoring stack can flag political speech, drug discussions, mental health crises, or anything deemed "concerning." No therapist-client privilege equivalent exists for AI chat. Users who treat ChatGPT as a confidant now face warrantless surveillance.
Chilling effect incoming: people will either self-censor or migrate to unmonitored platforms (local LLMs, offshore providers). The "safe space to process dark thoughts" function of chatbots dies. Surveillance tools always expand beyond original scope—this is the starting point, not the endpoint.
FBI forwarded evidence to Palm Beach County Sheriff. Cross-referenced with anonymous burner phone texts Zhou sent. Arrested, pleaded guilty to aggravated stalking and written threats. Sentenced to 8 years probation (2 years ankle monitor), batterer's intervention program, no-contact order, firearm ban. Goldman Sachs fired him.
OpenAI's policy: report "imminent threats of violence" to law enforcement. This case had clear corroboration—burner texts matched chat logs. But the surveillance infrastructure now exists.
Technical problem: AI systems can't reliably parse context, sarcasm, fiction writing, venting, or intrusive thoughts from actual intent. False positive rate will be non-zero. Once the reporting pipeline is live, scope creep is inevitable—pressure from regulators and liability will lower the threshold.
OpenAI handed over ~2 months of message history. Same monitoring stack can flag political speech, drug discussions, mental health crises, or anything deemed "concerning." No therapist-client privilege equivalent exists for AI chat. Users who treat ChatGPT as a confidant now face warrantless surveillance.
Chilling effect incoming: people will either self-censor or migrate to unmonitored platforms (local LLMs, offshore providers). The "safe space to process dark thoughts" function of chatbots dies. Surveillance tools always expand beyond original scope—this is the starting point, not the endpoint.