Mathematicians are questioning the reliability of OpenAI’s 719 AI-generated mathematical manuscripts after an article claimed that the Navier–Stokes proof published by the company does not match its computer-verified version.
Key points:
OpenAI’s catalog has shrunk from 722 to 719 manuscripts after three papers were withdrawn due to a sign error.
Three UK-based mathematicians have identified two discrepancies between the written Navier–Stokes proof and its Lean code.
The authors do not judge the proof to be false, but note that automated verification does not replace peer review.
OpenAI publishes its mathematics results
On October 6, the company posted a collection of manuscripts, and a report published Thursday showed that it had deviated from guidelines drawn up by an independent panel of mathematicians for AI labs. The catalog initially contained 722 manuscripts. OpenAI withdrew three the following day after an error in the sign was found that invalidated an argument, along with two papers that depended on it.
Published on September 29 by the nine-member Advisory Group on Mathematics and Artificial Intelligence, the recommendations begin by asking labs to stop testing advanced mathematics problems on proprietary models. According to OpenAI’s repository, the papers came from evaluating an unpublished internal model on open research problems.
Summaries of the model’s reasoning accompany only 10 results, and about 42% of the main claims have been formalized in Lean, a programming language that allows a computer to verify a proof.
Read also: How much does Anthropic’s CEO earn? The IPO filing has the answer
Flaws in the Navier–Stokes proof
Alexander Bastounis of King’s College London, along with Fabian Circelli and Anders Hansen of the University of Cambridge, examined the Navier–Stokes proof OpenAI announced in September. They identify two discrepancies between the written argument and the Lean code, including one case where the code proves only a weaker estimate than the one claimed in the paper.
The authors clarify that they are not taking a position on the validity of the written proof itself. Their criticism is more focused: Lean confirms that the final theorem has indeed been proved, they write, but it does not guarantee the soundness of the intermediate steps read by a human. Such proofs therefore still depend on peer review.
Fields Medalist Terence Tao wrote on October 6 that some problems are now being “solved” by “AI prompters” who understand the model’s outputs too poorly to answer questions or present the results. Harvard professor Melanie Matchett Wood, a member of the advisory group, notes that no one fully understands these results when they are published, “and that’s where the real work begins.”
Controversy over OpenAI’s proofs
The advisory group stresses that only the broader mathematical community can judge how well OpenAI has followed its recommendations. The review comes at the end of a tense month. OpenAI announced its Navier–Stokes result on September 8, sparking a dispute over attribution with New York University mathematician Tristan Buckmaster. Three days later, 25 Fields Medalists published a statement warning that a flood of AI-generated results could harm the discipline.
Coming up: OpenAI’s 722 AI mathematics papers are public; now it’s up to the community to evaluate them