Let AI look for flaws in its own system—what’s hardest isn’t finding issues, it’s that it’s not allowed to lie.
A large company disclosed an internal agent that its product security team built. From a pilot in November 2025 to becoming official this January, it has already found more than 500 vulnerabilities in the company’s own web applications. Its rule is simple: only when it can truly run a segment of exploit code and confirm the hole is really there will it be called in. 🔍
This constraint was forced by reality, and it’s also the most expensive part of the whole setup.
Over the past two years, the security team has received the most—generated reports that were fake. They look convincing line by line, but when you investigate, there’s nothing there. Distinguishing a real vulnerability from a fabricated yet plausible hallucination has become the core job of the field. When you ask a model to find vulnerabilities, it can always find one; the trouble is whether that flaw is real.
So the order is reversed. The model proposes hypotheses, which are handed to a dedicated verifier that tries to inject them into a live copy of the running application. If it can’t get in, it says nothing.
This system will then be paired with another agent that automatically fixes vulnerabilities.
One agent is responsible for proving the flaw is real, and the other is responsible for patching it. The most error-prone link in the middle is handed to two programs watching each other.
A large company disclosed an internal agent that its product security team built. From a pilot in November 2025 to becoming official this January, it has already found more than 500 vulnerabilities in the company’s own web applications. Its rule is simple: only when it can truly run a segment of exploit code and confirm the hole is really there will it be called in. 🔍
This constraint was forced by reality, and it’s also the most expensive part of the whole setup.
Over the past two years, the security team has received the most—generated reports that were fake. They look convincing line by line, but when you investigate, there’s nothing there. Distinguishing a real vulnerability from a fabricated yet plausible hallucination has become the core job of the field. When you ask a model to find vulnerabilities, it can always find one; the trouble is whether that flaw is real.
So the order is reversed. The model proposes hypotheses, which are handed to a dedicated verifier that tries to inject them into a live copy of the running application. If it can’t get in, it says nothing.
This system will then be paired with another agent that automatically fixes vulnerabilities.
One agent is responsible for proving the flaw is real, and the other is responsible for patching it. The most error-prone link in the middle is handed to two programs watching each other.