AI Proves Code Safety at Scale

One hundred and eighty million lines. That is the staggering volume of mathematical proof code generated by Anthropic’s Claude model in just a few weeks. This massive output represents a significant shift in how software correctness is established, moving the needle from traditional testing to rigorous mathematical verification. The effort, conducted by Anthropic’s Frontier Red Team in collaboration with partner Theorem, marks a pivotal moment where artificial intelligence begins to handle the heavy lifting of proving software safety.

The core achievement lies in the ability to automatically verify existing software rather than just writing new code. For decades, formal verification has been prohibitively expensive, requiring specialized teams to spend years on single projects. The team allowed Claude to translate crash tests into mathematical problems, analyze compiled machine code, and produce extensive proofs. An independent program called Rocq then automatically checks these proofs, a process Anthropic describes as fully autonomous, despite the human-built foundations provided by Theorem.

This approach directly addresses the historical bottleneck of cost. To put the efficiency into perspective, a team of specialists previously required twenty person-years to formally verify the small operating system kernel seL4. Such a time commitment reserved formal verification for extreme cases, such as avionics software for Airbus. By applying AI to this task, the process has been compressed from years of human labor to mere weeks of computational effort, making a previously niche discipline accessible to a much wider range of software applications.

The practical impact was immediate and significant during the verification of 123 open-source libraries from Google’s OSS-Fuzz security project. While generating the proofs, Claude identified hundreds of bugs that conventional testing methods had missed. Many of these defects only manifested under very specific conditions, such as unusually large input files or particular sequences of data access. This suggests that AI verification can uncover latent issues that remain hidden in standard development cycles.

The discovery rate was particularly striking in certain areas. During an effort to verify a widely used PDF reader, the team found as many security bugs in one week as are typically reported for that software in an entire year. In another instance, Claude caught a logical error that had gone undetected for over a decade. These findings highlight the potential of formal verification to act as a deep-dive inspection tool, surfacing problems that have persisted for years without causing catastrophic failures.

However, the term formally verified carries specific technical weight and limitations. Anthropic acknowledges that these proofs do not guarantee the software is error-free in all possible scenarios. The mathematical correctness holds only under strict conditions, including a specific processor architecture, single-threaded execution, and input sizes under 64 MB. In real-world deployments, these libraries often run on diverse hardware and in complex configurations, meaning the verification covers a defined subset of the software’s potential behavior.

The computational cost of this process is also a factor to consider. Individual proofs can consume hundreds of millions of tokens, and while Anthropic has not released total project costs, the efficiency of newer models is improving. For example, the Opus 5.5 model solves half of the proofs using approximately 40 percent fewer tokens than its predecessor, Opus 5. This trend suggests that the economic barriers to formal verification may continue to decrease as models become more efficient.

The broader implication is a shift in the question facing developers. The industry is moving from asking if AI can program to asking if it can prove that its code is safe. If AI can write code and guarantee its correctness under defined conditions, the trust in AI-generated software will increase, potentially reducing the time spent on manual review. This could lead to significant cost savings across the entire coding and testing pipeline.

Looking ahead, Anthropic expects formal verification with AI to become as routine as standard testing. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL. As these tools mature, the integration of mathematical proof into the standard development workflow could redefine the baseline for software reliability, offering a new layer of assurance in an increasingly automated digital landscape.