End-to-End Security Research and Automated Cryptographic Analysis

Anthropic

Initial estimates for the timeline of automated coding research were approximately $2$ to $5$ years in the best-case scenario and more than $5$ years on average.
End-to-end research timelines were previously estimated to be more than $10$ years.
Observations indicate that while coding estimates were somewhat accurate, the estimates for automated research were significantly underestimated, as capabilities are developing much faster than anticipated.

End-to-end research refers to the use of Large Language Models (LLMs) to manage the entire scientific process rather than using them simply as tools or agents for isolated tasks.
The full research cycle encompasses:
- Generating original ideas.
- Formulating hypotheses.
- Developing a specific methodology for testing.
- Executing the implementation.
- Conducting observations and analysis.
This approach goes beyond the standard practices allowed in conservative academic environments, including security conferences, where LLMs are typically restricted to auxiliary roles.

Recent research utilizing the Claude model demonstrated the ability to perform complex cryptographic analysis and identify flaws in algorithms.
The study successfully identified a new attack on the Advanced Encryption Standard (AES), a symmetric cryptographic algorithm.
The AES Algorithm Background:
- Standardized in $2001$ by the National Institute of Standards and Technology (NIST).
- It is a foundational algorithm used globally for internet security.
- The standard version of AES is an iterative algorithm consisting of $10$ rounds.
- Each round involves four primary operations: substitution (S-boxes), permutation, bit shuffling, and the addition of a key function.
- Substitution boxes (S-boxes) function by mapping specific inputs to a different set of outputs.

The research focused on a reduced-round version of AES, specifically a version limited to $7$ rounds, which is a standard target for academic cryptographical studies.
Complexity metrics for attacking $7$ -round AES:
- Brute-force complexity: $2^{128}$ operations. This remains computationally impossible within the lifespan of the sun.
- Previous best-known academic attack (established in $2013$ ): approximately $2^{99}$ operations, estimated to take hundreds of thousands of years to complete.
- New automated attack generated by Claude: approximately $2^{89}$ operations.
The automated attack is empirically evaluated to be between $200$ and $800$ times faster than the previous state-of-the-art methodology.
Specific Mechanics of the Attack:
- The attack identifies relationships between inputs and outputs without full emulation of the algorithm.
- It exploits the specific S-box formula involving inversion in a field, followed by scaling and shifting.
- The model recognized that the ratio between shapes is preserved during scaling and shifting, using this property to counteract the encryption.

Human Methodology:
- Generally involves isolated thinking for several weeks.
- Includes iterative discussions with peers.
- Typically results in a small handful of ideas to be submitted for testing.
LLM Methodology:
- The model generates a massive volume of ideas simultaneously.
- Instead of discussing ideas, it attempts to implement all of them to empirically verify which ones are viable.
- This shifts the burden of research from execution (now handled by the model) to the conceptual quality of the ideas.
The Research Harness:
- A specialized framework or "harness" is currently necessary to provide a controlled environment for the models.
- The harness facilitates communication between different agents and prevents the model from attempting to implement logically impossible tasks.
- The model iterates on ideas by discussing them with other delegated agents before proceeding to implementation.

Total ideas generated by the model: more than $3,000$ .
Applicable ideas: more than $2,000$ were found to be relevant to the specific problem.
Novel ideas: approximately $200$ ideas were identified as unique and previously unknown.
Final Publication Grade: Only $4$ of the original $3,000$ ideas were of sufficient quality and novelty to be included in the final academic paper.

The automated research capability extends beyond AES to other cryptographic systems:
- Hawk: A post-quantum cryptographic algorithm.
- Via: A cryptographic algorithm utilized in specific regional contexts.
Verification and Peer Review:
- The findings were presented to professional cryptographers who confirmed the interest and validity of the results.
- Researchers involved include Nicholas Carlini (specializing in system and machine learning security) and specialists in network security.
- The research is expanding into broader security domains, including general network security and privacy research.
While these attacks are academic achievements and have not necessitated immediate changes to internet infrastructure, they prove that LLMs can achieve high-level, novel research results previously thought to be exclusive to human experts.