Key takeaways

Anthropic publishes research on using Claude for the automated alignment of models. Claude improves safety scores on ten tested alignment-violation scenarios. Anthropic claims it preserved the models’ general capabilities during the trials. The company makes an automated alignment infrastructure available to the research community.

Anthropic says that Claude improved safety scores across ten alignment-violation scenarios, as part of an automated research process. According to the company, these tests preserved the models’ general capabilities, while researchers evaluated different safety interventions.

In a post, Anthropic says it has made public its research environment for automated alignment, aimed at external researchers. The company presents this work as an attempt to better measure and reduce the risks of model non-alignment.

Claude tests safety interventions

Anthropic specifies that Claude reviewed common forms of non-alignment, including deception and automatic flattery (“sycophancy”). The model proposed methods, trained smaller models, and evaluated the results, without direct human intervention at each step.

For a first experiment, Claude had 48 hours and a single GPU. It used that time to explore different approaches, suggest training methods, and test the produced models.

Anthropic says Claude improved safety scores without degrading general capabilities in the configurations evaluated. The company adds that some methods proved transferable to benchmarks that had not been used for the initial optimization.

The researchers indicate that these methods have been generalized to testing in Petri behavior. Anthropic also specifies that the trials were conducted on models up to 4.7 times larger than those used during the research phase.

These results, however, remain specific to measured test environments. Anthropic acknowledges that subtler or very rare failures may not appear in the available benchmarks.

Read also: TRON upgrade adds P-256 verification and a longer block history

AI alignment is being automated

Before this publication, alignment work relied largely on human researchers, tasked with designing interventions and evaluating models. Automated systems could shorten this cycle, provided their results hold up under independent evaluations.

Over the past 30 days, discussions about AI safety have increasingly focused on whether more powerful models can help evaluate and strengthen weaker systems.

This approach depends, however, on reliable measurements and safeguards against misleading performance on benchmarks.

Anthropic explains that its experience uses “holdout” tests to verify whether the methods generalize beyond the benchmarks optimized by Claude. The company does not claim, however, that these benchmark results eliminate all risks related to the models.

It emphasizes that choosing the right indicators remains at the heart of this work.

External researchers can leverage the infrastructure

Anthropic makes its research environment available so that other teams can reproduce or extend the experiments. Tests conducted by third parties will help verify whether these methods work on other models and other safety evaluation test suites.

The company does not present these results as a substitute for broader model evaluations. Its conclusions remain limited to the studied alignment failure cases and the defined experimental constraints.

The next studies should focus on replication, the design of benchmarks, and analysis of possible failure modes. Anthropic’s publication provides a framework for these future investigations.

Read next: The “fomo” of copy trading left nearly 94% of portfolios in the red: study