Building the Preparedness Team: Safeguarding AI Advances and Mitigating Catastrophic Risks

The Importance of Preparedness in Advancing AI Technology | Company X

The Significance of Preparedness in AI Development

In order to address the potential risks associated with the continuous improvement of AI models, Company X has established a new team called Preparedness. This team, led by Aleksander Madry, is dedicated to closely integrating capability assessment, evaluations, and internal red teaming for cutting-edge models, ranging from near-future models to those with AGI-level capabilities. The primary objective of the Preparedness team is to effectively track, evaluate, forecast, and safeguard against catastrophic risks across various domains, including:

  • Individualized persuasion
  • Cybersecurity
  • Chemical, biological, radiological, and nuclear (CBRN) threats
  • Autonomous replication and adaptation (ARA)

Developing a Risk-Informed Development Policy (RDP)

The mission of the Preparedness team also entails the development and maintenance of a Risk-Informed Development Policy (RDP). This policy outlines our approach to conducting rigorous evaluations of the capabilities of frontier models, as well as monitoring their progress. Additionally, the RDP aims to establish a spectrum of protective measures and a governance structure to ensure accountability and oversight throughout the entire development process. It complements our current risk mitigation efforts, contributing to the safety and alignment of advanced systems before and after deployment.

Source link

Stay in the Loop

Get the daily email from AI Headliner that makes reading the news actually enjoyable. Join our mailing list to stay in the loop to stay informed, for free.

Latest stories

You might also like...