The California State Assembly has approved the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act (SB 1047).
The bill, which has sparked intense debate in Silicon Valley and beyond, aims to impose a series of safety measures on AI companies operating within California. These precautions must be implemented before training advanced foundation models.
Key requirements of the bill include:
- Implementing mechanisms for swift and complete model shutdown
- Safeguarding models against “unsafe post-training modifications”
- Establishing testing procedures to assess the potential risks of models or their derivatives causing “critical harm”
Senator Scott Wiener, the primary author of SB 1047, said: “We’ve worked hard all year, with open source advocates, Anthropic, and others, to refine and improve the bill. SB 1047 is well calibrated to what we know about foreseeable AI risks, and it deserves to be enacted.”
The senator emphasised that the bill simply asks large AI laboratories to follow through on their existing commitments to test their extensive models for catastrophic safety risks.
However, the proposed legislation has faced opposition from various quarters, including AI companies OpenAI and Anthropic, politicians Zoe Lofgren and Nancy Pelosi, and California’s Chamber of Commerce. Critics argue that the bill places excessive focus on catastrophic harms and could disproportionately affect small, open-source AI developers.
In response to these concerns, several amendments were made to the original bill. These changes include:
- Replacing potential criminal penalties with civil ones
- Limiting the enforcement powers granted to California’s attorney general
- Modifying requirements for joining the “Board of Frontier Models” created by the bill
The next step for SB 1047 is a vote in the State Senate, where it is expected to pass. Should this occur, the bill will then be presented to Governor Gavin Newsom, who will have until the end of September to make a decision on its enactment.
As one of the first significant AI regulations in the US, the passage of SB 1047 could set a precedent for future legislation. The outcome of this bill may have far-reaching implications for the AI industry, potentially influencing the development and deployment of advanced AI models not only in California but across the nation and beyond.
