Commentary. The latest argument over artificial-intelligence safeguards is being framed as a choice between innovation and regulation. That is the wrong choice. The real question is whether the companies building increasingly autonomous systems should be trusted to decide, without enforceable consequences, when their products are safe enough for the public.
In Washington, President Donald Trump has promoted a voluntary AI safety agreement signed by major technology companies including Nvidia, SpaceX, OpenAI, Anthropic, Meta and Google. The pact calls for internal controls against unauthorized hacks and cooperation with independent auditors, but it does not specify penalties for companies that fail to comply.Reuters reported that the administration views the approach as a way to address public concern without imposing restrictions that could slow development.
That instinct is understandable. AI is developing quickly, and rigid rules written before the technology’s capabilities are clear could entrench large incumbents, burden smaller firms and push investment abroad. Companies also possess technical knowledge that governments often lack. A flexible code of conduct can be updated faster than legislation, while voluntary commitments may produce useful safeguards before lawmakers agree on a comprehensive framework.
The industry’s strongest argument is therefore not that safety is unimportant, but that the state may be too slow or blunt to manage it. Some technology leaders have warned that excessive regulation could limit beneficial applications in medicine, research and productivity. Peter Thiel, a prominent critic of AI regulation, has cast government oversight as a threat to individual freedom and technological progress.Fortune reported on his position.
But voluntary safeguards have a structural weakness: the companies asked to restrain themselves are also competing for market share, investment and influence. When the commercial reward for releasing a more capable system is immediate, while the consequences of a failure may be delayed, diffuse or borne by someone else, a promise is not an adequate substitute for accountability.
That concern is not theoretical. Former regulators and lawmakers have argued that existing consumer-protection laws are already being ignored or inadequately enforced as questions mount about autonomous AI agents and potential harm to users.Fox Baltimore reported that proposals in Congress include requirements for emergency shutdown mechanisms and a federal body to establish safety standards.
Europe’s experience offers a further warning against treating the current framework as finished. Microsoft and Google executives have questioned whether the European Union’s 2024 AI Act fits a technology now producing more autonomous systems, while analysts have identified gaps involving models tested internally rather than placed on the market.Politico reported on the pressure to revise the law.
Those gaps do not prove that every proposed rule is wise. Nor should regulators demand approval for ordinary software updates or impose one standard on a chatbot and a medical diagnostic system. A proportionate framework would distinguish between low-risk applications and frontier systems capable of taking actions, operating at scale or causing substantial physical, economic or social harm.
Our view is that governments should require documented risk assessments, independent testing, incident reporting and meaningful penalties for concealment or reckless deployment. Audits must be genuinely independent, not consultants selected and paid solely by the companies being evaluated. Regulators also need technical expertise and the authority to intervene before a dangerous system reaches millions of users—not merely after damage has occurred.
There is room for voluntary cooperation inside that system. Industry standards, shared testing protocols and early commitments can help officials move faster. But they should supplement enforceable rules, not replace them. Even Anthropic chief executive Dario Amodei has called for binding oversight modeled on regulators in aviation and pharmaceuticals, while arguing that AI development may need to slow enough for safety work to catch up.The Mercury News reported on the proposal.
The fairest policy will impose real obligations without freezing innovation: clear thresholds, transparent evidence, due process and periodic review. That is less dramatic than a promise of total freedom or a demand for an immediate moratorium. It is also more credible.
AI companies should help write practical rules. They should not be allowed to write the consequences for breaking them. Public trust will depend not on how reassuring a voluntary pact sounds, but on whether an independent authority can say no—and make that decision stick.