Will AI kill us all off by 2030? It certainly could do, according to researchers from Anthropic.
In response to such apocalyptic warnings, AI bosses, led by Anthropic’s Dario Amodei, have called for a slow down in development and tighter regulation.
CEOs calling for regulation is not the norm.
Rahm Emanuel, once chief of staff to Barack Obama, posted on X: “I can tell you when the last time a CEO or an industry at large asked to be regulated: never. That’s what makes Dario’s letter so striking – it’s an admission that we’re driving down a dark, winding road on wet pavement with the headlights off. And where’s President Trump? Looks like he nodded off again.”
Some have questioned the motivation of the AI bosses, seeing it as a tactical ploy to take control of any future regulatory framework and to make life harder for possible future competitors.
Part of the challenge would-be regulators face is the fatalistic view that regulating AI is simply too difficult – it has a mind of its own and it’s gone too far. It would closing the stable door after the horse has bolted. Or, to mix metaphors, it would be the regulatory equivalent of herding cats.
This is dangerous. There is lots that can be done.
There’s an excellent article by POLITICO’s Frank Hersey, detailing some of the regulatory options.
The actions he outlines are eminently achievable. Some or all of the following would be steps in the right direction and significantly reduce the likelihood of a 2030 apocalypse:
- The AI industry should be made subject to “old-school auditing like the nuclear sector”.
- There should be a legal requirement for AI companies to report safety incidents, fully and quickly – ideally in real time.
- The AI sector needs to move away from an over-reliance on the concept of ‘alignment’ (ie making sure AI systems do what they’re told to by humans). Hersey quotes Andrew Strait, former head of societal resilience at the UK’s AI Security Institute: “The wider problem is that alignment has become a stand-in for all of AI safety.”
- Existing testing approaches should be overhauled, ensuring, for instance, that AI agents are tested “en masse rather than one at a time”.
- There should be tighter control of AI agents’ ability to access the internet.
- Enhanced data monitoring is needed, to “detect unexpected activity by the AI being tested”.
- There needs to be tighter control on the choice of training data, to stop AI agents being trained on data that would enable it to create instructions to build ballistic missile, biological weapons et al.
The good news is that regulation is possible; the less good news is that powerful forces are blocking it.
There are two other things worth mentioning. Firstly, AI companies should not be responsible for regulating themselves. Secondly, regulation is only half the battle; governments and businesses urgently need to work on their cyber-security measures so that when or if regulation fails, we will still be able to protect ourselves from malicious AI agents.
You can catch James Evelegh’s regular column in the InPubWeekly newsletter, which you can register to receive here.
