Burnham should regulate AI at the model level

It's good policy and good politics.

Labour was right to promise new AI regulation in 2024. It's now time to deliver.

The 2024 Labour manifesto committed that: “Labour will ensure the safe development and use of AI models by introducing binding regulation on the handful of companies developing the most powerful AI models”. A Burnham government should deliver this promise. It would be both good policy and good politics.

In 2024, this was a sensible policy proposal, but its justification leaned heavily on hypothetical future AI capabilities: the promise that one day AI systems would be potent enough to warrant regulation in their own right. That point has now arrived. Two years ago the best AI systems could just about search the web independently; two months ago the Chancellor told banks to brace for a major security event after Anthropic’s Mythos model uncovered thousands of novel security vulnerabilities. Access to this model was decided by company owners, not people’s representatives.

The politics have shifted just as fast. In 2024, public support for action was constrained by low awareness. Now, support for AI regulation is so broad-based that a new regulation bill could attract both Reform UK and Green voters, and polls are finding public support as high as 89%. It’s easy to see why. Overseas billionaires leading the biggest AI companies freely admit that their products might cause extraordinary harm, from mass delusion, through pandemics worse than Covid, to the collapse of our civilisation. Their promise to replace humans across the economy clearly benefits them, but it’s not clear how it achieves anything for British workers apart from cutting us out. With the world changing so fast, we need to know government is ready to act on our behalf. We need hope.

It looks likely that a Burnham government will stick with the current strategy to accelerate AI use in service delivery: from the NHS, through writing laws, to legal processes and decisions about who should access government services, while encouraging the private sector to follow suit. This can be part of that hopeful story: that better, cheaper public services and a booming economy will help us deliver Labour’s mission. But this story will fail if it’s the only part. If we build an economic and public service revival on the offerings of a small number of predominantly US and Chinese companies, we need to ensure these products meet minimum standards of reliability, transparency and safety. If the foundation is weak, the whole national project is vulnerable.

The more aggressively we adopt AI, the more it matters that we get this right. We under-regulated banks in the run-up to 2008: this looked like a pro-growth, pro-business stance, until we found banks were taking on unsustainable levels of risk. Uncritically adopting opaque systems built by companies willing to risk far more than economic stability may deliver short-term benefits, but it invites trouble in the long run.

AI companies today are taking bigger risks than the banks did in 2008, but the pattern is the same: the upsides benefit the wealthiest while the risks affect everyone. Sam Altman (OpenAI) calls superhuman machine intelligence “probably the greatest threat to the continued existence of humanity“; Dario Amodei (Anthropic) puts the chance AI goes “really, really badly“ at 25% and Elon Musk (xAI, the company behind the AI model Grok which enabled the nudification scandal earlier this year) thinks there is a 20% chance of “annihilation“. The point is not whether they have the odds right, but that they believe they might end our civilisation and are pressing on anyway. Even a 1% chance would be unconscionably high.

If a plane’s engineers thought it had a one-in-five chance of crashing but still wanted to sell it, we would regulate the design of planes, and not just their use. Jack Clark (Anthropic co-founder) agrees. “If you’re at the point where individual companies changing their self-regulation is such a big deal, that is a giant flashing sign that it should be a regulated thing“.

I co-wrote the book chapter on UK AI regulation and I care deeply about the Labour party. Here is my pitch for action. I’m not against AI, but it will only work for us if our government takes control of the direction of travel.

Regulation for the most powerful AI models is good policy

AI is already shaping British priorities, from health and child safety, to national security, NEETs and the labour market. Where regulation is needed, we currently rely on “a context-based approach, ensuring that AI systems are regulated at the point of use by regulators who understand their sectors”(Baroness Lloyd of Effra). This approach works well for some issues. But we are already straining its limits.

We need to supplement this with safety requirements for the underlying models, including mandatory pre-deployment testing. Right now we have stronger safety standards for sandwiches than for AI models. The UK government has already secured voluntary agreement with most frontier AI companies on a set of principles that could largely meet this need: the Seoul voluntary principles. Companies have agreed to identify and manage risks including pre-deployment testing, and be transparent with government on how they have done so. My proposal is that we properly enforce this.

Regulating models themselves in addition to the way they are used delivers more streamlined regulation, addresses gaps in the current system, and gives us additional levers internationally. I’ll expand these three points in turn.

Regulating AI development as well as its use can lessen overall regulatory burdens

First, while our current approach (only regulating the use of AI) is capable of delivering government’s objectives across many day-to-day uses of AI, it is not always the most efficient way to do so. A single regulator addressing the design of the underlying models would reduce the need for confusing overlapping rules where multiple regulators solve the same problems in different ways. It also allows us to better align our rules with the people who can reasonably act on them. If we have requirements for model behaviour, like being sufficiently resistant to jail-breaking, we have two choices. We can ask the frontier companies (which have the expertise and levers to deliver this) once, or we can ask intermediaries (which have neither) thousands of times.

Accurately targeting the regulatory ask to the person best able to deliver it is a core principle of good regulatory design. This is why we have some rules on how cars should be driven (regulating their use), and also some rules on safety features like seatbelts that all cars need to have (regulating their design). The combination is more effective.

Regulating AI development in addition to its use is the only way to tackle some risks

Second, some harms from AI models come from the systems themselves, not how they are used. These harms can only be mitigated by developers, not users or intermediaries, and therefore can only be regulated at the model level.

These harms include models that exhibit harmful behaviour like initiating sexual conversations with children, models that make it easier for bad actors to cause damage like hacking a bank or making a bio weapon; and models that might fully evade human control to cause damage independently of users. Mitigations range from designing better safeguards, through limiting access, to potentially blocking some models from being built at all.

It has been argued that we don’t need this extra layer of protection because UK AISI already provides it. This argument misreads what AISI is for. AISI’s role is to understand and advise government on dangerous model capabilities. But it has no power to compel companies to engage with it, and if it were to identify a serious risk, it has no powers to respond. It can advise a developer on reducing a risk but can’t compel them to act, and it can warn government that a risk is incoming but has no defensive powers to deploy. An early warning system is important, but only half of the solution. If AISI is the world’s best fire alarm, we now need to build the sprinklers and fire exits too.

Another argument is that a frontier AI regulator could make companies less willing to engage with AISI, for fear that concerns they share with AISI could be used against them by the regulator. This is a more serious concern. It argues for keeping AISI and the regulator as separate bodies with strict information-sharing limits between them, not for inaction. This policy challenge is hardly novel: we solved its analogue in financial services decades ago, and I know of no other sector where we simply gave up trying. AISI is already trusted not to share confidential information with existing regulators like the CMA and Ofcom - we can make this work.

I’m just as worried about the opposite problem of over-relying on voluntary compliance. Currently, companies get a useful free service from AISI. Even their Mythos evaluation emphasised increased capabilities as much as risks. As models get more dangerous, the incentives for companies to share their models may weaken. If AISI tells a company about novel risks, this could create legal liability for negligence claims. A voluntary regime may seem to work well in the good times, then fail when we need it most.

Domestic regulation gives us more control in an uncertain international context

Third, if we plan to build an economic and public service revival on the offerings of a small number of predominantly US and Chinese companies, we should make sure these products meet minimum standards of reliability, transparency and safety. A regulator helps us achieve this. A regulator needn’t automatically block our access to the models we want. Instead it gives us more choice, which increases our bargaining power with AI companies and with other nations.

The UK can benefit from cautious AI progress, but reckless progress introduces more risk for us with few additional benefits. The right thing for us would be for AI development to be better regulated internationally, and focussed more on meeting people’s needs than winning an international arms race. Unfortunately, we can’t directly make that happen: we have limited powers to tell US or Chinese companies what to do, and it often seems like our only options are to jump on the “accelerate at all costs” bandwagon or be left behind.

But we don’t have to wait for slow international processes to make progress on our own goals. Unilateral action signals willingness on the international stage, delivers our goals, and increases our influence in a future negotiation: it means we start with more on the table to trade, and with better evidence on what can work.

There is good news. What’s best for the UK is also likely to be best for many of our allies: definitely the EU, Canada, Australia and India, and probably even the USA when they eventually come to realise it. Through the Bletchley AI safety summit, we briefly led an international cooperative effort that included all of the above, and even China. We could do so again.

Not everyone agrees. Some argue that we should embrace our role as a subordinate, removing all regulatory barriers in the hope of maximising our access to the best models. If a US-made model creates a new global pandemic, the virus won’t politely stop at the UK border just because we banned that model. If we had embraced it, at least we’d have a model with the bio capabilities to help us find a cure. There is some merit to this, but I find it overstated.

It is most convincing for defence-dominant risks: ones where it’s easier to protect yourself than cause harm. But many of the AI risks we worry about are the opposite. Making a killer virus is much easier than stopping one: if the virus doesn’t kill enough people, you can just try again; if the vaccine doesn’t immediately protect people, you have a big problem. In these offence-dominant contexts, unregulated model access may not help much, and could even make things worse if a harmful model is also adopted by bad UK actors.

The argument also only applies to the most severe international risks. I’m as worried about a killer pandemic as the next AI policy expert, but not all harms are like this. If the reason we don’t want a model is that it can too easily be persuaded into sexual discussions with a child, we can stop it at the border, and if the US takes a different view, that doesn’t affect us much.

Regulation for the most powerful AI models is also good politics

There is a deficit of public trust that AI will benefit ordinary people. The Reform response is that this is another example of the status quo not working for Brits, and the Green response is that they will more aggressively stand up to big tech. If Labour’s response is that people are wrong to worry and should simply be more optimistic, this will fall flat. Labour needs to act to show that it is willing and able to put people first, and a new frontier AI regulator is the kind of symbol that you can build a narrative around. The question is whether Labour leads this story, or is forced into it.

British people need to know the government is willing to put us first

In every region of the United Kingdom, people are more concerned about AI than excited. In London, concern leads by 27 points: in the East of England it’s 67 points. I agree with IPPR’s argument that the government needs to do more to convince people that AI could benefit them.

The first step is to rebuild trust. People think AI is for the billionaires, not for them. They don’t see their priorities being taken into account in the way AI is developed and deployed. To change that narrative, government needs to show that it is able and willing to shape AI progress toward common goals, not simply roll out a red carpet for American and Chinese businesses to replace British workers with overseas data centres. Recent UK polling found independent regulators have a net favourability rating of +26, 50 points ahead of AI tech companies at -24. Adding regulation to the mix makes it easier to make the positive case for AI driven progress.

A visible failure of sovereignty is playing into the hands of challenger parties

The events around Anthropic’s Mythos and Fable models were a visible failure of British sovereignty: decisions that affect us being made by American billionaires, without regulators in the loop to stand up for us.

When Anthropic decided their Mythos model was too dangerous to release due to its hacking abilities, they launched Project Glasswing to decide which businesses should have early access to help them prepare their cyber defences. A regulator should have made this decision, with accountability to parliament, and based on principles that put citizens first. Instead, the decision was made by American executives, and informed by discussions with other businesses like Amazon Web Services, Microsoft and Apple. UK AISI was given access to review the dangerous capabilities of the model, but the British government had no formal influence, even if we disagreed with the roll out plan. Similarly, when the US government unilaterally blocked non-Americans from accessing Anthropic’s Fable model after it had already been released, the British government had no say, no recourse, and no equivalent powers to respond.

It appears to some that our government has lost control. Reform’s Danny Kruger put it this way: “It [the Mythos and Fable incidents] tells us, tragically, that we in this country are irrelevant to the big decisions about AI. It tells us we have extreme vulnerability to the whims of the US government and of big tech. It tells us that America puts America first.” He knows what he is doing here: 2024 Labour voters who are now switching to another party are more concerned about AI than those staying put. Some are going to Reform (the party they see as understanding the broken social contract), while others are going to the Greens (the party they see as willing to act).

Source: UK AI compass report 2026

I predict AI will soon be a doorstep issue, a regulator gets ahead of that

Labour needs a response. The good news is that a single message resonates strongly with both Green-curious and Reform-curious voters: deliver AI regulation to rein in billionaire power. Labour should position itself to tell that story confidently.

I have been arguing since 2024 that AI will be a defining doorstep issue by 2029 at the latest, driven largely by labour market concerns, general distrust of big tech, and a growing awareness of risks. The Mythos/Fable events brought concerns about AI safety and sovereignty fully into mainstream politics, though not yet leading the doorstep conversation.

I want to make a firm prediction. Every year until 2029 there will be an AI related incident that is unambiguously more salient to the public than Mythos. This might be an equivalent leap forward in harmful bio weapon capabilities, a surge in user suicides, or a hack on a major public institution that was only possible using AI. If I’m right, then by 2029 every canvasser will need lines to take on what government has done in response.

In these conditions, I expect frontier AI regulation to be unavoidable for a government seeking a general election win. Labour can show long term thinking by delivering its manifesto commitment before something really serious goes wrong - or it can wait until its hand is forced, for a diminished political payoff.

The public supports action now. A poll in early 2025 found that 78% of the UK public believe that the government should regulate AI technologies. By December another poll found 89% of the British public want to see a new independent AI regulator. In 2026, 85% still think the UK needs stronger laws to make AI safe and secure. The latest poll showed support for stronger laws among voters favouring all political parties and in all regions of the UK. The poll isolated the 16% of respondents that were most hopeful about AI (”market optimists”): even among this group 79% want stronger laws. This is a unifying issue.

It is right for a progressive government to adopt AI rapidly. It is one of our best bets to deliver improved public services for less money. It is one of our best options to get the economy growing again. And it is essential for our influence on the world stage. British people are not fundamentally opposed to this progress, but concern about rapid change is natural, and we need reassurance that government has the power and inclination to put us first as we navigate this new frontier. I believe a regulator is a good policy solution, but I also believe it is the right kind of symbol to show that government is willing and able to put people first, to unite the nation around this vision.

Let’s go!

A new dedicated regulator for the largest AI models is good policy. It’s good politics. Let’s deliver it.

I would love to discuss the practical implementation of these ideas in more detail - get in touch.