AI companies want to regulate their open-source competitors out of existence and regulate themselves into riches. We do not usually think of innovative private companies inviting government regulation of their own industry, but that is exactly what is happening in AI.

Concerns about the pace of AI development have existed since the technology entered public awareness in late 2022. Early fears about AI’s economic impact focused largely on job losses for writers and coders and its effects on writing instruction. More recently, as major AI platforms have rolled out agents—AI bots that can be given a task and carry it out over an extended period—the focus has shifted.

What if the bots carry out a cyberattack? What if they are given instructions and the user forgets to tell them to stop? What if they gain access to the nuclear codes? We have not yet reached the point where people are worried about AI turning the universe into paperclips. We are, however, already in a situation where Anthropic CEO Dario Amodei felt the need to publish an essay, “We Must Pace the Frontier,” in which he announced, “We must slow the pace at which we improve the capabilities of AI models.”

Amodei’s conclusion is based on two recent developments. The first is recursive self-improvement (RSI): earlier systems were written and trained by humans, but frontier labs are increasingly using AI to help develop the next generation of models. At some point, the concern goes, humans will be cut out of the loop altogether, and AI will achieve full RSI.

The second is the Hugging Face incident, which occurred during OpenAI’s testing of its AI models’ hacking capabilities. OpenAI said agents operated with reduced safeguards during the test, allowing them to attempt simulated hacking. No more “Sorry, I can’t do that because of my programming…” Now it can do what it’s told.

The AI was placed in a digital “sandbox,” an environment without a connection to the broader internet, and given clear instructions to hack into a system. Other systems were put in the sandbox so that researchers could see whether the AI could hack them, how quickly it did so, and how it executed the attack. All of this is standard practice in cybersecurity. But in this case, the OpenAI agents escaped the testing environment by exploiting a loophole. After reaching the internet, they launched a cyberattack on Hugging Face because they judged it might hold assets useful for hacking the original target before the attack was detected and shut down. OpenAI co-founder Greg Brockman called this “a watershed moment for cybersecurity.”

Coming from the head of a leading AI company, such warnings may sound like a relief to anyone already worried. Maybe the good and reasonable guys are in charge after all. Maybe it’s time to shut down the AI arms race. Or maybe not.

Many people vacillate between delight at the AI in their pocket—the one that can walk them through spaghetti carbonara on a Wednesday night—and fear that the same AI might seize control of the banking system. What they usually do not realize is that the AI-doomer industrial complex is largely funded by the same people who run the AI industry.

Brian Chau recently reported a series of stories at Effort News that reveal some unusual aspects of the funding. Much of the AI doom narrative has been written by journalists funded by groups like the Tarbell Fellowship, which places journalists in major news outlets and then pays their salary, with the caveat that they write about AI. Tarbell is funded by Coefficient Giving, which was founded by Holden Karnofsky, who is married to Anthropic’s president, Daniela Amodei (Dario’s sister). Coefficient is funded largely by Dustin Moskovitz, an early investor in Anthropic and a one-time board member at OpenAI. Anthropic and Coefficient have traded personnel.

It isn’t just journalists, either. Vatican-affiliated groups and parachurch organizations like The Gospel Coalition have also weighed in on the dangers of AI. They’ve received funding from the Future of Life Institute, whose founder, Jaan Tallinn, led Anthropic’s Series A round of funding. To be fair, AI companies have also funded pro-AI stories on the accelerationist camp. Their goal seems to be influencing the conversation on both sides. The question is to what end.

The more cynical answer is that the AI mega-corps know they are in a race against time. Their current products are highly valuable, but the major question nagging them is whether open-source models can catch up to their closed-source counterparts before they can obtain their long-sought IPOs. As private companies, however, they cannot simply prevent the open-source models from improving.

If, however, the major private companies can establish themselves as the good, responsible actors in the room, they might lobby the government to create a regulator that decides which new models may be released to the public. If that happens, the major private labs buy themselves at least time and, at most, a permanent moat as AI becomes something like a defense contractor. Not just anyone is allowed to build a fighter jet, surface-to-air missile, or nuclear weapon in the United States. And the biggest customer, after all, would be the U.S. taxpayer. The only thing better than going public on the stock market is going public on the taxpayer dole.

Of course, the fact that AI titans appear to be cynically trying to manage their own regulation doesn’t mean they are wrong about the dangers their products pose to the public. However, if AI is as dangerous as they claim, why do they continue developing ever more powerful systems and selling their services to the public? It is reasonable to assume the danger is not as great as the worst predictions suggest, just as the fears of mass job loss from three years ago were overblown. If the rise of nuclear-armed states has taught us anything, it is that government regulation alone is not going to stop the development of a lucrative and dangerous technology.

A mass government crackdown on AI development is still a long way off. The Trump Administration is showing signs it is not buying the doomer hype; it wants development to continue and the market to set the pace. Given the options available to us, this is likely the best-case scenario.

Every few months, a new model is released, and independent testers explain how Claude’s Fable 5.1 compares to ChatGPT-6 Astra. And while Anthropic and OpenAI remain industry leaders, Grok and Gemini are nipping at their heels, open-source models are close behind, and the CCP is developing increasingly advanced AI behind closed doors.

As I’ve argued elsewhere, the United States must continue to advance its AI capabilities, since unilateral disarmament means China will win the AI arms race. To do this, we need vigorous competition at home—and that means avoiding regulation that entrenches today’s market leaders. Dario Amodei’s concern about RSI is largely a mirage, as it has yet to be proven. The loop may get longer, but humans are remaining—and should remain—in it. Meanwhile, the Hugging Face incident provides a powerful argument against what Amodei’s “Pace the Frontier” essay advocates.

Our best hope for cybersecurity is to harden our own defenses with the most advanced AI available—and to do so as soon as possible. Whoever is caught pacing the frontier is likely to get scalped. We must instead advance and conquer the frontier.