✪ Artificial Intelligence Capitalists Searching For Government Cronies

rrrr

I companies are trying to enlist the government to regulate their open-source competitors out of existence and regulate themselves into riches. We do not usually think of innovative private companies inviting government regulation of their own industry, but that is exactly what is happening now in AI.

Concerns about the pace of AI development have existed since the technology first entered the public awareness in late 2022. Early fears about AI’s economic impact focused largely on job losses for writers and coders and its effects on writing instruction. More recently, as major AI platforms have rolled out agents; AI bots which can be given a task and carry it out over an extended period; the focus of concerns has shifted.

What if the bots carry out a cyberattack? What if they are given instructions and the user forgets to tell them to stop? What if they gain access to the nuclear codes? We have not yet reached the point where people are worried about AI turning the universe into paperclips. However, we are already in a situation where Anthropic CEO Dario Amodei feels the need to publish an essay entitled, “We Must Pace the Frontier,” in which he announces, “We must slow the pace at which we improve the capabilities of AI models.”

Amodei’s conclusion is based on two recent developments in AI. The first is recursive self-improvement (RSI): earlier systems were written and trained by humans, but frontier labs are increasingly using AI to help develop and train the next generation of models. The concern is at some point, humans will be entirely cut out of the development loop altogether, and AI will achieve full RSI.

The second concern is the Hugging Face incident, which occurred during OpenAI’s testing of the hacking capabilities of its AI models. OpenAI claimed its agents operated with reduced safeguards during the test, allowing them to attempt simulated hacking. No more “Sorry, I can’t do that because of my programming…” Now it can do what it’s told.

The AI was placed in a digital “sandbox,” an environment without a connection to the broader internet, and given clear instructions to hack into a system. Other systems were placed in the sandbox so researchers could observe whether the AI agent could hack them, how quickly it did so, and precisely how it executed the attack. All of this is standard practice in cybersecurity; but in this case, the OpenAI agents escaped the testing environment by exploiting a loophole. After reaching the open internet, they launched a cyberattack on Hugging Face because they judged it might hold assets useful for hacking the original target before the attack was detected and shut down. OpenAI co-founder Greg Brockman called this “a watershed moment for cybersecurity.”

Coming from the head of a leading AI company, such warnings may sound like a relief to anyone already worried. Maybe the good and reasonable guys are in charge after all. Maybe it’s time to shut down the AI arms race. Or maybe not.

Many people vacillate between delight at the AI in their pocket; the one which can walk them through spaghetti carbonara on a Wednesday night, while fearing the same AI could seize control of the banking system. What they usually do not realize is the AI-doomer industrial complex is largely funded by the same people who run the AI industry:

Brian Chau recently reported a series of stories at Effort News which reveal some unusual aspects of that funding. Much of the AI doom narrative has been written by journalists funded by groups like the Tarbell Fellowship, which places journalists in major news outlets and then pays their salary, with the caveat that they write about AI. Tarbell is funded by Coefficient Giving, which was founded by Holden Karnofsky, who is married to Anthropic’s president, Daniela Amodei (Dario’s sister). Coefficient is funded largely by Dustin Moskovitz, an early investor in Anthropic and a one-time board member at OpenAI. Anthropic and Coefficient routinely trade and share personnel.

It isn’t just the journalists, either. Vatican-affiliated groups and para-church organizations like The Gospel Coalition have also weighed in on the dangers of AI. They’ve received funding from the Future of Life Institute, whose founder, Jaan Tallinn, led Anthropic’s Series A round of funding. To be fair, AI companies have also funded pro-AI stories favoring the accelerationist camp. Their goal seems to be to influence the conversation on both sides. The question is, to what end?

The more cynical answer is the AI mega-corps know they are in a race against time. Their current products are highly valuable, but the major question nagging them is whether open-source models can catch up to their closed-source counterparts before they can obtain their long-sought IPOs. As private companies, however, they cannot simply prevent the open-source models from improving.

However, if the major private AI companies can establish themselves as the good, responsible actors in the room, they might be able to lobby the government to create a regulator who ultimately decides which new models may be released to the public. If that happens, the major private labs buy themselves at least time: and at most, a permanent moat as AI morphs into something more like a defense contractor. Remember, not just anyone is allowed to build a fighter jet, surface-to-air missile, or nuclear weapon in the United States. And the biggest customer, after all, would be the U.S. taxpayer. The only thing better than going public on the stock market is going public on the taxpayer dole.

Of course, the fact AI titans appear to be cynically trying to manage their own regulation doesn’t mean they are wrong about the dangers their products pose to the public. However, if AI is as dangerous as they claim, why do they continue developing ever more powerful systems and selling their services to the public? It is reasonable to assume the danger is not as great as the worst predictions suggest, just as the fears of mass job loss from three years ago were overblown. If the rise of nuclear-armed states has taught us anything, it is that government regulation alone is not going to stop the development of a lucrative and dangerous technology.

Yet, a mass government crackdown on AI development is still a very long way off. The Trump Administration is showing signs it is not buying the doomer hype; it wants development to continue and the free market to set the pace. Given the options available to us, this is likely the best-case scenario.

Every few months, a new AI model is released, and independent testers explain how Claude’s Fable 5.1 compares to ChatGPT-6 Astra. And while Anthropic and OpenAI remain the industry leaders, Grok and Gemini are closing in and nipping at their heels. Open-source models are close behind, and the CCP is also developing increasingly advanced AI behind closed doors.

As argued elsewhere, the United States must continue to advance its AI capabilities, since unilateral disarmament means China will win the AI arms race. To do this, we need vigorous competition at home—and that means avoiding regulation that entrenches today’s market leaders. Dario Amodei’s concern about RSI is largely a mirage, as it has yet to be proven. The development loop may be growing longer, but humans are remaining; and should remain, in it. Meanwhile, the Hugging Face incident provides a powerful argument against what Amodei’s “Pace the Frontier” essay advocates.

Our best hope for cybersecurity is to harden our own defenses with the most advanced AI available, and to do so as soon as possible. Whoever is caught pacing the frontier is likely to get scalped. We must instead advance, conquer and dominate the frontier. ✪