OpenAI Seeks Expert to Manage Growing Risk of Self-Improving AI

OpenAI is searching for a new executive to manage the emerging dangers of its most advanced AI models, and they're willing to pay more than a half million dollars to find them.

The company has posted a job listing for “Head of Preparedness,” a role that CEO Sam Altman publicly described as "stressful" and requires candidates to "jump into the deep end pretty much immediately." The position offers a base salary of $555,000 plus equity, making it a highly lucrative safety-focused role.

But the real story isn't the paycheck. It's what the job listing, and Altman's unusually candid framing, reveals about where AI capabilities are headed and how seriously the leading labs are now taking the risks.

To understand what this move signals, I spoke with SmarterX and Marketing AI Institute founder and CEO Paul Roetzer on Episode 189 of The Artificial Intelligence Show.

Urgent Risk Management Needed

The Head of Preparedness will lead OpenAI's technical strategy for tracking "frontier capabilities that create new risks of severe harm," according to the job post. The role spans some of the most sensitive domains in AI: cybersecurity, biosecurity, and what the company calls "deceptive behavior."

In his public post announcing the search, Altman was unusually direct about the stakes. He noted that AI models are now "so good at computer security they are beginning to find critical vulnerabilities." He also referenced the "potential impact of models on mental health," noting the company saw a preview of that last year.

Altman went further, explicitly naming self-improving AI systems as a core concern: "If you want to help the world… gain confidence in the safety of running systems that can self-improve, please consider applying."

Says Roetzer: "I definitely think it's a sign that they are very confident that they're very close and that they need to take their preparedness framework much more seriously."

The Capability That Changes Everything

OpenAI's preparedness framework, most recently updated in April 2025, outlines several categories of risk the company is tracking. Two of the most prominent are biological and chemical capabilities and cybersecurity threats.

But there's a third category that might matter most of all: AI self-improvement.

The framework describes this as capabilities that "in addition to unlocking helpful capabilities faster, could also create new challenges for human control of AI systems."

This is a topic Roetzer has discussed extensively, particularly over the last year. Today's models are essentially frozen after training. They don't learn or update their own knowledge base after deployment. They rely on external tools like search to access new information.

But that could be changing.

"If the models can come out and then continually learn and make updates to their own knowledge base, that is a major unlock," says Roetzer. "And it leads to other things like memory and this ability to do self-improvement."

What the Labs Aren't Saying …Yet

There's growing talk in AI circles that the major labs are significantly further along in these areas than they've publicly acknowledged.

Roetzer points to recent buzz around Google in particular, with reports suggesting the company has made advances in continual learning that haven't been formally announced.

"There's a lot of chatter around self-improvement, continual learning that tell me that the labs are much farther along in both of those areas than we might be aware of," he says. "And so I think all the labs are starting to take this area very seriously."

This would explain the urgency behind OpenAI's hiring push and Altman's public admission that the company needs "more nuanced understanding and measurement of how those capabilities could be abused."

It’s also a likely signal that the leading AI lab believes it's approaching a threshold where the risks of frontier models require a fundamentally more serious response.