Anthropic published an article with potentially profound implications for the near future.
“When AI builds itself” details the AI lab’s progress toward recursive self-improvement, and what it could mean to business, the economy, and society.
“For most of AI’s history, humans drove every step in its development cycle. But at Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work," the article states.
“Taken far enough, and given enough compute, that trend points to an AI system capable of fully autonomously designing and developing its own successor. This is called recursive self-improvement. We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for.”
AI labs are focused on automating AI research to accelerate model development and capabilities. These breakthroughs will then be applied to achieve the same results in all other industries and professions.
“The rate at which AI models improve is accelerating. The length of tasks that they can reliably complete on their own has been doubling roughly every four months, up from an earlier trend of doubling every seven months.”
“The comparative advantage of humans as of right now is still in seeing the bigger picture and thinking beyond the confines of the immediate task.”
“The evidence suggests that the human role is narrowing at each step in the AI development process. . . . Put simply: the doing (i.e., writing the code, running the experiment, producing the result) now costs almost nothing in human time, even if it still has costs in compute.”
I’ve summarized them below with excerpts from the post:
1) The trend stalls, but today’s AI capabilities are widely diffused. This article features many exponential trajectories. But these trajectories may actually turn out to be S-curves. We may be approaching the bend in the curve, where returns to scale diminish and the line straightens, then flattens.
Even if model capabilities were frozen at today’s level, we would expect major changes to occur in the world.
We are still early in the diffusion of today’s models into the wider economy, where a 100-person company can increasingly do the work of a 1,000-person one, because each employee will sit atop a pyramid of agents.
We include this scenario for completeness, but we don’t believe it’s likely.
2) AI labs continue to see compounding efficiency gains. In this scenario, AI development becomes substantially automated, but humans continue to set research directions and judge results.
Organizations that use AI systems would become much more efficient as time goes on, so we could expect to see significant productivity multipliers on each person in this organization. 100-person companies could do the work of 10,000- or 100,000-person organizations.
This would revolutionize knowledge work and government services . . .
The evidence we’ve laid out here suggests that we’re likely heading into this scenario.
3) AI systems themselves become capable of full recursive self-improvement, and begin building their successors. If technical trends in advancing capabilities continue, and AI systems are able to develop the capabilities inherent to transformative human ingenuity, then it is plausible that AI systems could design and refine themselves.
In this world, the pace of progress in AI development becomes determined entirely by the availability of compute (or the speed of discovering various efficiencies in algorithmic training or inference) for AI systems.
Humans play a substantially diminished role in their development, likely moving most of our effort towards oversight, validation, and verification of an expanding “virtual lab” run by AI systems.
We do not have good intuitions for what this world would look like, because our economy is currently driven by humans and human-built tools.
I would highly recommend reading the full article to understand the state of AI progress.
Most organizations are stuck on the possibilities of the first scenario (i.e. dealing with the AI tech that is already here), while they should be racing to prepare for the second scenario (i.e. exponentially improving models with widespread impact across industries).
Don’t wait for the world to get smarter around you. Take steps now to prepare your organization and its people for an accelerated future.
Want This Content Sooner?
You can gain access to these insights before they appear on our website! Subscribe to Paul’s ExecAI Newsletter and get it delivered to your inbox every Sunday to start your week.