The Messy Reality of Running AI Agents in a Business

In Brief

Agent hype is loud, but the leaders actually deploying agents are:

• surpassing token budgets in days

• pushing ahead despite no governance for the 20 agents already running loose 

• dealing with pricing that makes planning impossible

The takeaway? The case for moving deliberately is stronger than the case for going all-in.

 

What Happened

Three signals in the same week pushed AI agents from a future consideration to a right-now decision. SaaStr's Jason Lemkin posted a viral take that "three A+ specialized AI agents beat one B+ all-in-one platform every time," naming his stack of Artisan, Qualified, Agentforce, and Monaco. Microsoft made Copilot's agentic capabilities generally available in Word, Excel, and PowerPoint, and turned on by default for every Microsoft 365 Copilot subscriber. And OpenAI launched Workspace Agents in ChatGPT.

So, why aren't you all in on agents right now? 

SmarterX founder and CEO Paul Roetzer broke down the gap between that hype and what enterprises are actually navigating on Episode 211 of The Artificial Intelligence Show.

The Key Numbers

2 - Days it took for a company to use up its monthly AI token budget  

3 - Days into billing cycle that it took for SmarterX to burn through their AI credits in HubSpot

23 - Minutes it took an autonomous Deep Research agent to run more than 300 searches, and generate a full, comprehensive report on AI and startup creation for Roetzer

Push for AI Agents Misses the Hard Part

Token budgets are blowing up in days. At Google Cloud Next, Roetzer spent time with enterprise leaders responsible for governing token spend. "I talk to somebody who's in charge of tokens and they're burning through our whole monthly budget in two days," he says. "How are we supposed to budget for that?"

He hit the same wall in his own company. "I just went into HubSpot today and we're already out of credit. It's three days old, the billing cycle."

Vendor selection is paralyzing. Pick Anthropic, OpenAI, or Google's new agent platform now but you might want to change next quarter? Companies are being asked to commit to multi-year decisions on technology that produces major changes every few weeks.

Twenty agents are already loose with no governance. "Now there's 20 agents running loose that have access to all these different connectors," Roetzer says. "These agents, they function off of knowledge bases and skills. Those things get outdated, right? How are you managing those? Is that in a Google sheet?"

Roetzer also watched a demo from the co-founder of Wiz on how to manage that risk. It underscored how unprepared most companies are for what governing 20 agents actually requires.

Even the most agent-forward shops are improvising. SaaStr is what people point to as the case study. "They're being totally transparent about the fact that they're just figuring this all out," Roetzer says. "They'll build something and rep it, and then they launch it and then it breaks."

There is no way to plan a budget. There are three questions any serious buyer must ask: How do you think clearly about different types of agents? What use cases justify always-on persistent agents? How do you predict cost? A persistent agent could cost five cents a month or $5,000. Companies can't operate on such disparities. 

Roetzer thinks pricing has to change before any of this scales. He suggests value-based pricing tied to human-replacement cost. For example: If an agent is doing the work of three people at a value of $300,000 a year, and OpenAI charged $3,000 a month for those agents? A CEO might be inclined to approve the costs of the AI agents. But right now, it's not that clearcut. 

"It's so messy when you actually get into the real stories of adoption. It's easy to just see the technology and think, 'Oh my God, everybody should be doing this!' No, they shouldn't. It's not ready for prime time yet."

Paul Roetzer, founder and CEO of SmarterX, Episode 211 of The Artificial Intelligence Show

SmarterX Take

Agents are the future, but the loudest voices telling business leaders to go all-in right now are mostly agile, aggressive, AI native startups. These startups can rip up their stack every six months. But AI-emergent companies, working inside legacy systems and regulated industries, can't do that. They have to make decisions that hold up against governance, finance, and procurement, and the agent ecosystem is not built for that yet.

The deliberate move right now is to build skills and workspace agents inside platforms employees already use, get early reps with low-risk workflows, and wait for pricing and governance tooling to catch up.

What to Watch

Pricing models are the next domino. Token and credit billing is already breaking inside finance teams. Watch for the first major lab to roll out value-based pricing tied to outcomes. That shift is the precondition for agents at scale.

Governance tooling will become a category overnight. Expect a wave of products to inventory, monitor, and shut down rogue agents before the first major enterprise incident forces the issue.

The "good-enough" model question is real. If a current-generation model is strong enough to run most agent workflows, and open-source keeps closing the gap, companies might stop chasing every frontier release. That will change how labs sell.

Further Reading

Jason Lemkin: Three Specialized Agents Beat One All-in-One Platform → x.com

Microsoft: Copilot Agentic Capabilities Generally Available → microsoft.com

OpenAI: Introducing Workspace Agents in ChatGPT → openai.com

Google Cloud: Gemini Enterprise Agent Platform → cloud.google.com

Heard on The Artificial Intelligence Show, Episode 211
Paul Roetzer and Mike Kaput discuss the messy reality of deploying agents inside a company. Listen →