KETTLE Like a drug dealer who's hooked you and raised their prices, business leaders are simply shocked to learn the AI their organizations are becoming dependent on is suddenly a lot more expensive. You can listen to the latest episode of The Kettle right here on this page, as well as on Spotify, Apple Music, or YouTube where you can subscribe to get notified of the latest episode.
Kettle host Brandon Vigliarolo is joined by Reg reporter Lindsay Clark and contributor Joab Jackson this week to discuss their recent stories about the rising cost of enterprise AI - and one way a popular open source project is trying to fight tokenmaxxing with tokenminning - but the question remains whether such measure will be enough to prevent cost-benefit analyses from popping that bubble.
Will the AI industry adapt to the fact it's still unprofitable, blowback from usage-based billing, and a desire to not pay AI models as much as the human devs they're supplementing or replacing? That's what's on the hob for this week's episode. A lightly edited transcript is below: Brandon: Welcome back to The Register Kettle podcast.
I'm Reg Reporter Brandon Vigliarolo, and you're going to be absolutely shocked to discover what we're talking about this week. I'm kidding, of course, it's AI. Specifically the fact that it seems the world is starting to wake up to how much it costs to actually run these giant models that are supposed to make life easier for enterprises and their employees, but it seems like they're leading to some invoice shocks.
With me to talk about the latest panic over AI costs are Reg reporter Lindsay Clark and our contributor Joab Jackson. Thanks to both of you for coming on. Lindsay Clark: No problem, thank you.
Joab Jackson: Thank you. Brandon: So Lindsay, let's start with a story you wrote recently about the fact that C-suite occupants are apparently having trouble getting a handle on new usage-based AI costs. What exactly has them so confused?
Lindsay Clark: I get the impression that big companies are diving in with both feet with AI. It's the latest trend. They're using it for a lot of coding and business apps, trying to do stuff in the business with it.
KPMG is a massive global consultancy. They provide IT services and outsourcing services. Brandon: And they're the ones who wrote the report, correct?
Lindsay Clark: That's right; they wrote the report and they have some skin in the game. They did a survey of more than 2,000 senior execs over 20 countries and found that 29 percent of them struggled to understand the operating costs as they scale with enterprise AI deployments. Nearly half of them were also looking to re-phase their AI deployments when the costs outweigh the expected value.
Brandon: Explain re-phase. Are they rethinking the deployment itself or are they changing the scope? What's that mean exactly?
Lindsay Clark: It's just slowing down and looking at what they're doing. They're looking at lower-cost models and high-fidelity models. It's looking for a mix of models to deploy rather than just maxing out on the most expensive ones.
Brandon: Usage-based billing seems to be a relatively recent development in this space. It was all free samples until we get you in the door to the point where you're dependent on this, and then we realize we actually have to make money off this. Speaking as an AI frontier lab, we're going to have to charge you per token because we're just not able to turn a profit.
Lindsay Clark: Anthropic, OpenAI, and GitHub have all moved from a subscription, flat-fee, all-you-can-eat model to usage-based billing based on tokens. The vendors, both the model providers and the application vendors that want you to use AI agents in their applications, want people to jump in with both feet and use this stuff as much as possible. Then, as is typical for the IT sector, they try to change the commercials as we go.
Brandon: It's a big enough issue that more than a quarter of C-suite people are getting bill shock and realizing that they might not be able to afford this. But they're maybe a little bit hooked in because their engineers have been using this long enough. I wrote a story recently about an open source tool that someone wrote to test engineers to make sure they're not losing their edge in this environment, because a lot of them are.
We've written plenty of stories about developers becoming dependent on this, forgetting how to do some of the basic things they used to be able to do. It gives these AI companies a big inroad to basically say, "Well, now we're going to actually try to make money." But it does put them in a precarious position.
If you charge too much, these enterprise customers are going to try to find a way around it, whether it's an open source Chinese model or some other solution, rethinking their deployments and trying to go with smaller, large-scale models. But if you charge too little, you're never going to make enough money. Is this a needle that these AI labs can thread, or is it one that's pointed straight at the bubble?
Lindsay Clark: There's a report from Gartner from a few weeks ago that was quite interesting. They had done some research about the cost on this topic for AI-assisted coding. Brandon: Right, this is one you covered back at the end of June, right?
Lindsay Clark: That's right. A researcher called Nitish Tyagi was saying that there's a real lack of transparency from the vendors over the costs of their coding agents and they don't have cost optimization tools that you would expect in the cloud, for example. Because of this, the costs of the coding agent per developer was going to exceed the actual salary of the developer in 2028.
That is the average salary globally. He was already finding that in areas of the world where salaries are a lot lower, like India, the cost of agents is actually exceeding the salary of the developer.
Originally published at theregister.com
