This blog post is the second of a four-part series called The Economics of Agent Optimization which shares the strategies, capabilities, and proof points to help you optimize agent costs and run AI as a managed investment system on Microsoft Foundry. The first post set out the three decisions that system rests on: optimize each request at runtime, optimize each workflow over time, and govern spend continuously.
This post takes the first, the one that touches every dollar you will ever spend on AI. An agent is a loop around a model. It plans, calls a tool, reads the result, and reasons again, so a single completed outcome can take a dozen model requests.
That is why the number the business cares about is the cost of a successful outcome, not the price of a token. Every turn in that loop is still one model request, and each request carries decisions about the model, the offer it runs on, what gets reused, and what the model is told. When those decisions are right, the saving repeats on every turn.
That is why agent optimization starts here. Learn how Microsoft Foundry can help you save time and money The most expensive habit in production AI Most AI applications are built the same way. In the prototype, you pick the strongest model available, put everything the model might need into the prompt, and confirm the idea works.
That is the correct instinct for a prototype; the problem is what happens next. The prototype’s defaults quietly become the production architecture, and a pattern designed to answer “can this work?” becomes responsible for answering “can this scale economically?”
Two things break at that point. First, AI workloads are not uniform . A single application mixes intent classification, extraction, formatting, summarization, and genuine multi-step reasoning—AI workloads vary enormously in complexity.
Routing all of them to one frontier model means overpaying on the majority of requests that never needed that capability. Second, one outcome is many requests. A prototype pays for a single call.
An agent pays for the whole loop, so anything wasteful gets multiplied. That is true of tokens, and it is more true of mistakes. An agent that takes a wrong turn calls the wrong tool and loops to recover, burning tokens on turns that should never have happened and still landing on a weaker answer.
Cost per outcome is set as much by the turns you avoid as by the tokens in each one. In production, the goal is not to minimize tokens. It’s to reduce the cost of a successful outcome while maintaining quality, safety, and latency.
Every runtime decision must balance those factors together, which is why the economics of a request come down to four decisions. Four levers you control at runtime Microsoft Foundry gives you four levers for making those tradeoffs deliberately, rather than accepting the ones your prototype happened to choose. Each can be adopted on its own, measured against your quality bar, and reversed if the tradeoff does not hold.
Lever Foundry capability Models and offers Model router, deployment types, provisioned throughput, batch, fine-tuning. Caching Prompt caching, semantic caching through the AI Gateway in Azure API Management. Prompt and agent optimization Prompt optimizer, agent optimizer across instructions, skills, tool descriptions, and model selection.
Observability and evaluation Foundry observability and evaluation, agent traces, Azure budgets, alerts, and cost tagging. 1. Send each request to the right model The principle is simple: optimize the outcome based on the task complexity.
Routine requests should not pay frontier-model economics, while complex requests should not sacrifice quality simply to save tokens. Model router in Foundry Models removes that tradeoff. It assesses each incoming request and dispatches it to the most suitable underlying model in real time, behind a single endpoint and a single deployment.
Routing modes let you prioritize cost, quality, or a balance of the two. Model subsets, which now align with Azure Policy , constrain routing to an approved allow-list where a compliance boundary applies. Built-in failover moves a request to the next best model when one is unavailable, so routing also buys resilience.
The same request can carry very different economics depending on how it is deployed, and this is the lever teams most often leave untouched. Organizations must decide: Where data is processed (Global, Data Zone, or Regional). How throughput is purchased (pay-per-token, provisioned capacity).
Which workloads truly require interactive responses. Foundry provides multiple deployment options that allow these choices to align with business requirements. Most workloads can start with standard deployments, which provide the greatest flexibility and cost-efficient pay-as-you-go pricing.
Interactive applications that require faster and more consistent response times can benefit from priority processing, while high-volume workloads with predictable demand can achieve better economics through Provisioned Throughput Units (PTUs), with overflow traffic handled through pay-as-you-go capacity. Large asynchronous workloads such as document processing, classification, and evaluation runs are often best suited for Batch deployments, which provide up to 50% lower costs for work that doesn’t require immediate responses.
Even within a single application, different experiences often benefit from different deployment strategies. Developer-facing tools that can tolerate some latency variability may run efficiently on Standard deployments. Interactive chat experiences may warrant priority processing, while agentic applications with sustained throughput demands can maximize value with PTUs.
Background tasks such as document analysis, knowledge extraction, and large-scale classification can move to Batch without affecting the end-user experience, reducing cost simply by selecting the deployment model that matches the workload.
Originally published at azure.microsoft.com
