Azure OpenAI cost optimization

In 2024, only 31% of practitioners managed their AI spend. In 2025, the number grew to 63%, and now, in 2026, 98% try to keep an eye on every penny that AI utilizes. However, AI cost visibility is not simple, and it’s also the top challenge for practitioners applying FinOps to AI.

Azure OpenAI charges you for tokens you use, not servers. This means each prompt, answer, and the reasoning behind those answers you don’t even see gets billed on your invoice. These invoices are harder to read than your regular VM bills and, as a result, harder to control.

To help you out, we’ve written this guide that helps you understand how Azure OpenAI costs work. We talk about pricing, how you see the charges in cost management, and how to manage them.

Key takeaways

• With Azure OpenAI, you can have standard deployments, batch deployments, and provisioned deployments.
• Standard deployments charge per token, whereas provisioned deployments charge for PTU per hour (whether you use it or not).
• Every cost is split into input, cached input, and output tokens per meter.
• You can minimize and control costs by implementing output caps, prompt caching, picking the model as per the complexity of tasks, and using batch deployments for non-urgent tasks.
• PTUs are better than pay-as-you-go when the traffic is high, steady, and cache-heavy.
• PTU reservations don’t change with model, but they are deployment-type-specific.
• You can’t carry reserved PTU hours forward if you don’t use them.
• In addition to the Azure OpenAI line, you’ll see costs in search, gateway, logging, and network fields, too.

What is Azure OpenAI cost optimization?

Azure OpenAI cost optimization means keeping the cost as low as possible without compromising the usefulness of model responses.

OpenAI cost optimization is different from general VM cost optimization, as VM costs don’t change for each query. However, OpenAI costs depend on tokens, which grow with users, prompts, and even retries.

Let’s understand the pricing model in detail.

How does Azure OpenAI pricing work?

Azure OpenAI charges you either per token or per unit of reserved capacity, depending on the traffic shape. Let’s understand each scenario.

Standard deployments

Standard deployment is your regular pay-as-you-go model, where you pay for the tokens you use. The bill accounts for input, cached input, and output tokens. The token price depends on the deployment type. There are three options for standard deployments:

1. Global: Routes traffic worldwide
2. Data Zone: Processing stays in the US or EU
3. Regional: Processing happens in a specific region

So, for example, GPT 4.1 mini costs 40 cents for a million global input tokens and 44 cents for Data Zone and Regional.

Batch deployments

If you need to process a large amount of workload, but it can wait, then use batch deployments. It costs half as much as standard deployment, but you don’t get results instantly. Azure returns a response within 24 hours.

Not all models provide batch deployment, so check the pricing table here.

Provisioned throughput

With this option, you reserve deployment capacity, but you need to pay for a fixed allocated Provisioned Throughput Units (PTUs), instead of the amount of tokens you use. There are three reservation options:

1. Hourly: Best for benchmarks or short events
2. Monthly: Stable workload over a project
3. Annual: Stable continuous workload for multiple projects

Fine-tuned model hosting and training charges

Fine-tuning costs you twice because you pay for training first and then an hourly rate for deployment of every fine-tuned model. You pay the fees even when no one calls the model. So, for example, fine-tuned model hosting for GPT-4.1 is $1.70 per hour in East US, which amounts to $1,241 per month for every idle deployment.

Reasoning models and hidden output tokens

Reasoning models think to answer. While you don’t see the text for that thinking, you still pay for it as reasoning_tokens under output tokens.

Table 1: pricing and deployment options compared

Where does Azure OpenAI cost show up in your Azure bill?

When your Azure OpenAI bill comes, you see the total cost is visible; however, what contributes to it is difficult to understand. Let us break it down for you.

Finding it in Cost Analysis (service family, resource type, meters)

Go to Cost Analysis in the Azure portal by going to Cost Management + Billing > Cost Management > Cost Analysis. Filter resource type Microsoft.CognitiveServices/accounts. You’ll see Azure OpenAI’s charges under the service name ‘Foundry Models.’ You may also see it under ‘Cognitive Services’ if the records are older.

You can also group it by meter and turn a one line item into a per-model breakdown.

Splitting input, output, and cached tokens by meter

Meter names show what kind of tokens you’ve been spending. You’ll know if they are standard input, cached input, or output tokens. This way, you can see if you’re saving money with prompt caching and if output tokens are driving the cost most.

Telling deployments apart with AdditionalInfo

While you may see the model on the meters, you won’t see the app. So if different teams are using GPT-4.1 under the same resource, their bills will look the same. You will have to go to your cost exports and check the deployment name in AdditionalInfo field.

Why PTU charges appear on days with zero traffic

We mentioned earlier that you pay for fixed Provisioned Throughput Units (PTUs), and you’re paying for capacity rather than usage. So, what it means is that once the meter starts upon creation of the deployment, it can’t be paused. It will only stop when you delete it. So, even when you’re not using OpenAI, and there’s a day without any requests, it still shows a full day of PTU spend.

What are the fastest ways to reduce Azure OpenAI costs?

The easiest way to save on Azure OpenAI costs is using fewer tokens, using cheaper models, and making use of discounts. Here’s how you can do that:

Cap output tokens

Output tokens are often more expensive than input tokens, so you want to ask for shorter responses directly in the prompt. Cap response lengths by setting max_completion_tokens on every API call. It also limits use of reasoning tokens on reasoning models.

Structure prompts so caching actually triggers

You can also save money by reusing text with prompt caching. However, you need to follow two rules:

1. The prompt must be at least 1,024 tokens long
2. The first 1,024 tokens must be word-for-word the previous prompt

If you break any of these rules, you break the cache. So, add the static content with the system prompt, tool definition, and references first, then add the user question or timestamps. If cached_tokens in the response is zero, then it means you paid the full price.

Route simple tasks to mini and nano models

The GPT-Chat Latest Global is $5 for every million tokens, whereas GPT 4.1 mini is $0.40. Not all tasks need the latest model. Choose smaller models for simple tasks like summarizing, data extraction, or classification.

Move async work to batch deployment

For tasks that don’t need an instant response, like nightly reports or document backfiles, use batch deployments. Turnaround time is 24 hours, but it offers 50% discounts on standard deployment rates.

Control reasoning effort on reasoning models

You pay for hidden reasoning in the reasoning model. To save money, minimize the time spent from none to max depending on the model. So, run simple tasks on lower settings and increase reasoning only if you need better quality responses.

Delete idle fine-tuned and dev deployments

When you have provisioned deployments, idle time quietly eats up budget. Azure often deletes a fine-tuned model after 15 days of inactivity, but you should not let 15 days of hosting add up. Delete test deployments after the test ends, spot orphans early, and trim down on token bills.

When do PTUs beat pay-as-you-go?

You don’t want to run idle days on PTUs, but they can be a really good option over the pay-as-you-go model when you have predictable traffic that’s close to full capacity. Let us give you an example to show you when PTUs beat pay-as-you-go.

Say a model costs you 10 bucks for 1,000 requests, and PTU costs $5,000 a month with a capacity to handle a million requests. PTU is profitable only if your workload has over 500,000 requests every month. Now, understand with real numbers for Global deployment using GPT-4.1.

• Retail price: $2.00 for input, $0.50 for cached input, and $8.00 for output per million tokens
• PTU price: $260 per PTU per month (as per an example in Microsoft’s sizing guide)
• PTU throughput: 3K input tokens per minute per PTU. (One output token = four input tokens)
• Workload: 2K input tokens and 400 output tokens per request with a 50% cache hit rate over a 30-day month
• Peak traffic: 1.5x the average

Note: Cached tokens don’t consume PTU capacity for older models like GPT-4.1, but for new models like GPT-6 Sol, it uses weighted accounting.

Refer to the table below, and you’ll see that PTU gets wasted when you can’t utilize the peak capacity all month.

Table 2: PAYG vs PTU cost over three different volumes over a month for GPT-4.1 Global

There are two conditions that can actually flip the scenario:

1. Flat traffic
2. Heavy prompt caching

Cached GPT-4.1 will cost $0.50 per million tokens on pay-as-you-go. Cached tokens don’t consume PTU capacity at all.

So, now consider an internal support agent that’s online throughout the day. All calls have a lengthy, fixed system prompt with a knowledge pack. Let’s use these inputs for example:

• Steady traffic: 300 requests every minute, peaking at 330 (1.1x average)
• Workload: 4K input tokens and 300 output tokens for each request
• Caching: 80% of input tokens cached

Considering the prices are the same as above, for pay-as-you-go, you’ll send:

• 51,840 million input tokens a month: 10,368 million uncached tokens and 41,472 million cached tokens, costing $20,736 and $20,736, respectively.
• Output tokens are 3,888 million, which cost $31,104

So, the total cost comes to: $72,576.

Now, for PTU reservation, you size for 330 RPM peak.

• Input contributes to 264,000 tokens (uncached) per minute
• Output contributes 396,000 tokens per minute (given its four times the weight)

Total is 660,000 normalized tokens per minute or 220 PTUs, costing $57,200.

On pay-as-you-go, you send 51,840 million input tokens a month. The 10,368 million uncached tokens cost $20,736. The 41,472 million cached tokens cost another $20,736. Output adds 3,888 million tokens, or $31,104. The total comes to $72,576.

For PTUs, you size for the 330-RPM peak. Uncached input contributes 264,000 tokens per minute. Output contributes 396,000 after the four-times weighting. That’s 660,000 normalized tokens per minute, or 220 PTUs. At $260 each, you pay $57,200.

In this case, PTUs save you $15,400 a month, and an annual reservation could save even more.

So, the lesson isn’t that PTUs save you money, but choosing as per traffic shape and cache rate can make you most of these pay models.

Reading the Provisioned-managed utilization V2 metric

This metric gives you information on how much PTU capacity you’re using by comparing consumed capacity to total deployed capacity. You can break it down by individual deployment. When capacity is utilized 100%, you get a brief pause on requests with HTTP 429 and a retry warning. This is a traffic management signal rather than an error.

When you go over your capacity, the extra requests get charged by standard deployment. It’s known as spillover, and you can use it as a tool to estimate your capacity to set PTU for your average traffic. Split the AzureOpenAI Requests metric with IsSpillover to track the spillover.

Scheduling hourly PTUs for business-hours workloads

While you can create hourly PTUs to match your business hours, it is not recommended. Once you delete a deployment, you release its reserved capacity. You have no guarantee that you can get it back later. Plus, hourly billing often costs more than a monthly or yearly reservation.

Table 2: PAYG vs PTU cost at three monthly volume levels

How do Azure Reservations for PTUs work?

With a PTU reservation, you get a discount for committing to a fixed number of PTUs every month or year. Let’s understand in detail.

Reservation scope and the Global multi-region rule: For every deployment, be it global, data zone, or regional, you can buy a specific reservation that can’t be used interchangeably. So, as a rule of thumb, create the deployment first and then buy the reservation for it.

Utilization vs coverage for PTU reservations: Utilization means how much of the reservation you have used, and coverage means the PTU spend discounted under the reservation. PTUs deployed under the reserved amount will get the discounted rate; anything that goes beyond gets charged hourly. If you don’t utilize reserved PTU in its given time, you lose them.

When you change models: Nothing happens to your reservation when you change models as long as your new model is within the scope of your PTU reservation. This means the new model must match deployment type, region, and scope. However, it is important to note that different models consume different amounts of tokens. For example, GPT-4.1 handles 3,000 input tokens per minute per PTU, while GPT-5.4 handles 2,400.

Note: If you delete a deployment under reservation, it will not cancel the reservation automatically. You must do it manually in the Azure portal.

What hidden costs come with an Azure OpenAI workload?

Azure OpenAI is rarely covered by a single line on the bill. There are different charges under its own service that you need to look through the filter in Azure OpenAI. Here’s where you can find hidden costs:

• Search: If you look up documents on your app, there is a fee specifically for search capacity. There’s also an additional fee for the advanced ranking feature that you can find under Azure AI Search.
• API management (APIM): There are separate service charges to use the gateway to manage traffic and limits.
• Network: You pay an extra data fee on top of the hourly rate for private endpoints. Sending data across different regions will add bandwidth charges too.
• Storage: Batch deployments save input and output files in storage unless deleted, and add to cost.
• Fine-tuning: As we mentioned earlier, fine-tuning tests come with training tokens even when you don’t end up using the model.

Tip: If you log every prompt and response, you’ll lose money as Azure Monitor charges by data volume. Instead, you can save money by logging metadata by default through Log Analytics and diagnostic settings.

This covers Azure AI Search, an APIM AI gateway, Log Analytics and diagnostic settings, private endpoints and egress, storage for batch input and output, and fine-tuning training runs.

How do you allocate Azure OpenAI costs to teams and features?

You can allocate Azure OpenAI costs to teams by separating them at the resource, deployment, or request level. The approach you choose depends on how many teams share a model.

The one resource per team approach gives you the cleanest way to separate costs. All resources have their own tag, making standard Azure cost allocation possible. However, this approach can be sprawled as resources grow; you’ll have more quotas and more keys to manage. If you do shared resources with separate deployments, you cut the overhead. You’ll export costs for each team by deployment instead of resources. This works if you only have a few teams.

You can also use API management to count tokens for each call using the llm-emit-token-metric policy. Doing this will log each prompt, response, and the tokens used in Application Insights, where you can add five custom labels like user or subscription ID. You can track how many tokens are used for each feature to calculate costs easily by tagging calls by team or feature name.

Tagging practices stop working beyond the resource level. If you have ten features under a single deployment, all the costs go under a single tagged resource, and it becomes impossible to split them.

You can track cost per request using a consistent key across services for API gateway metrics or application logs.

How do you govern Azure OpenAI spend?

To keep track of and control your Azure OpenAI spend, you need to implement a four-layer strategy:

1. Restrict models and deployment types using Azure policy
2. Set a spend ceiling and give every standard deployment a token-per-minute limit based on workload needs (and not max availability).
3. Scope budgets to the resource and delegate accountability so someone responds when the budget is closing or surpassed.
4. Set daily anomaly checks and alerts on token meters so you don’t have to catch Azure cost anomalies fast without waiting till month end to see it reflected on your bill.

Which KPIs should you track for Azure OpenAI cost?

The table below shows six metrics to track that help you tie token spend to work done.

Table 3: Azure OpenAI cost KPIs to track

Common Azure OpenAI cost mistakes

If you’re trying to minimize costs related to Azure OpenAI, don’t make these common mistakes (you’re probably making right now):

1. Always using the biggest model for all queries
2. Breaking the cache by putting dynamic content like user ID or timestamp in the first line
3. Buying PTU reservations when your traffic and usage mostly don’t reach the peak
4. Buying a reservation before deployment because reservations don’t guarantee capacity
5. Not accounting for reasoning tokens in the bill
6. Leaving the test deployments idle and running long after they’re useful
7. Keeping an eye only on the Azure OpenAI line and not search, gateway, logging, and network costs

Tools for managing Azure OpenAI costs

Microsoft’s native tools for cost management provide you with the basics for cost tracking. Here’s a list of tools you can get started with right now:

1. Microsoft Cost Management: Lets you see Azure OpenAI spend by meter, set budgets, and enable anomaly detection.
2. Azure Monitor: Lets you track tokens, requests, PTU utilization, and spillover.
3. Azure Advisor: Suggests fixes for general cost and reliability issues that are present in your subscriptions.

Microsoft’s native tools are free and often sufficient for a team with few subscriptions. However, they fall short when the environment grows. At one point, it becomes difficult to stitch the cost data and token telemetry together to understand the spend, as both live in different places. Additionally, a shared deployment with several teams will need custom gateway work, and you’ll need someone on the dashboard daily to catch all token spikes.

This is where Turbo360 comes into the picture. Turbo360 provides FinOps for AI in Cost Analyzer to bridge the gaps left by native tools. It provides:

• Cost breakdowns at meter level: Shows input, output, and cached tokens for each model. If the model changes, you see it immediately in the cost mix.
• AI spend allocation: By model, project, team, or business application so you can assign a responsible owner.
• Daily anomaly monitoring: Checks spend each day against baselines, and an AI agent verifies any spike before alerting the owner.
• Real-time token alerts: Pulls data from Azure Monitor and Log Analytics to set thresholds on total tokens without waiting for bills to arrive.

Take control of your Azure OpenAI cost expenditure starting today. Book a demo with our experts and see how Turbo360 can help you analyze token metrics, set up proactive alerts, and draw cost-saving opportunities before your bill arrives.