Provisioned throughput vs pay-per-token for LLM serving
Reserved LLM capacity against per-token billing: what each buys, the units and terms each vendor uses, and how to work out which is cheaper for your traffic.
For: Anyone choosing between reserved and per-token LLM capacity, on any platform
Last verified How we verify
Every vendor in the matrix sells LLM serving in one of two ways, and several sell both. Pay-per-token charges for what you send and receive. Provisioned (or reserved, or dedicated) capacity charges for time, whether or not you use it. This guide compares the two on primary sources checked on 22 October 2026.
What each one buys
Pay-per-token prices input and output tokens separately on shared serving. Microsoft’s comparison gives Standard deployments no latency SLA and recommends them for development, testing, low volume and variable traffic.
Provisioned capacity reserves throughput for you. Microsoft charges for it hourly whether or not the deployment is handling requests, and AWS bills Provisioned Throughput until you delete it. In return you get a defined latency target (Azure) or a guaranteed throughput level per unit (AWS). Microsoft recommends it for predictable traffic, latency-sensitive applications and production volume where per-token billing becomes expensive.
Reserved capacity is not automatically single-tenant hardware. Some vendors say their dedicated capacity is single-tenant (Together AI states every dedicated deployment runs on single-tenant, isolated GPUs); others reserve throughput without describing the hardware. Check the deployment and notes on each profile.
Units and terms differ by vendor
| Vendor | Reserved unit | Terms (from the vendor’s documentation) |
|---|---|---|
| Azure OpenAI | Provisioned throughput unit (PTU) | Hourly, or 1-month or 1-year reservations |
| Amazon Bedrock | Model unit (MU) | No commitment, 1 month or 6 months |
| Vertex AI | Generative AI scale unit (GSU) | Fixed-cost terms, including 1-week terms for select models |
| Databricks | Model units or throughput bands | On-demand with no term commitment, or reserved |
| Together AI | Throughput units (PTUs) or GPU-hours | Reserved capacity, or per GPU per hour |
| CoreWeave | GPU-hours | On-demand, spot, or up to 60% off for committed usage |
Dedicated GPU platforms bill time directly instead: Fireworks AI per GPU second, Modal per second, Baseten down to the minute, and Nebius dedicated endpoints while at least one replica is running.
Two units share a name: Azure’s PTU and Together’s throughput unit are different vendors’ measures and cannot be compared one for one.
Working out which is cheaper
The method is the same everywhere:
- Price the smallest reservation you could buy for the hours you need it.
- Price your expected token volume at the same model’s per-token rates.
- Compare, then check that the reservation’s throughput covers your peak.
With Microsoft’s published GPT-5.5 rates on 22 October 2026 (estimates, in US dollars): the minimum Global provisioned deployment is 15 PTUs at $1 per PTU per hour, so $15 an hour. At Standard Global rates of $5 per million input tokens and $30 per million output tokens, $15 buys 3 million input tokens or 500,000 output tokens. If your hourly Standard bill for real traffic would exceed $15, and 15 PTUs can carry that traffic, provisioned is cheaper; the throughput per PTU is model-specific and not verified here. A monthly reservation ($260 per PTU on Global) moves the line further towards provisioned for steady traffic.
Spiky traffic usually favours per-token billing, or a reservation sized for the baseline with overflow sent elsewhere; Azure supports this with spillover to a Standard deployment.
Vendor detail
Sources (13), each checked on the date shown
- https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/provisioned-throughput (2026-10-22)
- https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/provisioned-throughput-billing (2026-10-22)
- https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html (2026-10-22)
- https://www.together.ai/dedicated-endpoints (2026-10-22)
- https://cloud.google.com/blog/products/ai-machine-learning/provisioned-throughput-on-vertex-ai (2026-10-22)
- https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/deploy-prov-throughput-foundation-model-apis (2026-10-22)
- https://www.together.ai/pricing (2026-10-22)
- https://www.coreweave.com/pricing (2026-10-22)
- https://fireworks.ai/pricing (2026-10-22)
- https://modal.com/pricing (2026-10-22)
- https://www.baseten.co/pricing (2026-10-22)
- https://docs.tokenfactory.nebius.com/ai-models-inference/dedicated-endpoints/billing-policy (2026-10-22)
- https://azure.microsoft.com/en-us/pricing/details/azure-openai/ (2026-10-22)