No per-token markup on your own models, no platform fee, and no charge for idle. Bring your weights; you're billed for the compute they use.
For evaluation and side projects.
For serving models in production.
For committed, steady-state capacity.
For regulated, high-throughput workloads.
On-demand managed inference, billed to the second. Scale-to-zero means you're never billed for an idle replica.
| Accelerator | Memory | Best for | On-demand | Reserved |
|---|---|---|---|---|
| L4 | 24 GB | embeddings, small models | $0.70 / hr | $0.42 / hr |
| A10G | 24 GB | 7–8B models, high concurrency | $1.10 / hr | $0.66 / hr |
| L40S | 48 GB | 13–30B models, best price-per-token | $1.90 / hr | $1.14 / hr |
| A100 · 40 GB | 40 GB | 13–30B models | $2.20 / hr | $1.32 / hr |
| A100 · 80 GB | 80 GB | 30–70B models, tensor-parallel | $3.50 / hr | $2.10 / hr |
| H100 · 80 GB | 80 GB | 70B+ models, lowest latency | $5.10 / hr | $3.06 / hr |
Rates are representative for on-demand managed inference. Reserved pricing locks a fixed rate against a minimum commit. Spot capacity is available at a further discount for batch, offline, and evaluation workloads that can tolerate interruption.
No. You're running your own model, so there's no model provider taking a cut. You pay for GPU compute and minimal egress — nothing else.
Time a replica is loaded and serving. When an endpoint scales to zero, billing stops. Warm resumes are part of the replica's run time, not billed separately.
No. Weights can be pulled from Hugging Face, a private registry, or your own object storage at deploy time. You keep full ownership and can remove them at any time.
Directly — a smaller memory footprint can fit the same model on a cheaper GPU. An 8B model that needs an A100 at BF16 might fit an A10G with INT4 at roughly half the hourly rate.
Per accelerator, by the second. A tensor-parallel deployment split across two A100s runs at twice the single-GPU rate, with no parallelism overhead fee on top.
Yes, at a discount for workloads that can be interrupted — nightly evals, bulk inference, and offline fine-tuning runs.
Deploy your first model on the starter tier and move to reserved or dedicated capacity as traffic grows.
Get started