Release notes.

Shift-aware autoscaling schedules

Autoscaling can now follow a weekly schedule, so fleets spin up for known peaks — first shift, settlement windows, a launch — and drop to zero outside them.

• Recurring schedule rules in addition to metric-based scaling
• Cost attribution API for per-deployment and per-tag spend
• Scale-to-zero cooldown configurable down to 30 seconds

Compliance posture: SOC 2 & HIPAA

• SOC 2 Type I report available under NDA
• HIPAA posture for healthcare workloads on dedicated fleets
• Audit logs and role-based access controls

Fleet product & reserved capacity

Dedicated and reserved GPU fleets graduate to their own product, with VPC peering and committed-use pricing.

• Reserved capacity at a discounted GPU-hour rate
• VPC peering and private connectivity
• Enterprise tiers with dedicated support and SSO

Quantization: FP8, INT8, INT4

• FP8, INT8, and INT4 quantization applied at deploy time
• Automatic quantization matched to GPU memory and model size
• KV-cache FP8 dtype to roughly double cache capacity

Training product enters beta

• Distributed fine-tuning and training jobs
• Spot capacity with automatic rescheduling on interruption
• Deterministic resume from checkpoint

Observability dashboard

• Per-model latency, throughput, and GPU utilization
• Token and cost accounting per deployment
• OTLP export to Datadog, Grafana, and Prometheus

VPC peering & role-based access

• VPC peering for private connectivity
• Workspace roles and quotas
• Multi-region serving

General availability

Managed inference for open-weights and fine-tuned models is generally available, with the first version of the pricing model.

• Autoscaling to zero
• Warm pools for cold-start reduction
• OpenAI-compatible endpoint

Private beta: autoscaling & cold starts

• Autoscaling to zero on request volume
• Warm-container resumes
• CLI deploy and logs

Private beta

Managed inference for open-weights models, offered to a small set of early teams.

• Single-GPU serving
• vLLM runtime
• Bearer-token authentication