Platform — ModelCloud

01 / Inference

Serve any model behind one endpoint

Bring open-weights or your own fine-tune. ModelCloud pulls the weights, applies quantization, and stands up an OpenAI-compatible endpoint you can call from any existing SDK.

Autoscaling runs replicas from zero to whatever traffic demands, so you never pay for idle GPUs. Warm pools keep a ready container around so the first request after a scale-up doesn't pay the full cold-start penalty.

OpenAI-compatible /v1/chat/completions and /v1/completions
FP8, INT8, and INT4 quantization matched to your GPU
Continuous batching and prefix caching for shared-prompt traffic
Multi-GPU tensor and pipeline parallelism for models that exceed one accelerator
Autoscale to zero, with configurable cooldowns

deployment · qwen2.5-7b

runtimevllm

gpuh100 · 80 GB

quantizationfp8

replicas0 – 8

warm poolon

cold startoverlapped pull

endpointlive

02 / Training BETA

Fine-tune and train on managed fleets

Run distributed training and fine-tuning jobs across many GPUs without standing up a cluster. Spot capacity keeps batch work cheap, and checkpointing means an interruption is a pause, not a loss.

Jobs resume deterministically from the last checkpoint, so a preempted spot instance costs you minutes, not a full re-run.

Distributed data-parallel and model-parallel training
Spot capacity with automatic rescheduling on interruption
Deterministic resume from any checkpoint
Dataset caching so repeated epochs don't re-fetch
Resulting weights stored in your object storage, not ours

job · fine-tune

gpus8 × a100-80gb

capacityspot

checkpointevery 500 steps

resumedeterministic

interruptionauto-reschedule

03 / Observability

Know where every token and every dollar goes

Per-model latency, throughput, GPU utilization, queue depth, and token accounting — surfaced in a dashboard or streamed to the stack you already run.

The data is yours. Export traces over OTLP, scrape a Prometheus endpoint, or push metrics into your existing backend. We show you the numbers; we don't hide them in a walled garden.

Time-to-first-token and inter-token latency per request
Token and cost accounting per model, workspace, and deployment
GPU utilization, memory, and KV-cache hit rate per replica
OTLP export to Datadog, Grafana, or Prometheus

trace · req_7f2c

ttft142 ms

inter-token31 ms

tokens412

gpu_util78%

cache_hit0.93

queued0 ms

spanprefill 118ms / decode 294ms

04 / Fleet

Run dedicated, compliance-ready capacity

For regulated and high-throughput workloads, run a dedicated or reserved fleet with private connectivity, quotas, and role-based access.

Peer a VPC so inference traffic never touches the public internet, hold workloads to a compliance posture (SOC 2, HIPAA, GDPR), and control who can deploy, scale, or read logs.

VPC peering and private connectivity
Quotas, role-based access, and audit logs
Compliance posture: SOC 2, HIPAA, GDPR
Reserved or dedicated GPU capacity with committed-use pricing

fleet · enterprise

connectivityvpc peering

capacityreserved / dedicated

complianceSOC 2 · HIPAA · GDPR

accessrbac + audit logs

sla99.95%

One GPU fleet. Four ways to run models.

Serve any model behind one endpoint

Fine-tune and train on managed fleets

Know where every token and every dollar goes

Run dedicated, compliance-ready capacity

Run your model, not the infrastructure.