ModelCloud is the serving layer for your AI models. Bring your own weights or an open-weight model — we handle GPU inference, autoscaling, and observability, so your team ships product instead of babysitting infrastructure. No vendor lock-in, ever.
We're not a reseller of someone else's models. ModelCloud is the infrastructure your models run on — the runtime, the scaling, the telemetry — while you keep full ownership of your weights and your data.
Point at a Hugging Face repo, a container, or a private registry. We pull the weights, quantize, and prep the runtime — vLLM, TensorRT-LLM, or SGLang.
modelcloud deploySet a min and max replica range. ModelCloud scales on request volume or queue depth, and scales to zero when traffic stops — you pay only for what you run.
--autoscale 1..8Token-level latency, GPU utilization, queue depth, and traces per request. Export to your own metrics stack — the data is yours, not locked in ours.
modelcloud logs -fvLLM, TensorRT-LLM, SGLang, and llama.cpp for CPU — pick the engine that fits your latency and throughput targets. We tune the serving parameters so you don't have to.
Weights stay warm on a preloaded cache, so scale-from-zero doesn't mean a multi-minute stall. Warm containers resume in under a second; cold pulls are optimized and pipelined.
Tensor-parallel and pipeline-parallel sharding for large open models that don't fit on a single accelerator. One flag, not a config labyrinth.
Weights and inference traffic stay in your account. Deploy to your own VPC, bring your own cloud, or run fully managed. There's no shared multi-tenant wall behind your requests.
One OpenAI-compatible endpoint for every model you deploy, with streaming, function calling, and structured output where the runtime supports it.
/v1/chat/completions compatibility# stream from your own model import modelcloud client = modelcloud.Client() resp = client.chat.completions.create( model="llama-3-8b", messages=[{"role": "user", "content": "Summarize this log"}], stream=True, ) for chunk in resp: print(chunk.choices[0].delta.content)
"We moved our fine-tuned code model off a hand-rolled k8s setup. Cold starts went from 'go get coffee' to under a second, and we deleted a whole repo of YAML."
"The observability is what sold me — token-level traces I could export straight into our existing Grafana. I finally know exactly where latency goes."
"Scale-to-zero changed our compute bill without me touching a single model. We serve a dozen open-weight models and pay for what actually runs."
Preloaded weight cache and pipelined pulls bring scale-from-zero resumes to sub-second for warm containers.
platform v2.3.0 · 2026-07-02Scale on queue depth and GPU saturation, not just request rate — plus configurable scale-to-zero windows down to 30 seconds.
autoscaling v2.2.0 · 2026-05-21Per-request traces with time-to-first-token and per-token generation metrics, exportable via OTLP.
observabilityDeploy your first model in minutes — open-weight or your own fine-tune. The sandbox tier is free, no credit card.
Read the quickstart →