DeepInfra

DeepInfra is an AI inference cloud: an OpenAI-compatible API sitting in front of hundreds of open-source models — language, vision and OCR, embeddings and rerankers, image and video, speech — with serverless and dedicated deployment modes. The billing pitch is explicitly anti-overhead: you pay per token, with no idle GPU time, no minimums and no seat fees, so a hobby project and a spike-heavy production app both start at zero. When shared serverless is not enough, the same catalog can be lifted onto private deployments or dedicated instances on enterprise accelerators (A100 through the current Blackwell generation) with autoscaling, or you can rent the GPUs and run your own stack.

The judgement: on popular open models DeepInfra routinely prices below the big managed clouds, and per-token serverless means capacity risk sits with them, not you — until demand spikes queue the shared endpoint and dedicated capacity appears on the invoice. There is no self-hosted tier; the alternative is open weights plus vLLM on rented GPUs, which trades their margin for your pager. Retention and residency terms are not spelled out on the public pages read on 2026-09-20, so put traffic through a docs review before regulated data flows. Cheapest competent token across the widest model zoo — a rational default; guaranteed latency at scale — you will pay for isolation somewhere.