OpenLLM

OpenLLM is BentoML's Apache-2.0 CLI over its serving stack: pip install openllm, then openllm serve llama3.2:1b raises an OpenAI-compatible endpoint on localhost:3000 with a chat UI at /chat. It stores no weights — models come from Hugging Face, gated ones need an HF_TOKEN — and vLLM is the backend. Model choice comes from a catalog: the default bentoml/openllm-models repository plus public ones added by openllm repo add. Each entry pins a model tag to a required GPU shape, which is the useful part: deepseek r1-671b at 80Gx16, gemma2:2b at 12G. openllm containerize produces a Bento image; openllm deploy ships it to BentoCloud.

The judgement is about timing. The last tagged release is v0.6.30 from 2025-04-21, the README's supported-model table is still the 2024-2025 cohort — Llama 3.3, Qwen2.5, Phi3, DeepSeek R1 — and open issues number in the dozens, so this is a thin, stable wrapper rather than an advancing product. Want the ergonomics and you are largely adopting BentoML underneath, so BentoML or vLLM directly is the better long-term bet; for one-command local models Ollama holds the mindshare. It still earns a place as the quickest route to a reproducible containerised endpoint for a model the catalog already knows. There is no hosted OpenLLM and no price list: the cost is GPU time you rent yourself, plus reliance on someone else's pinned recipe, and gated weights still need your own Hugging Face approval.