Ray

Ray is an Apache-2.0 distributed compute engine: a handful of primitives — tasks, actors, objects — that turn ordinary Python functions and classes into work scheduled across machines, with libraries on top: Ray Data, Train, Tune, RLlib and Serve for online inference. Serve is framework-agnostic — PyTorch, TensorFlow, scikit-learn or plain Python logic. Distinctive features: fractional GPUs, so several models share one card; model composition, where a pipeline of models and code is one Python program rather than several wired services; and LLM handling — response streaming, dynamic request batching, multi-GPU serving. KubeRay, a separate Apache-2.0 operator, runs clusters on Kubernetes. Release 2.58.0 shipped 2026-08-23.

Ray is infrastructure you operate, not a service you buy: it deletes the queue-and-thread layer and hands you a cluster in exchange. The laptop-to-thousands-of-GPUs claim is real, but assumes you will reason about object-store memory and scheduling. Read the security docs before a multi-team rollout: Ray serialises with cloudpickle, expects trusted code on an isolated network, does not isolate jobs inside a single cluster, and ships no developer access controls. Token authentication arrived in 2.52.0 but is documented as defence in depth, not a substitute for isolation. Anyscale, from Ray's own makers, is the managed answer, advertising a $100 starting credit and selling the governance, dashboard hardening and support the OSS build leaves to you.