Petals

Petals lets a person run a large open model that would never fit on their own hardware by holding only part of it and joining a network of other people serving the remaining layers, in the manner of a peer-to-peer file transfer. The project's site describes generating text with Llama 3.1 up to 405B, Mixtral 8x22B, Falcon above 40B and BLOOM at 176B from a consumer graphics card or a hosted notebook, and allows fine-tuning as well as sampling, with access to hidden states and custom execution paths rather than only a text-in-text-out interface. It is built for people who want a very large model without owning a data centre.

It suits experimentation, research on distributed inference and fine-tuning under a hobby budget, and it suits anyone studying how to split a model's layers across unreliable machines. It suits less well anything interactive that must answer on demand: the site's own figures are single-batch rates of a few tokens per second on the large models, which is enough to watch a model think and not enough to serve users. Anything that must be reproducible also sits badly on a network whose capacity is other people's machines.

Not verified: the project's shorter domain did not load at all this time — every fetch failed at the secure connection stage — so the description above comes from the project's other site and from the hosting service's repository interface. That repository reported about 10,618 stars, 651 forks and the MIT licence, with the last push dated 7 September 2024, which is a long time before this entry was written; nothing was read about whether the network is still operating or maintained at that level. The token-per-second figures are the project's own claims, and the network status display on the site failed to load when fetched here. No model was run and no adapter was trained, so the throughput and the fine-tuning path were not exercised.

Status note: as read on 2026-10-12, the project's repository at github.com/bigscience-workshop/petals is still served, but GitHub does not present it as archived, and its last push is dated 7 September 2024. The public network health page linked from petals.dev did not respond when fetched here, and the project's shorter domain still failed at the secure-connection stage. The description above therefore records what Petals was, not a service confirmed running today; the listing is kept and points at the project's own site, petals.dev.