llama.cpp
ggml (community)
A C/C++ inference engine for supported GGUF models, with CLI, server and hardware-specific backends for local or separately managed deployment.
- Rating
- 4.7★
- Pricing
- Free
Editorial verdict
Check the exact model's rights and backend support, then measure memory, quality and operating cost for your deployment.
Strengths
- Source-level control and documented CLI/server paths
- Multiple backend and quantization choices
Key features
- GGUF model execution and quantization options
- CLI and OpenAI-compatible server path
- CPU/GPU backends with specific requirements
- MIT-licensed engine code
Best for
- Evaluate a permitted GGUF on a selected machine
- Embed or serve an engine under controlled access