llama.cpp and ggml hit version 1.0.0 with clear API compatibility rules

llama.cpp is an open source AI inference project mainly written in C/C++. It lets large language models run on personal computers and servers, for both local and cloud use. Developer Georgi Gerganov announced that llama.cpp and its underlying engine ggml have now reached version 1.0.0.

The key update is clear compatibility rules. libllama works as the C/C++ inference library, and libggml serves as the underlying tensor library. Breaking API or ABI changes will now be marked with major version bumps, so developers can easily tell when an upgrade might need code changes.

The team is also using Hugging Face's HF Buckets and HF Jobs to grow its CI setup, with hardware support from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel for cross-platform testing. llama.cpp now supports over 150 model architecture variants and mainstream speculative decoding methods.

SponsoredDomoAI
AI Tools Mentioned

Comments

Loading comments…