ggml

ggml

Build AI that runs on your own devices with ggml’s computation components, quantization support and hardware backends.

Visit website

What is ggml?

Want to build an AI tool that runs on a computer without writing every model calculation yourself? ggml provides low-level computation components for machine learning. Projects such as llama.cpp and whisper.cpp use it to run language and speech models.

Developers supply model structures and weights, organize their computation with these components and connect the result to a chat, transcription or other application. The application supplies the user-facing features; ggml supplies part of the computation underneath.

It supports quantization—using more compact numerical representations for model parameters—which can reduce memory and storage requirements for some models. Together with different hardware backends, this lets developers test how a model fits their devices and compare speed, resource use and output quality.

FindAIGo reports, ggml suits teams building local AI applications or on-device inference, with some C/C++ and model-development knowledge required. The project using ggml determines which models run and what functions reach its users.

加载中 / Loading

Discussion

Loading comments…