Open-Source Strata Engine Runs a 125B-Parameter AI Model on 12GB GPUs

Huge AI models usually need data-center hardware, and limited video memory makes it hard to run them on consumer GPUs. Developer Niko1221 has open-sourced the Strata engine, which can run a quantized version of the Qwen3.8-Flash-Next model on consumer graphics cards with 12GB or more of VRAM. The model is a compressed version of a multimodal mixture-of-experts model from Alibaba's Qwen team, with 125 billion parameters.

Strata uses two tricks to save VRAM: it loads the whole MoE model into RAM and only keeps frequently used experts in VRAM, and it speeds up inference with speculative decoding from a lightweight model.

In tests, the 2-bit quantized version hit 94 tokens per second on an RTX 5070 (12GB), and 60 tokens per second on an AMD RX 9070 XT (16GB). It needs at least 32GB of RAM and 80GB of storage, works on Windows and Linux, and the code is on GitHub.