ESP32-S3 Runs AI Model with 28.9M Parameters

A Ukrainian developer named Slava S released ESP32-AI, a project that runs a 28.9 million-parameter AI model on Espressif's ESP32-S3 board. The board has 512KB SRAM, 8MB PSRAM, and 16MB flash, while previous attempts maxed out at 260,000 parameters.

The developer used Google Gemma's Per-Layer Embeddings technique. After 4-bit quantization, the model file is 14.9MB. By storing 25 million parameter embeddings in the 16MB flash, each token generation only reads a small amount of data.

Right now the model can't do chat or coding, but the developer suggested an offline coffee machine app that could generate content about coffee beans, grinding methods, ratios, and water temperature.