Open-source Garlic LLM runs a 9B AI model offline on a jailbroken PS4

Garlic LLM, an open-source project, lets a jailbroken PlayStation 4 run a 9B-parameter local AI model offline, with generation speeds up to 43 tokens per second. The developer bypasses the console's game graphics drivers and sends low-level instructions directly to its AMD GCN GPU, turning the console into an AI inference machine.

The project uses Alibaba's Qwen3.5-9B as its base model and compresses the weights to 3-bit or 4-bit format, keeping the total around 4.5 GiB. It also uses a decoding method that checks several tokens ahead to improve efficiency. In tests, normal text generation runs at about 31 to 39 tokens per second, while code generation reaches up to 43 tokens per second, with a context window of about 16,000 tokens. It supports voice input and conversation, but voice mode lowers text generation to about 22 tokens per second.

SponsoredDomoAI
AI Tools Mentioned

Comments

Loading comments…