400B Parameter Large Language Model Runs on iPhone 17 Pro via Open-Source Project Flash-MoE

Typically, a 400 billion parameter large language model requires at least 200GB of memory, making it hard to run on mobile phones. However, the open-source project Flash-MoE has successfully run on the iPhone 17 Pro, which only has 12GB of memory.

The project uses the device's SSD to stream data to the GPU and combines Mixture of Experts (MoE) technology, calling only part of the parameters for each word generated. However, the running speed is slow at 0.6 tokens per second, and it consumes a lot of battery, but this proves the possibility of running large models locally on mobile phones, which may be more practical after future optimizations.