Apple launches CoreAI engine for on-device AI

Apple has introduced a new CoreAI engine at WWDC 2026 to replace the CoreML framework. CoreML was mainly for small tasks like image classification, but CoreAI is designed for on-device AI and edge computing, focusing on running large language models locally. It supports more flexible model formats and uses more memory.

Early tests show CoreAI is 2.47 times faster than MLX when running a small model on M4 Mac, and 1.6 times faster on iPhone 17 Pro. But with bigger models, the speed advantage drops to just 5%. In long-running tasks, the iPhone 17 Pro's GPU slows down due to heat, but CoreML with Apple's Neural Engine keeps performance stable.

Compared to others, Google's LiteRT-LM runs Gemma on iPhone 17 Pro at 55.4 tokens per second, using only 641 MB of RAM, while Apple's MLX uses 2900 MB.