Alibaba's Qwen Team Releases Qwen3.8-Flash with Enhanced Long-Context and Multimodal Abilities

Alibaba's Qwen team has released Qwen3.8-Flash, a multimodal MoE model with 125B main parameters plus a 51B N-gram Embedding, activating 6B parameters per token. It natively supports 262,144 tokens of context, extendable to 1M tokens via YaRN.

The model upgrades in four areas: Attention, Residual, Embedding, and Optimization. It uses a hybrid GDN and QSA architecture to cut long-context attention costs, a Gated Residual mechanism with FP8 storage, and an N-gram Embedding that can offload to host memory to save GPU VRAM.

Training costs are about one-ninth of Qwen3.7-Plus, with stronger coding and office task performance. With 6B activated parameters, the Next-Base version tops 8 out of 14 benchmarks.

Qwen3.8-Flash will soon be available on the Qwen AI platform, priced at 1 yuan per million input tokens and 3 yuan per million output tokens. Model weights are now open-sourced, including an FP8 quantized version.