Xiaomi open-sources MiDashengLM-Spatial, a model that understands where sounds come from

Xiaomi has open-sourced a new audio model called MiDashengLM-Spatial. The model weights and code are fully public under the Apache 2.0 license. Over the past two years, Xiaomi has also released the audio model Xiaomi Dasheng and MiDashengLM-7B. These tools are already used in more than 30 real cases, like smart homes and car cockpits.

Built on MiDashengLM-7B, the new model does more than understand what a sound is. It can also tell where the sound comes from and how it spreads in space, like giving the model a pair of ears.

Xiaomi says this is the industry's first open-source spatial audio model of its kind. It will soon be used in Xiaomi smart homes and car cockpits to make the human-car-home experience smoother.