JD.com open-sources real-time video AI model JoyAI-VL-Interaction

JD.com has open-sourced JoyAI-VL-Interaction, the world's first fully open-source real-time video vision-language interaction model. It gets day-0 native support from vLLM-Omni. This model lets AI move from 'ask and answer' to 'watch and talk', helping developers build real-time AI assistants that observe, decide, and respond instantly.

Compared to traditional models, JoyAI-VL-Interaction has three key breakthroughs: it proactively judges instead of passively answering, responds in real-time instead of post-summarizing, and can delegate tasks to other AI agents while staying focused on the video stream. It supports cameras, live streams, and surveillance feeds, along with voice input/output, visual interfaces, long-term memory, and vLLM deployment—all modules are replaceable.

In 58 blind tests, JoyAI-VL-Interaction beat Doubao's video assistant with 77.6% win rate, and Gemini's video assistant with 87.9% win rate. It can be used for security monitoring, elderly/child care, live-stream commentary, e-commerce guidance, operation instructions, AI glasses, and accessibility assistance.