MILab is a University of Washington research lab directed by Professor Zhi-Qi Cheng at UW Tacoma.
We develop multimodal AI and world models that connect perception, prediction, and planning in the physical world.
We study how intelligent systems can understand environments from incomplete observations, anticipate change, and reason about the consequences of actions. Our methods combine 3D geometry, semantic understanding, and generative modeling to connect spatial representations, memory, and action planning.
Our work spans visual navigation, human–AI interaction, and transportation infrastructure mapping, monitoring, and management. We develop models, benchmarks, and research systems, alongside methods for efficient inference and adaptation, to bring multimodal intelligence into changing physical environments.
Our research connects multimodal understanding with world models and intelligent action, from reasoning about people and events to interacting with environments and mapping infrastructure.
We connect visual observations, language, and structured knowledge to reason about activities, events, and their relationships in context. This supports research in grounded question answering, event understanding, and human–AI interaction.
Explore multimodal understandingTwo USDOT-funded research projects. Support from the U.S. Department of Transportation through PacTrans advances our work in 3D urban scene reconstruction and transportation asset mapping.
NeurIPS 2026 Oral. Language-Conditioned World Modeling for Visual Navigation.
ICLR 2026 Oral. Lossless Hierarchical Speculative Decoding.
ECCV 2026. UniWM integrates memory, planning, and foresight for visual navigation.
IROS 2026. HA-VLN 2.0 benchmarks navigation with dynamic human interactions.
Two papers at ACL 2026. Sign-Language Datasets at Scale (main conference) and GoViG (Findings).
NeurIPS 2026 · Oral
Language-grounded future prediction and action generation from an initial visual observation.
ECCV 2026
A memory-augmented world model that grounds navigation decisions in imagined visual outcomes.
IROS 2026
An open benchmark for navigation around multiple moving people in discrete and continuous environments.
ICLR 2026 · Oral
Hierarchical verification accelerates language-model inference while preserving the target model’s output distribution.
Findings of ACL 2026
Navigation instructions generated from initial and goal views through multimodal reasoning.
NeurIPS 2025 · Oral
Regularization that addresses representation collapse and preserves variation within classes.
NeurIPS 2024 · Datasets and Benchmarks · Spotlight
A simulator, dataset, and evaluation framework for navigation around dynamic human activities.
NeurIPS 2024
Instruction-tuned emotion recognition and reasoning grounded in video, speech, and text.
Professor Cheng’s earlier work at Carnegie Mellon University connects research methods with integrated systems and public-interest applications.
Core technical leadership in CMU’s DARPA KAIROS and KAIROS Plus systems (2019–2024), integrating events across video, language, and knowledge sources.
Research leadership & collaborationsLed video-based traffic activity analysis through CMU’s USDOT Mobility21 program, documented in a 2023 research report.
USDOT project reportContributed technical analysis to The Washington Post’s reporting recognized with the 2022 Pulitzer Prize for Public Service, alongside crowd analysis of the Astroworld tragedy.
Media collaborationsProfessor Cheng recruits Ph.D. and master’s students primarily through UW’s Computer Science & Systems (CSS) programs. He also supervises and co-advises research by students in CSE, ECE, and other UW programs and serves on doctoral committees.
Prospective and current students are welcome to contact Professor Cheng with a CV and research interests.
