Awesome World Models for Robotics
This repository provides a curated list of papers for World Models for General Video Generation, Embodied AI, and Autonomous Driving. Template from Awesome-LLM-Robotics and Awesome-World-Model
Contributions are welcome! Please feel free to submit pull requests or reach out via email to add papers!
If you find this repository useful, please consider citing and giving this list a star ⭐. Feel free to share it with others!
Maintainers can use the arXiv candidate pipeline to discover recent papers for review.
Overview
Foundation paper of World Model
Blog or Technical Report
ST-WAM, ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts. [Paper] [Website] [Code]NVIDIA OmniDreams, NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation. [Paper]Cosmos 3, Cosmos 3: Omnimodal World Models for Physical AI. [Paper] [Website]X Square Robot, WALL-WM: Carving World Action Modeling at the Event Joints. [Paper] [Website]GE-Sim 2.0, GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation. [Paper] [Website]Xiaomi EV World Model, Xiaomi EV World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving. [Paper] [Website]Coowa, The DAWN of World-Action Interactive Models. [Paper]- , Being-H0.7: A Latent World-Action Model from Egocentric Videos. []
Survey
- "World-Action Models for Robot Learning and Control: A Survey",
arxiv 2026.09. [Paper]
- "Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Models",
arxiv 2026.09. [Paper]
- "Surgical Video Generation From Diffusion to World Models: A Survey",
arxiv 2026.08. [Paper]
- "From Generation to Simulation: How Far Are World Models from Being True Simulators?",
arxiv 2026.08. [Paper]
- "Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation",
arxiv 2026.07. [Paper]
- "From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence",
arxiv 2026.07. [Paper]
- "From World Models to World Action Models: A Concise Tutorial for Robotics",
arxiv 2026.07. [Paper]
- "Autonomous Video Generation with Counterfactual Controllability for Self-Evolving World Models",
arxiv 2026.06. [Paper]
- "Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies",
arxiv 2026.06. [Paper]
- "Bridging the Agent-World Gap: Text World Models for LLM-based Agents",
arxiv 2026.06. [Paper]
Datasets & Benchmarks & Evaluation
- WorldMemBench: "WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models",
arxiv 2026.10. [Paper] [Website] [Code]
- NavSafe-∞: "NavSafe-∞: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments",
arxiv 2026.09. [Paper] [Website] [Code]
- VeriPhy: "VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement",
arxiv 2026.09. [Paper]
- SolarWM: "SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models",
arxiv 2026.09. [Paper]
- PAWBench: "PAWBench: How Far Are We from Probabilistically Aligned World Modeling?",
arxiv 2026.08. [Paper]
- R2M-Bench: "R2M-Bench: Evaluating Revisit Memory via Relative Consistency in Interactive Video World Models",
arxiv 2026.08. [Paper]
- MiniMax H3 1K prompt dataset: curated 1K text-to-video prompts with 3-field structure anatomy, 10 reusable prompts, and an H3 vs. peer model comparison. [Website] [Code] []
General World Models
- "AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video",
arxiv 2026.09. [Paper]
- "World in World: Explore the World with World Models",
arxiv 2026.09. [Paper]
- "Programmable World Model",
arxiv 2026.09. [Paper]
- Arti-JEPA: "Arti-JEPA: Adapting Video World Model to Real-Time MRI of the Vocal Tract for Speech-Production Analysis",
arxiv 2026.09. [Paper]
- Valerant: "Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration",
arxiv 2026.09. [Paper]
- VeriScene: "VeriScene: Reconstructing Crime Scenes from Legal Evidence via World-Model Agent",
arxiv 2026.09. [Paper]
- ActionSplice: "ActionSplice: In-Flight Action Editing for Interactive World Models",
arxiv 2026.09. [Paper]
- "A radiographic world model for clinical reasoning and evidence generation",
arxiv 2026.09. [Paper]
- "Diagnosing and Dynamically Filtering Occupancy World Models for Active Mapping",
arxiv 2026.09. [Paper]
- WorldSculpt: "WorldSculpt: Generating Compositional Worlds from Grounded Videos",
arxiv 2026.09. [Paper]
World Models for Embodied AI
- UNITAS: "UNITAS: A 3D-Native World Action Model for Embodied Manipulation",
arxiv 2026.10. [Paper] [Website] [Code]
- EMPIRIC: "EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning",
arxiv 2026.09. [Paper] [Website] [Code]
- LeWAM: "Latent evolving World Action Model",
arxiv 2026.09. [Paper] [Code]
- "Modality-Autoregressive World-Action Models",
arxiv 2026.09. [Paper]
- XPACE: "XPACE: Joint World and Action Modeling from Heterogeneous Experience",
arxiv 2026.09. [Paper]
- "World Models for Embodied Intelligence: From Plausible to Controllable to Actionable",
arxiv 2026.09. [Paper]
- WholeBodyWAM: "WholeBodyWAM: Generalizing Pre-trained World-Action Priors to Humanoid Loco-Manipulation via WBC-Grounded Coordination",
arxiv 2026.09. [Paper]
- WLA$^3$: "WLA$^3$: World Latent Action Modeling for Semantics, Dynamics, and Kinematics", . []
World Models for VLA
- WISE: "WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models",
arxiv 2026.09. [Paper]
- "CheckVLA: Execution-Time Verification with Action-Conditioned World Model for Long-Horizon Mobile Manipulation",
arxiv 2026.07. [Paper]
- World Pilot: "World Pilot: Steering Vision-Language-Action Models with World-Action Priors",
arxiv 2026.06. [Paper]
- "Vision-Language-Action Models Meet World Models: Embodied Agentic AI for Low-Altitude Wireless Networks",
arxiv 2026.06. [Paper]
- "World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis",
arxiv 2026.06. [Paper]
- PiL-World: "PiL-World: A Chunk-Wise World Model for VLA Policy-in-the-Loop Evaluation",
arxiv 2026.06. [Paper]
- "Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation",
arxiv 2026.06. [Paper]
- Pre-VLA: "Pre-VLA: Preemptive Runtime Verification for Reliable Vision-Language-Action and World-Model Rollouts",
arxiv 2026.05. [Paper]
- "Reinforcing VLAs in Task-Agnostic World Models",
arxiv 2026.05. [Paper]
- DIAL: "DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA", . [] []
World Models for Visual Understanding
- Spatial-Interactor: "Spatial-Interactor: Learning Spatial Reasoning through Interaction with the Observable Physical World",
arxiv 2026.09. [Paper] [Website] [Code]
- "Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators",
arxiv 2026.06. [Paper]
- "Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning",
arxiv 2026.05. [Paper]
- GeoWorld-VLM: "GeoWorld-VLM: Geometry from World Models for Vision-Language Models",
arxiv 2026.05. [Paper]
- World2VLM: "World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning",
arxiv 2026.03. [Paper] [Website] [Code]
- DILLO: "Describe-Then-Act: Proactive Agent Steering via Distilled Language-Action World Models",
arxiv 2026.03. [Paper]
- WorldVLM: "WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning",
arxiv 2026.03. [Paper]
- "When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoning", . [] []
World Models for Autonomous Driving
- PV-WM: "PV-WM: A Heterogeneous Micro-Macro World Model for Articulated Pedestrian-Vehicle Co-Rollout",
arxiv 2026.09. [Paper]
- "Rethinking World Models for Safety-Critical Embodied Systems",
arxiv 2026.09. [Paper]
- SV-WAM: "SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving",
arxiv 2026.09. [Paper]
- Drive-HWM: "Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving",
arxiv 2026.09. [Paper]
- "Long-Horizon Consistent and Interaction-Aware World Models for Multi-Style End-to-End Driving",
arxiv 2026.09. [Paper]
- 4DGS-WAM: "4DGS-WAM: Bridging Past and Future with an Object-Centric World Action Model based on 4D Gaussian Splatting",
arxiv 2026.08. [Paper]
- GeoWAM: "GeoWAM: Visual Geometry World Action Models for Autonomous Driving",
arxiv 2026.08. [Paper]
- BehaviorWorldGen: "BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generation",
arxiv 2026.08. [Paper]
- WA-JEPA: "WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving",
arxiv 2026.08. [Paper]
- DriveWAM: "DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving",
arxiv 2026.05. [Paper]
- X-Foresight: "X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling",
arxiv 2026.05. [Paper]
- SparseWorld: "SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation",
arxiv 2026.05. [Paper] [Website]
- "Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving",
IEEE ITSC 2026. [Paper]
- HEAT: "HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Models",
arxiv 2026.05. [Paper]
- EponaV2: "EponaV2: Driving World Model with Comprehensive Future Reasoning",
arxiv 2026.05. [Paper]
- HorizonDrive: "HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation",
arxiv 2026.05. [Paper]
- DeepSight: "DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving",
ICML 2026. [Paper] [Code]
Citation
If you find this repository useful, please consider citing this list:
@misc{leo2024worldmodelspaperslist,
title = {Awesome-World-Models},
author = {Leo Fan},
journal = {GitHub repository},
url = {https://github.com/leofan90/Awesome-World-Models},
year = {2024},
}
