Fetching the paper…
Reading the bibliography…
We present GEM, a Generalizable Ego-vision Multimodal world model that predicts future frames using a reference frame, sparse features, human poses, and ego-trajectories.
Stereo vision and laser odometry for autonomous helicopters in gps-denied indoor environments
Markus Achtelik, Abraham Bachrach, Ruijie He, Samuel Prentice, and Nicholas Roy · 2009
Earlier work this paper cites.
Blind image quality evaluation using perception based features
Features Venkatanath, Praneeth, Maruthi Chandrasekhar Bh., Sumohana S. Channappayya, and Swarup S. Medasani · 2015
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter · 2017
Earlier work this paper cites.
End-to-end learning of driving models from large-scale video datasets
Huazhe Xu, Yang Gao, Fisher Yu, and Trevor Darrell · 2017
Earlier work this paper cites.
End-to-end learning of driving models with surround-view cameras and route planners
Simon Hecker, Dengxin Dai, and Luc Van Gool · 2018
Earlier work this paper cites.
Toward driving scene understanding: A dataset for learning driver behavior and causal reasoning
Vasili Ramanishka, Yi-Ting Chen, Teruhisa Misu, and Kate Saenko · 2018
Earlier work this paper cites.
Towards accurate generative models of video: A new metric & challenges
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly · 2018
Earlier work this paper cites.
Grounding human-to-vehicle advice for self-driving vehicles
Jinkyu Kim, Teruhisa Misu, Yi-Ting Chen, Ashish Tawari, and John Canny · 2019
Earlier work this paper cites.
Learning real-world robot policies by dreaming
Aj Piergiovanni, Alan Wu, and Michael S. Ryoo · 2019
Earlier work this paper cites.
Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving
Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark Campbell, and Kilian Q Weinberger · 2019
Earlier work this paper cites.
On the continuity of rotation representations in neural networks
Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li · 2019
Earlier work this paper cites.
Uncertainty-based traffic accident anticipation with spatio-temporal relational learning
Wentao Bao, Qi Yu, and Yu Kong · 2020
Earlier work this paper cites.
nuscenes: A multimodal dataset for autonomous driving
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom · 2020
Earlier work this paper cites.
Dream to control: Learning behaviors by latent imagination
Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi · 2020
Earlier work this paper cites.
Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He · 2020
Earlier work this paper cites.
Fourier features let networks learn high frequency functions in low dimensional domains
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng · 2020
Earlier work this paper cites.
Raft: Recurrent all-pairs field transforms for optical flow
Zachary Teed and Jia Deng · 2020
Earlier work this paper cites.
When, where, and what? a new dataset for anomaly detection in driving videos
Yu Yao, Xizi Wang, Mingze Xu, Zelin Pu, Ella Atkins, and David Crandall · 2020
Earlier work this paper cites.
Diffusion models beat gans on image synthesis, 2021
Prafulla Dhariwal and Alex Nichol · 2021
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
J. Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen · 2021
Earlier work this paper cites.
Drivegan: Towards a controllable high-quality neural simulation
Seung Wook Kim, Jonah Philion, Antonio Torralba, and Sanja Fidler · 2021
Earlier work this paper cites.
Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras
Zachary Teed and Jia Deng · 2021
Cited alongside, same era.
Videogpt: Video generation using vq-vae and transformers, 2021
Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas · 2021
Cited alongside, same era.
Generating long videos of dynamic scenes
Tim Brooks, Janne Hellsten, Miika Aittala, Ting-Chun Wang, Timo Aila, Jaakko Lehtinen, Ming-Yu Liu, Alexei Efros, and Tero Karras · 2022
Cited alongside, same era.
Ego4d: Around the world in 3,000 hours of egocentric video
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al · 2022
Cited alongside, same era.
Temporal difference learning for model predictive control
Nicklas A Hansen, Hao Su, and Xiaolong Wang · 2022
Cited alongside, same era.
Elucidating the design space of diffusion-based generative models
Learning interactive real-world simulators
Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Dale Schuurmans, and Pieter Abbeel · 2023
Later among the works it cites.
Occworld: Learning a 3d occupancy world model for autonomous driving
Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu · 2023
Later among the works it cites.
Limt: Language-informed multi-task visual world models
Elie Aljalbout, Nikolaos Sotirakis, Patrick van der Smagt, Maximilian Karl, and Nutan Chen · 2024
Closest in time.
Muvo: A multimodal world model with spatial representations for autonomous driving, 2024
Daniel Bogdoll, Yitian Yang, Tim Joseph, and J. Marius Zöllner · 2024
Closest in time.
Enabling visual composition and animation in unsupervised video generation
Aram Davtyan, Sepehr Sameni, Bjorn Ommer, and Paolo Favaro · 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine · 2022
Cited alongside, same era.
A self-supervised descriptor for image copy detection
Ed Pizzi, Sreya . Dutta Roy, Sugosh Nagavara Ravindra, Priya Goyal, and Matthijs Douze · 2022
Cited alongside, same era.
High-resolution image synthesis with latent diffusion models, 2022
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · 2022
Cited alongside, same era.
Laion-5b: An open large-scale dataset for training next generation image-text models
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev · 2022
Cited alongside, same era.
Make-a-video: Text-to-video generation without text-video data, 2022
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman · 2022
Cited alongside, same era.
Mcvd: Masked conditional video diffusion for prediction, generation, and interpolation, 2022
Vikram Voleti, Alexia Jolicoeur-Martineau, and Christopher Pal · 2022
Cited alongside, same era.
Videocrafter1: Open diffusion models for high-quality video generation, 2023
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan · 2023
Cited alongside, same era.
Closest in time.
Vista: A generalizable driving world model with high fidelity and versatile controllability
Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li · 2024
Closest in time.
Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al · 2024
Closest in time.
Animatediff: Animate your personalized text-to-image diffusion models without specific tuning, 2024
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai · 2024
Closest in time.
Hierarchical world models as visual whole-body humanoid controllers
Nicklas Hansen, Jyothir SV, Vlad Sobal, Yann LeCun, Xiaolong Wang, and Hao Su · 2024
Closest in time.
Large-scale actionless video pre-training via discrete diffusion for efficient policy learning, 2024
Haoran He, Chenjia Bai, Ling Pan, Weinan Zhang, Bin Zhao, and Xuelong Li · 2024
Closest in time.
Depthcrafter: Generating consistent long depth sequences for open-world videos
Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan · 2024
Closest in time.
Yolov11: An overview of the key architectural enhancements
Rahima Khanam and Muhammad Hussain · 2024
Closest in time.
Fit: Flexible vision transformer for diffusion model, 2024
Zeyu Lu, Zidong Wang, Di Huang, Chengyue Wu, Xihui Liu, Wanli Ouyang, and Lei Bai · 2024
Closest in time.
Reasoning with large language models, a survey
Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back · 2024
Closest in time.
GeoCalib: Single-image Calibration with Geometric Optimization
Alexander Veicht, Paul-Edouard Sarlin, Philipp Lindenberger, and Marc Pollefeys · 2024
Closest in time.
Pre-training contextualized world models with in-the-wild videos for reinforcement learning
Jialong Wu, Haoyu Ma, Chaoyi Deng, and Mingsheng Long · 2024
Closest in time.
Pandora: Towards general world model with natural language actions and video states
Jiannan Xiang, Guangyi Liu, Yi Gu, Qiyue Gao, Yuting Ning, Yuheng Zha, Zeyu Feng, Tianhua Tao, Shibo Hao, Yemin Shi, et al · 2024
Closest in time.
Progressive autoregressive video diffusion models
Desai Xie, Zhan Xu, Yicong Hong, Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou · 2024
Closest in time.
Drivedreamer-2: Llm-enhanced world models for diverse driving video generation
Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang · 2024
Closest in time.
Is sora a world simulator? a comprehensive survey on general world models and beyond, 2024
Zheng Zhu, Xiaofeng Wang, Wangbo Zhao, Chen Min, Nianchen Deng, Min Dou, Yuqi Wang, Botian Shi, Kai Wang, Chi Zhang, Yang You, Zhaoxiang Zhang, Dawei Zhao, Liang Xiao, Jian Zhao, Jiwen Lu, and Guan Huang · 2024
Closest in time.