2021

Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset

Ettinger, Scott, Cheng, Shuyang, Caine, Benjamin et al.

Understand

As autonomous driving systems mature, motion forecasting has received increasing attention as a critical requirement for planning.

Of particular importance are interactive situations such as merges, unprotected turns, etc., where predicting individual object motion is not sufficient.
Joint predictions of multiple objects are required for effective route planning.
There has been a critical need for high-quality motion data that is rich in both interactions and annotation to develop motion planning models.

Built on

Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Crowds by example
Alon Lerner, Yiorgos Chrysanthou, and Dani Lischinski · 2007
Earlier work this paper cites.
The pascal visual object classes (voc) challenge
M. Everingham, L. Gool, C. K. Williams, J. Winn, and Andrew Zisserman · 2009
Earlier work this paper cites.
You’ll never walk alone: Modeling social behavior for multi-target tracking
Stefano Pellegrini, Andreas Ess, Konrad Schindler, and Luc Van Gool · 2009
Earlier work this paper cites.
Stable multi-target tracking in real-time surveillance video
Ben Benfold and Ian Reid · 2011
Earlier work this paper cites.
Vision meets robotics: The kitti dataset
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun · 2013
Earlier work this paper cites.
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick · 2014
Earlier work this paper cites.
Social lstm: Human trajectory prediction in crowded spaces
Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese · 2016
Earlier work this paper cites.
Learning social etiquette: Human trajectory understanding in crowded scenes
Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese · 2016
Earlier work this paper cites.
A critical evaluation of the next generation simulation (ngsim) vehicle trajectory dataset
Benjamin Coifman and Lizhe Li · 2017
Earlier work this paper cites.
Desire: Distant future prediction in dynamic scenes with interacting agents
Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher B Choy, Philip HS Torr, and Manmohan Chandraker · 2017
Earlier work this paper cites.
Intentnet: Learning to predict intention from raw sensor data
Sergio Casas, Wenjie Luo, and Raquel Urtasun · 2018
Earlier work this paper cites.
R2p2: A reparameterized pushforward policy for diverse, precise generative path forecasting
Nicholas Rhinehart, Kris M Kitani, and Paul Vernaza · 2018
Earlier work this paper cites.
Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction
Original
Yuning Chai, Benjamin Sapp, Mayank Bansal, and Dragomir Anguelov · 2019
Earlier work this paper cites.

Similar

Argoverse: 3d tracking and forecasting with rich maps
Ming-Fang Chang, John Lambert, Patsorn Sangkloy, Jagjeet Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, et al · 2019
Cited alongside, same era.
Multimodal trajectory predictions for autonomous driving using deep convolutional networks
Henggang Cui, Vladan Radosavljevic, Fang-Chieh Chou, Tsung-Han Lin, Thi Nguyen, Tzu-Kuo Huang, Jeff Schneider, and Nemanja Djuric · 2019
Cited alongside, same era.
Rules of the road: Predicting driving behavior with a convolutional model of semantic interactions
Joey Hong, Benjamin Sapp, and James Philbin · 2019
Cited alongside, same era.
Lyft level 5 perception dataset 2020
R. Kesten, M. Usman, J. Houston, T. Pandya, K. Nadhamuni, A. Ferreira, M. Yuan, B. Low, A. Jain, P. Ondruska, S. Omari, S. Shah, A. Kulkarni, A. Kazakova, C. Tao, L. Platinsky, W. Jiang, and V. Shet · 2019
Cited alongside, same era.
CoverNet: Multimodal behavior prediction using trajectory sets
Original
Tung Phan-Minh, Elena Corina Grigore, Freddy A Boulton, Oscar Beijbom, and Eric M Wolff · 2019
Cited alongside, same era.
Precog: Prediction conditioned on goals in visual multi-agent settings
Nicholas Rhinehart, Rowan McAllister, Kris Kitani, and Sergey Levine · 2019
Cited alongside, same era.
Multiple futures prediction
Charlie Tang and Russ R Salakhutdinov · 2019
Cited alongside, same era.
Analyzing the variety loss in the context of probabilistic trajectory prediction
Luca Anthony Thiede and Pratik Prabhanjan Brahma · 2019
Cited alongside, same era.
INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps
Original
Wei Zhan, Liting Sun, Di Wang, Haojie Shi, Aubrey Clausse, Maximilian Naumann, Julius Kümmerle, Hendrik Königshof, Christoph Stiller, Arnaud de La Fortelle, and Masayoshi Tomizuka · 2019
Cited alongside, same era.
Plop: Probabilistic polynomial objects trajectory planning for autonomous driving
Original
Thibault Buhet, Emilie Wirbel, and Xavier Perrotton · 2020
Cited alongside, same era.
nuscenes: A multimodal dataset for autonomous driving
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom · 2020
Cited alongside, same era.
Spagnn: Spatially-aware graph neural networks for relational behavior forecasting from sensor data
Sergio Casas, Cole Gulino, Renjie Liao, and Raquel Urtasun · 2020
Cited alongside, same era.

Then

Deep kinematic models for kinematically feasible vehicle trajectory predictions
Henggang Cui, Thi Nguyen, Fang-Chieh Chou, Tsung-Han Lin, Jeff Schneider, David Bradley, and Nemanja Djuric · 2020
Later among the works it cites.
VectorNet: Encoding hd maps and agent dynamics from vectorized representation
Jiyang Gao, Chen Sun, Hang Zhao, Yi Shen, Dragomir Anguelov, Congcong Li, and Cordelia Schmid · 2020
Later among the works it cites.
One thousand and one hours: Self-driving motion prediction dataset
Original
John Houston, Guido Zuidhof, Luca Bergamini, Yawei Ye, Ashesh Jain, Sammy Omari, Vladimir Iglovikov, and Peter Ondruska · 2020
Later among the works it cites.
Learning lane graph representations for motion forecasting
Original
Ming Liang, Bin Yang, Rui Hu, Yun Chen, Renjie Liao, Song Feng, and Raquel Urtasun · 2020
Later among the works it cites.
Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data
Original
Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, and Marco Pavone · 2020
Later among the works it cites.
Scalability in perception for autonomous driving: Waymo open dataset
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al · 2020
Later among the works it cites.
Bdd100k: A diverse driving dataset for heterogeneous multitask learning
Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell · 2020
Later among the works it cites.
Tnt: Target-driven trajectory prediction
Original
Hang Zhao, Jiyang Gao, Tian Lan, Chen Sun, Benjamin Sapp, Balakrishnan Varadarajan, Yue Shen, Yi Shen, Yuning Chai, Cordelia Schmid, et al · 2020
Later among the works it cites.
End-to-end multi-view fusion for 3d object detection in lidar point clouds
Yin Zhou, Pei Sun, Yu Zhang, Dragomir Anguelov, Jiyang Gao, Tom Ouyang, James Guo, Jiquan Ngiam, and Vijay Vasudevan · 2020
Later among the works it cites.
Offboard 3d object detection from point cloud sequences, 2021
Charles R. Qi, Yin Zhou, Mahyar Najibi, Pei Sun, Khoa Vo, Boyang Deng, and Dragomir Anguelov · 2021
Closest in time.
Trafficsim: Learning to simulate realistic multi-agent behaviors
Simon Suo, Sebastian Regalado, Sergio Casas, and Raquel Urtasun · 2021
Closest in time.
Identifying driver interactions via conditional behavior prediction
Ekaterina Tolstaya, Reza Mahjourian, Carlton Downey, Balakrishnan Vadarajan, Benjamin Sapp, and Dragomir Anguelov · 2021
Closest in time.
Auto4d: Learning to label 4d objects from sequential point clouds, 2021
Bin Yang, Min Bai, Ming Liang, Wenyuan Zeng, and Raquel Urtasun · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…