2017

Imitating Driver Behavior with Generative Adversarial Networks

Kuefler, Alex, Morton, Jeremy, Wheeler, Tim et al.

Understand

The ability to accurately predict and simulate human driving behavior is critical for the development of intelligent transportation systems.

Traditional modeling methods have employed simple parametric models and behavioral cloning.
This paper adopts a method for overcoming the problem of cascading errors inherent in prior approaches, resulting in realistic behavior that is robust to trajectory perturbations.
We extend Generative Adversarial Imitation Learning to the training of recurrent policies, and we demonstrate that our model outperforms rule-based controllers and maximum likelihood models in realistic highway simulations.

Built on

“A new approach to linear filtering and prediction problems”
Rudolph˜Emil Kalman · 1960
Earlier work this paper cites.
“A program for simulating the dispersion of platoons of road traffic”
Philip˜A Seddon · 1972
Earlier work this paper cites.
“Estimating the dimension of a model”
Gideon Schwarz · 1978
Earlier work this paper cites.
“A behavioural car-following model for computer simulation”
Peter˜G Gipps · 1981
Earlier work this paper cites.
“ALVINN: An autonomous land vehicle in a neural network”, 1989
Dean˜A Pomerleau · 1989
Earlier work this paper cites.
“Reinforcement Learning: An Introduction”
Richard˜S Sutton and Andrew˜G Barto · 1998
Earlier work this paper cites.
“Congested traffic states in empirical observations and microscopic simulations”
Martin Treiber, Ansgar Hennecke and Dirk Helbing · 2000
Earlier work this paper cites.
“The Elements of Statistical Learning”
Jerome Friedman, Trevor Hastie and Robert Tibshirani · 2001
Earlier work this paper cites.
“Develop a car-following model using data collected by ”five-wheel system””
Jia Hongfei, Juan Zhicai and Ni Anning · 2003
Earlier work this paper cites.
“Apprenticeship learning via inverse reinforcement learning”
Pieter Abbeel and Andrew˜Y Ng · 2004
Earlier work this paper cites.
“US Highway 80 Dataset”, 2006
James Colyar and John Halkias · 2006
Earlier work this paper cites.
“General lane-changing model MOBIL for car-following models”
Arne Kesting, Martin Treiber and Dirk Helbing · 2007
Earlier work this paper cites.
“A game-theoretic approach to apprenticeship learning”
Umar Syed and Robert˜E. Schapire · 2007
Earlier work this paper cites.
“US Highway 101 Dataset”, 2007
James Colyar and John Halkias · 2007
Earlier work this paper cites.

Similar

“Neural Agent Car-Following Models”
S. Panwai and H. Dia · 2007
Cited alongside, same era.
“Signal Modelling and Hidden Markov Models for Driving Manoeuvre Recognition and Driver Fault Diagnosis in an Urban Road Scenario”
P. Boyraz, M. Acar and D. Kerr · 2007
Cited alongside, same era.
“Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations”
Honglak Lee, Roger Grosse, Rajesh Ranganath and Andrew˜Y Ng · 2009
Cited alongside, same era.
“Efficient reductions for imitation learning”
St“’ephane Ross and J. Bagnell · 2010
Cited alongside, same era.
“Recurrent policy gradients”
Daan Wierstra, Alexander F“”orster, Jan Peters and J“”urgen Schmidhuber · 2010
Cited alongside, same era.
“Estimation of Multivehicle Dynamics by Considering Contextual Information”
G. Agamennoni, J.I. Nieto and E.M. Nebot · 2012
Cited alongside, same era.
“ImageNet Classification with Deep Convolutional Neural Networks”
Alex Krizhevsky, Ilya Sutskever and Geoffrey˜E. Hinton · 2012
Cited alongside, same era.
“A modified car-following model based on a neural network model of the human driver effects”
Alireza Khodayari, Ali Ghaffari, Reza Kazemi and Reinhard Braunstingl · 2012
Cited alongside, same era.
“Learning Context Sensitive Behavior Models from Observations for Predicting Traffic Situations”
Tobias Gindele, Sebastian Brechtel and R“:udiger Dillmann · 2013
Cited alongside, same era.
“Comparison of Parametric and Non-Parametric Approaches for Vehicle Speed Prediction”
St“’ephanie Lef“‘evre, Chao Sun, Ruzena Bajcsy and Christian Laugier · 2014
Cited alongside, same era.
“Vehicle lateral position prediction: A small step towards a comprehensive risk assessment system”
Qiang Liu, B. Lathrop and V. Butakov · 2014
Cited alongside, same era.
“Empirical evaluation of gated recurrent neural networks on sequence modeling”
Original
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho and Yoshua Bengio · 2014
Cited alongside, same era.

Then

“Memory-based control with recurrent neural networks”
Original
Nicolas Heess, Jonathan˜J Hunt, Timothy˜P Lillicrap and David Silver · 2015
Later among the works it cites.
“Fast and accurate deep network learning by exponential linear units (ELUs)”
Original
Djork-Arn“’e Clevert, Thomas Unterthiner and Sepp Hochreiter · 2015
Later among the works it cites.
“Trust Region Policy Optimization”
John Schulman et al · 2015
Later among the works it cites.
“DeepDriving: Learning affordance for direct perception in autonomous driving”
Chenyi Chen, Ari Seff, Alain Kornhauser and Jianxiong Xiao · 2015
Later among the works it cites.
“Analysis of Microscopic Behavior Models for Probabilistic Modeling of Driver Behavior”
Tim˜Allan Wheeler, Philipp Robbel and Mykel Kochenderfer · 2016
Later among the works it cites.
“End to End Learning for Self-Driving Cars”
Mariusz Bojarski et al · 2016
Later among the works it cites.
“Model-Free Imitation Learning with Policy Optimization”
Original
Jonathan Ho, Jayesh˜K Gupta and Stefano Ermon · 2016
Later among the works it cites.
“High-Speed Highway Scene Prediction Based on Driver Models Learned From Demonstrations”
David˜Sierra Gonz“’alez, Jilles Dibangoye and Christian Laugier · 2016
Later among the works it cites.
“Planning for autonomous cars that leverages effects on human actions”
Dorsa Sadigh, Shankar Sastry, Sanjit˜A Seshia and Anca˜D Dragan · 2016
Later among the works it cites.
“Generative Adversarial Imitation Learning”
Original
Jonathan Ho and Stefano Ermon · 2016
Later among the works it cites.
“Analysis of recurrent neural networks for probabilistic modeling of driver behavior”
Jeremy Morton, Tim˜A Wheeler and Mykel˜J Kochenderfer · 2016
Later among the works it cites.
“Benchmarking Deep Reinforcement Learning for Continuous Control”
Original
Yan Duan et al · 2016
Later among the works it cites.
“Probabilistic Airport Acceptance Rate Prediction”
Jonathan Cox and Mykel˜J. Kochenderfer · 2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…