Fetching the paper…

Reading the bibliography…

2023

Model-Based Runtime Monitoring with Interactive Imitation Learning

Liu, Huihan, Dass, Shivin, Martín-Martín, Roberto et al.

Original
PDF
Post on X

Understand

Robot learning methods have recently made great strides, but generalization and robustness challenges still hinder their widespread deployment.

Failing to detect and address potential failures renders state-of-the-art learning systems not combat-ready for high-stakes tasks.
Recent advances in interactive imitation learning have presented a promising framework for human-robot teaming, enabling the robots to operate safely and continually improve their performances over long-term deployments.
Nonetheless, existing methods typically require constant human supervision and preemptive feedback, limiting their practicality in realistic domains.

Model-Based Runtime Monitoring with Interactive Imitation Learning · Around

Built on

“Successor Features Support Model-based and Model-free Reinforcement Learning”
Original
Lucas Lehnert and Michael. Littman · 1901
Earlier work this paper cites.
“When to Trust Your Model: Model-Based Policy Optimization”, 2019
Original
Michael Janner, Justin Fu, Marvin Zhang and Sergey Levine · 1906
Earlier work this paper cites.
“Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder”
Original
Zhisheng Xiao, Qing Yan and Yali Amit · 2003
Earlier work this paper cites.
“Critic Regularized Regression”, 2020
Original
Ziyu Wang et al · 2006
Earlier work this paper cites.
“A survey of robot learning from demonstration”
Brenna Argall, S. Chernova, Manuela. Veloso and Brett Browning · 2009
Earlier work this paper cites.
“robosuite: A Modular Simulation Framework and Benchmark for Robot Learning”, 2020
Original
Yuke Zhu et al · 2009
Earlier work this paper cites.
“A reduction of imitation learning and structured prediction to no-regret online learning”
Stéphane Ross, Geoffrey Gordon and Drew Bagnell · 2011
Earlier work this paper cites.
“Neural networks for prediction of robot failures”
Ali Diryag, Marko Mitić and Zoran Miljković · 2014
Earlier work this paper cites.
“Deep Residual Learning for Image Recognition”, 2015
Original
Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun · 2015
Earlier work this paper cites.
“Safe Visual Navigation via Deep Learning and Novelty Detection”
Charles Richter and Nicholas Roy · 2017
Earlier work this paper cites.
“Learning Dexterous In-hand Manipulation”
Marcin Andrychowicz et al · 2018
Earlier work this paper cites.
“QT-Opt: Scalable Deep Reinforcement Learning for Vision-based Robotic Manipulation”
Dmitry Kalashnikov et al · 2018
Earlier work this paper cites.
“Dream to control: Learning behaviors by latent imagination”
Original
Danijar Hafner, Timothy Lillicrap, Jimmy Ba and Mohammad Norouzi · 2019
Earlier work this paper cites.
“Hg-dagger: Interactive imitation learning with human experts”
Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell and Mykel Kochenderfer · 2019
Earlier work this paper cites.
“HG-DAgger: Interactive Imitation Learning with Human Experts”
Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell and Mykel. Kochenderfer · 2019
Earlier work this paper cites.
“EnsembleDagger: A Bayesian approach to safe imitation learning”
Kunal Menda, Katherine Driggs-Campbell and Mykel Kochenderfer · 2019
Earlier work this paper cites.
“Fast Run-time Monitoring, Replanning, and Recovery for Safe Autonomous System Operations”
Esen Yel and Nicola Bezzo · 2019
Earlier work this paper cites.

Similar

“Model-based offline planning”
Original
Arthur Argenson and Gabriel Dulac-Arnold · 2020
Cited alongside, same era.
“Human-in-the-loop imitation learning using remote teleoperation”
Original
Ajay Mandlekar et al · 2020
Cited alongside, same era.
“Learning to be safe: Deep rl with a safety critic”
Original
Krishnan Srinivasan et al · 2020
Cited alongside, same era.
“Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning”
Original
Lukas Brunke et al · 2021
Cited alongside, same era.
“ThriftyDAgger: Budget-aware novelty and risk gating for interactive imitation learning”
Original

Then

“Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment”
Original
Huihan Liu et al · 2022
Later among the works it cites.
“Skill-based model-based reinforcement learning”
Original
Lucy Shi, Joseph Lim and Youngwoon Lee · 2022
Later among the works it cites.
“A System-Level View on Out-of-Distribution Data in Robotics”, 2022
Original
Rohan Sinha et al · 2022
Later among the works it cites.
“Safe Reinforcement Learning by Imagining the Near Future”, 2022
Original
Garrett Thomas, Yuping Luo and Tengyu Ma · 2022
Later among the works it cites.
“Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation”
Josiah Wong et al · 2022

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Ryan Hoque et al · 2021

Cited alongside, same era.

“What Matters in Learning from Offline Human Demonstrations for Robot Manipulation”, 2021

Ajay Mandlekar et al · 2021

Cited alongside, same era.

“A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges”

Mohammadreza Salehi et al · 2021

Cited alongside, same era.

“Recovery rl: Safe reinforcement learning with learned recovery zones”

Brijen Thananjeyan et al · 2021

Cited alongside, same era.

“Predictive Runtime Monitoring for Mobile Robots using Logic-Based Bayesian Intent Inference”

Hansol Yoon and Sriram Sankaranarayanan · 2021

Cited alongside, same era.

“RT-1: Robotics Transformer for Real-World Control at Scale”, 2022

Anthony Brohan et al · 2022

Cited alongside, same era.

“Interactive Imitation Learning in Robotics: A Survey”, 2022

Carlos Celemin et al · 2022

Cited alongside, same era.

“LEADER: Learning Attention over Driving Behaviors for Planning under Uncertainty”

Mohamad. Danesh, Panpan Cai and David Hsu · 2022

Cited alongside, same era.

Later among the works it cites.

“When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning”, 2022

Annie Xie, Fahim Tajwar, Archit Sharma and Chelsea Finn · 2022

Later among the works it cites.

“DITTO: Offline Imitation Learning with World Models”

Branton DeMoss, Paul Duckworth, Nick Hawes and Ingmar Posner · 2023

Closest in time.

“Asking for Help: Failure Prediction in Behavioral Cloning through Value Approximation”

Cem Gokmen, Daniel Ho and Mohi Khansari · 2023

Closest in time.

“Asking for Help: Failure Prediction in Behavioral Cloning through Value Approximation”

Cem Gokmen, Daniel Ho and Mohi Khansari · 2023

Closest in time.

“Mastering Diverse Domains through World Models”

Danijar Hafner, Jurgis Pasukonis, Jimmy Ba and Timothy Lillicrap · 2023

Closest in time.

“Fleet-dagger: Interactive robot fleet learning with scalable human supervision”

Ryan Hoque et al · 2023

Closest in time.

“The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems”, 2023

Kai-Chieh Hsu, Haimin Hu and Jaimeández Fisac · 2023

Closest in time.

“ALAN: Autonomously Exploring Robotic Agents in the Real World”, 2023

Russell Mendonca, Shikhar Bahl and Deepak Pathak · 2023

Closest in time.

“Structured World Models from Human Videos”, 2023

Russell Mendonca, Shikhar Bahl and Deepak Pathak · 2023

Closest in time.

“A Generalist Dynamics Model for Control”

Ingmar Schubert et al · 2023

Closest in time.

“Closing the Loop on Runtime Monitors with Fallback-Safe MPC” Submitted

R. Sinha, E. Schmerling and M. Pavone · 2023

Closest in time.

“Daydreamer: World models for physical robot learning”

Philipp Wu et al · 2023

Closest in time.