2017

Should Robots be Obedient?

Milli, Smitha, Hadfield-Menell, Dylan, Dragan, Anca et al.

Understand

Intuitively, obedience -- following the order that a human gives -- seems like a good property for a robot to have.

But, we humans are not perfect and we may give orders that are not best aligned to our preferences.
We show that when a human is not perfectly rational then a robot that tries to infer and act according to the human's underlying preferences can always perform better than a robot that simply follows the human's literal order.
Thus, there is a tradeoff between the obedience of a robot and the value it can attain for its owner.

Toward a Programming Laboratory
Warren Teitelman · 1970
Earlier work this paper cites.
On the consistency of Bayes estimates
Persi Diaconis and David Freedman · 1986
Earlier work this paper cites.
The First Law of Robotics (a call to arms)
Daniel Weld and Oren Etzioni · 1994
Earlier work this paper cites.
Reinforcement Learning: A Survey
Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore · 1996
Earlier work this paper cites.
Algorithms for inverse reinforcement learning
Andrew Y Ng, Stuart J Russell, et al · 2000
Earlier work this paper cites.
Using cursor prediction to smooth telepointer jitter
Carl Gutwin, Jeff Dyck, and Jennifer Burkitt · 2003
Earlier work this paper cites.
Apprenticeship learning via inverse reinforcement learning
Pieter Abbeel and Andrew Y Ng · 2004
Earlier work this paper cites.
What Should We Want From a Robot Ethic
Peter M Asaro · 2006
Earlier work this paper cites.

Bayesian Inverse Reinforcement Learning
Deepak Ramachandran and Eyal Amir · 2007
Cited alongside, same era.
The Burden of Embodied Autonomy: Some Reflections on the Social and Ethical Implications of Autonomous Robots
Matthias Scheutz and Charles Crowell · 2007
Cited alongside, same era.
Maximum Entropy Inverse Reinforcement Learning
Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, and Anind K Dey · 2008
Cited alongside, same era.
MAP Inference for Bayesian Inverse Reinforcement Learning
Jaedeug Choi and Kee-Eung Kim · 2011
Cited alongside, same era.
Formalizing assistive teleoperation
Anca D Dragan and Siddhartha S Srinivasa · 2012
Cited alongside, same era.
Improving the Efficiency of Bayesian Inverse Reinforcement Learning
Bernard Michini and Jonathan P How · 2012
Cited alongside, same era.
Superintelligence: Paths, Dangers, Strategies
Nick Bostrom · 2014
Cited alongside, same era.
The Case for Regulating Fully Autonomous Weapons
Johin Lewis · 2014
Cited alongside, same era.

Shared autonomy via hindsight optimization
Shervin Javdani, Siddhartha S Srinivasa, and J Andrew Bagnell · 2015
Later among the works it cites.
Research Priorities for Robust and Beneficial Artificial Intelligence
Stuart Russell, Daniel Dewey, and Max Tegmark · 2015
Later among the works it cites.
Corrigibility
Nate Soares, Benja Fallenstein, Stuart Armstrong, and Eliezer Yudkowsky · 2015
Later among the works it cites.
Learning the preferences of ignorant, inconsistent agents
Owain Evans, Andreas Stuhlmüller, and Noah D Goodman · 2016
Later among the works it cites.
Cooperative Inverse Reinforcement Learning
Dylan Hadfield-Menell, Stuart J Russell, Pieter Abbeel, and Anca Dragan · 2016
Later among the works it cites.
Safely Interruptible Agents
Laurent Orseau and Stuart Armstrong · 2016
Later among the works it cites.
The Off-Switch Game
Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, and Stuart Russell · 2017
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…