2018

TACO: Learning Task Decomposition via Temporal Alignment for Control

Shiarlis, Kyriacos, Wulfmeier, Markus, Salter, Sasha et al.

Understand

Many advanced Learning from Demonstration (LfD) methods consider the decomposition of complex, real-world tasks into simpler sub-tasks.

By reusing the corresponding sub-policies within and between tasks, they provide training data for each policy from different high-level tasks and compose them to perform novel ones.
Existing approaches to modular LfD focus either on learning a single high-level task or depend on domain knowledge and temporal segmentation.
In contrast, we propose a weakly supervised, domain-agnostic approach based on task sketches, which include only the sequence of sub-tasks performed in each demonstration.

Skill acquisition from human demonstration using a hidden markov model
Hovland, G. E., Sikka, P., and McCarragher, B. J · 1996
Earlier work this paper cites.
Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., and Singh, S · 1999
Earlier work this paper cites.
Learning task models from multiple human demonstrations
Ekvall, S. and Kragic, D · 2006
Earlier work this paper cites.
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Graves, A., Fernández, S., Gomez, F., and Schmidhuber, J · 2006
Earlier work this paper cites.
Dynamic movement primitives-a framework for motor control in humans and humanoid robotics
Schaal, S · 2006
Earlier work this paper cites.
A survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., and Browning, B · 2009
Earlier work this paper cites.
Learning and generalization of motor skills by learning from demonstration
Pastor, P., Hoffmann, H., Asfour, T., and Schaal, S · 2009
Earlier work this paper cites.
Sequence transduction with recurrent neural networks
Original
Graves, A · 2012
Earlier work this paper cites.
Robot learning from demonstration by constructing skill trees
Konidaris, G., Kuindersma, S., Grupen, R., and Barto, A · 2012
Earlier work this paper cites.

Learning and generalization of complex tasks from unstructured demonstrations
Niekum, S., Osentoski, S., Konidaris, G., and Barto, A. G · 2012
Cited alongside, same era.
Mujoco: A physics engine for model-based control
Todorov, E., Erez, T., and Tassa, Y · 2012
Cited alongside, same era.
Learning manipulation actions from a few demonstrations
Abdo, N., Kretzschmar, H., Spinello, L., and Stachniss, C · 2013
Cited alongside, same era.
Probabilistic movement primitives
Paraschos, A., Daniel, C., Peters, J. R., and Neumann, G · 2013
Cited alongside, same era.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Original
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y · 2014
Cited alongside, same era.
Learning to sequence movement primitives from demonstrations
Manschitz, S., Kober, J., Gienger, M., and Peters, J · 2014
Cited alongside, same era.
Dropout: A simple way to prevent neural networks from overfitting
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R · 2014
Cited alongside, same era.
Learning grounded finite-state representations from unstructured demonstrations
Niekum, S., Osentoski, S., Konidaris, G., Chitta, S., Marthi, B., and Barto, A. G · 2015
Cited alongside, same era.
Connectionist temporal modeling for weakly supervised action labeling
Huang, D.-A., Fei-Fei, L., and Niebles, J. C · 2016
Cited alongside, same era.

Listen, attend, and walk: Neural mapping of navigational instructions to action sequences
Mei, H., Bansal, M., and Walter, M. R · 2016
Later among the works it cites.
Modular multitask reinforcement learning with policy sketches
Andreas, J., Klein, D., and Levine, S · 2017
Later among the works it cites.
Gated-attention architectures for task-oriented language grounding
Original
Chaplot, D. S., Sathyendra, K. M., Pasumarthi, R. K., Rajagopal, D., and Salakhutdinov, R · 2017
Later among the works it cites.
Multi-level discovery of deep options
Original
Fox, R., Krishnan, S., Stoica, I., and Goldberg, K · 2017
Later among the works it cites.
OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning
Henderson, P., Chang, W.-D., Bacon, P.-L., Meger, D., Pineau, J., and Precup, D · 2017
Later among the works it cites.
Ddco: Discovery of deep continuous options for robot learning from demonstrations
Krishnan, S., Fox, R., Stoica, I., and Goldberg, K · 2017
Later among the works it cites.
Dart: Noise injection for robust imitation learning
Laskey, M., Lee, J., Fox, R., Dragan, A., and Goldberg, K · 2017
Later among the works it cites.
Learning movement primitive libraries through probabilistic segmentation
Lioutikov, R., Neumann, G., Maeda, G., and Peters, J · 2017
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…