Fetching the paper…

Reading the bibliography…

2019

MANGA: Method Agnostic Neural-policy Generalization and Adaptation

Bharadhwaj, Homanga, Yamaguchi, Shoichiro, Maeda, Shin-ichi

Original
PDF
Post on X

Understand

In this paper we target the problem of transferring policies across multiple environments with different dynamics parameters and motor noise variations, by introducing a framework that decouples the processes of policy learning and system identification.

Efficiently transferring learned policies to an unknown environment with changes in dynamics configurations in the presence of motor noise is very important for operating robots in the real world, and our work is a novel attempt in that direction.
We introduce MANGA: Method Agnostic Neural-policy Generalization and Adaptation, that trains dynamics conditioned policies and efficiently learns to estimate the dynamics parameters of the environment given off-policy state-transition rollouts in the environment.
Our scheme is agnostic to the type of training method used - both reinforcement learning (RL) and imitation learning (IL) strategies can be used.

MANGA: Method Agnostic Neural-policy Generalization and Adaptation · Around

Built on

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997
1997
Earlier work this paper cites.
S. Watanabe, Algebraic geometry and statistical learning theory . Cambridge University Press, 2009
2009
Earlier work this paper cites.
L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010 . Springer, 2010, pp. 177–186
2010
Earlier work this paper cites.
E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2012, pp. 5026–5033
2012
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114 , 2013
Original
2013
Earlier work this paper cites.
P. Christiano, Z. Shah, I. Mordatch, J. Schneider, T. Blackwell, J. Tobin, P. Abbeel, and W. Zaremba, “Transfer from simulation to real world through learning deep inverse dynamics model,” arXiv preprint arXiv:1610.03518 , 2016
Original
2016
Earlier work this paper cites.
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540 , 2016
Original
2016
Earlier work this paper cites.
J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” CoRR , vol. abs/1703.06907, 2017. [Online]. Available: http://arxiv.org/abs/1703.06907
Original
2017
Earlier work this paper cites.
W. Yu, J. Tan, C. K. Liu, and G. Turk, “Preparing for the unknown: Learning a universal policy with online system identification,” arXiv preprint arXiv:1702.02453 , 2017
Original
2017
Earlier work this paper cites.
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the 34th International Conference on Machine Learning-Volume 70 . JMLR. org, 2017, pp. 1126–1135
2017
Earlier work this paper cites.
J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in Advances in Neural Information Processing Systems , 2017, pp. 4077–4087
2017
Earlier work this paper cites.
F. Zhang, J. Leitner, Z. Ge, M. Milford, and P. Corke, “Adversarial discriminative sim-to-real transfer of visuo-motor policies,” arXiv preprint arXiv:1709.05746 , 2017
Original
2017
Earlier work this paper cites.

Similar

K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan, “Unsupervised pixel-level domain adaptation with generative adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 3722–3731
2017
Cited alongside, same era.
E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarial discriminative domain adaptation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 7167–7176
2017
Cited alongside, same era.
D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervised prediction,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops , 2017, pp. 16–17
2017
Cited alongside, same era.
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017
Original
2017
Cited alongside, same era.

Then

J. Yoon, T. Kim, O. Dia, S. Kim, Y. Bengio, and S. Ahn, “Bayesian model-agnostic meta-learning,” in Advances in Neural Information Processing Systems , 2018, pp. 7332–7342
2018
Later among the works it cites.
C. Finn, K. Xu, and S. Levine, “Probabilistic model-agnostic meta-learning,” in Advances in Neural Information Processing Systems , 2018, pp. 9516–9527
2018
Later among the works it cites.
L. M. Zintgraf, K. Shiarlis, V. Kurin, K. Hofmann, and S. Whiteson, “Caml: Fast context adaptation via meta-learning,” arXiv preprint arXiv:1810.03642 , 2018
Original
2018
Later among the works it cites.
Y. Burda, H. Edwards, A. J. Storkey, and O. Klimov, “Exploration by random network distillation,” CoRR , vol. abs/1810.12894, 2018. [Online]. Available: http://arxiv.org/abs/1810.12894
Original
2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017

2017

Cited alongside, same era.

X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics randomization,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2018, pp. 1–8

2018

Cited alongside, same era.

A. Zhang, H. Satija, and J. Pineau, “Decoupling dynamics and reward for transfer learning,” arXiv preprint arXiv:1804.10689 , 2018

2018

Cited alongside, same era.

S. M. Richards, F. Berkenkamp, and A. Krause, “The Lyapunov neural network: Adaptive stability certification for safe learning of dynamical systems,” in Conference on Robot Learning , 2018, pp. 466–476

2018

Cited alongside, same era.

F. Wang, B. Zhou, K. Chen, T. Fan, X. Zhang, J. Li, H. Tian, and J. Pan, “Intervention aided reinforcement learning for safe and practical policy optimization in navigation,” in Conference on Robot Learning , 2018, pp. 410–421

2018

Cited alongside, same era.

A. Srinivas, A. Jabri, P. Abbeel, S. Levine, and C. Finn, “Universal planning networks: Learning generalizable representations for visuomotor control,” in International Conference on Machine Learning , 2018, pp. 4739–4748

2018

Cited alongside, same era.

M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, et al. , “Learning dexterous in-hand manipulation,” arXiv preprint arXiv:1808.00177 , 2018

2018

Cited alongside, same era.

C. Richter and N. Roy, “Safe visual navigation via deep learning and novelty detection.”

Cited in the paper.

D. Gordon, A. Kadian, D. Parikh, J. Hoffman, and D. Batra, “Splitnet: Sim2sim and task2task transfer for embodied visual navigation,” in International Conference in Computer Vision (ICCV) , 2019

2019

Closest in time.

H. Bharadhwaj, Z. Wang, Y. Bengio, and L. Paull, “A data-efficient framework for training and sim-to-real transfer of navigation policies,” in 2019 International Conference on Robotics and Automation (ICRA) , May 2019, pp. 782–788

2019

Closest in time.

W. Yu, V. C. V. Kumar, G. Turk, and C. K. Liu, “Sim-to-real transfer for biped locomotion,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2019

2019

Closest in time.

W. Yu, C. K. Liu, and G. Turk, “Policy transfer with strategy optimization,” in International Conference on Learning Representations (ICLR) , 2019. [Online]. Available: https://openreview.net/forum?id=H1g6osRcFQ

2019

Closest in time.

Y. Yang, K. Caluwaerts, A. Iscen, J. Tan, and C. Finn, “NoRML: No-reward meta learning,” in Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems AAMAS , 2019, pp. 323–331

2019

Closest in time.

A. Pashevich, R. A. Strudel, I. Kalevatykh, I. Laptev, and C. Schmid, “Learning to augment synthetic images for sim2real policy transfer,” arXiv preprint arXiv:1903.07740 , 2019

2019

Closest in time.

F. Zhu, L. Zhu, and Y. Yang, “Sim-real joint reinforcement transfer for 3d indoor navigation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 11 388–11 397

2019

Closest in time.