Fetching the paper…
Reading the bibliography…
Learning representations in the joint domain of vision and touch can improve manipulation dexterity, robustness, and sample-complexity by exploiting mutual information and complementary cues.
Dream to control: Learning behaviors by latent imagination
D. Hafner, T. P. Lillicrap, J. Ba, and M. Norouzi · 1912
Earlier work this paper cites.
Pilco: A model-based and data-efficient approach to policy search
M. Deisenroth and C. E. Rasmussen · 2011
Earlier work this paper cites.
Auto-Encoding Variational Bayes
D. P. Kingma and M. Welling · 2013
Earlier work this paper cites.
Deep Residual Learning for Image Recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
D. P. Kingma and J. Ba · 2015
Earlier work this paper cites.
Improving pilco with bayesian neural network dynamics models
Y. Gal, R. McAllister, and C. E. Rasmussen · 2016
Earlier work this paper cites.
Attention is all you need, 2017
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
Model-based planning with discrete and continuous actions
M. Henaff, W. F. Whitney, and Y. LeCun · 2017
Earlier work this paper cites.
Multimodal machine learning: A survey and taxonomy
T. Baltrusaitis, C. Ahuja, and L. Morency · 2017
Earlier work this paper cites.
Curiosity-driven exploration by self-supervised prediction
D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell · 2017
Earlier work this paper cites.
D. Ha and J. Schmidhuber · 2018
Earlier work this paper cites.
Integrating state representation learning into deep reinforcement learning
T. De Bruin, J. Kober, K. Tuyls, and R. Babuška · 2018
Earlier work this paper cites.
Learning latent dynamics for planning from pixels
D. Hafner, T. P. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson · 2018
Earlier work this paper cites.
M. A. Lee, Y. Zhu, K. Srinivasan, P. Shah, S. Savarese, L. Fei-Fei, A. Garg, and J. Bohg · 2018
Earlier work this paper cites.
Soft actor-critic algorithms and applications
T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, and S. Levine · 2018
Earlier work this paper cites.
SOLAR: deep structured latent representations for model-based reinforcement learning
M. Zhang, S. Vikram, L. M. Smith, P. Abbeel, M. J. Johnson, and S. Levine · 2018
Earlier work this paper cites.
B. Amos, L. Dinh, S. Cabi, T. Rothörl, S. G. Colmenarejo, A. Muldal, T. Erez, Y. Tassa, N. de Freitas, and M. Denil · 2018
Earlier work this paper cites.
Deep variational reinforcement learning for POMDPs
M. Igl, L. Zintgraf, T. A. Le, F. Wood, and S. Whiteson · 2018
Cited alongside, same era.
Learning latent dynamics for planning from pixels
D. Hafner, T. P. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson · 2018
Cited alongside, same era.
Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor
T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine · 2018
Cited alongside, same era.
Improving sample efficiency in model-free reinforcement learning from images, 10 2019
D. Yarats, A. Zhang, I. Kostrikov, B. Amos, J. Pineau, and R. Fergus · 2019
Cited alongside, same era.
Imitation learning from video by leveraging proprioception
F. Torabi, G. Warnell, and P. Stone · 2019
Cited alongside, same era.
Multi-modality cross attention network for image and sentence matching
X. Wei, T. Zhang, Y. Li, Y. Zhang, and F. Wu · 2020
Later among the works it cites.
Curriculum learning for reinforcement learning domains: A framework and survey
S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, and P. Stone · 2020
Later among the works it cites.
Decoupling representation learning from reinforcement learning
A. Stooke, K. Lee, P. Abbeel, and M. Laskin · 2021
Later among the works it cites.
Coupling vision and proprioception for navigation of legged robots
Z. Fu, A. Kumar, A. Agarwal, H. Qi, J. Malik, and D. Pathak · 2021
Later among the works it cites.
Sim-to-real for robotic tactile sensing via physics-based simulation and learned latent projections
Y. Narang, B. Sundaralingam, M. Macklin, A. Mousavian, and D. Fox · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Connecting touch and vision via cross-modal prediction
Y. Li, J.-Y. Zhu, R. Tedrake, and A. Torralba · 2019
Cited alongside, same era.
See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion
N. Fazeli, M. Oller, J. Wu, Z. Wu, J. B. Tenenbaum, and A. Rodriguez · 2019
Cited alongside, same era.
Deepmdp: Learning continuous latent space models for representation learning
C. Gelada, S. Kumar, J. Buckman, O. Nachum, and M. G. Bellemare · 2019
Cited alongside, same era.
Multimodal transformer for unaligned multimodal language sequences
Y.-H. H. Tsai, S. Bai, P. P. Liang, J. Z. Kolter, L.-P. Morency, and R. Salakhutdinov · 2019
Cited alongside, same era.
Multi-modality latent interaction network for visual question answering
P. Gao, H. You, Z. Zhang, X. Wang, and H. Li · 2019
Cited alongside, same era.
Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation
A. Alspach, K. Hashimoto, N. Kuppuswamy, and R. Tedrake · 2019
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby · 2020
Cited alongside, same era.
Offline reinforcement learning from images with latent space models
R. Rafailov, T. Yu, A. Rajeswaran, and C. Finn · 2021
Later among the works it cites.
Model-based reinforcement learning via latent-space collocation
O. Rybkin, C. Zhu, A. Nagabandi, K. Daniilidis, I. Mordatch, and S. Levine · 2021
Later among the works it cites.
Laser: Learning a latent action space for efficient reinforcement learning
A. Allshire, R. Martín-Martín, C. Lin, S. Manuel, S. Savarese, and A. Garg · 2021
Later among the works it cites.
Dreaming: Model-based reinforcement learning by latent imagination without reconstruction
M. Okada and T. Taniguchi · 2021
Later among the works it cites.
Tokenlearner: What can 8 learned tokens do for images and videos?
M. S. Ryoo, A. J. Piergiovanni, A. Arnab, M. Dehghani, and A. Angelova · 2021
Later among the works it cites.
Partially Observable Markov Decision Processes (POMDPs) and Robotics
H. Kurniawati · 2021
Later among the works it cites.
Touch-based curiosity for sparse-reward tasks
S. Rajeswar, C. Ibrahim, N. Surya, F. Golemo, D. Vázquez, A. C. Courville, and P. O. Pinheiro · 2021
Later among the works it cites.
I. Taylor, S. Dong, and A. Rodriguez · 2021
Later among the works it cites.
Reinforcement learning with vision-proprioception model for robot planar pushing
L. Cong, H. Liang, P. Ruppel, Y. Shi, M. Görner, N. Hendrich, and J. Zhang · 2022
Closest in time.
Learning multi-object dynamics with compositional neural radiance fields
D. Driess, Z. Huang, Y. Li, R. Tedrake, and M. Toussaint · 2022
Closest in time.
Virdo: Visio-tactile implicit representations of deformable objects
Y. Wi, P. Florence, A. Zeng, and N. Fazeli · 2022
Closest in time.