2020

The Surprising Effectiveness of Linear Models for Visual Foresight in Object Pile Manipulation

Suh, H. J. Terry, Tedrake, Russ

Understand

In this paper, we tackle the problem of pushing piles of small objects into a desired target set using visual feedback.

Unlike conventional single-object manipulation pipelines, which estimate the state of the system parametrized by pose, the underlying physical state of this system is difficult to observe from images.
Thus, we take the approach of reasoning directly in the space of images, and acquire the dynamics of visual measurements in order to synthesize a visual-feedback policy.
We present a simple controller using an image-space Lyapunov function, and evaluate the closed-loop performance using three different class of models for image prediction: deep-learning-based models for image-to-image translation, an object-centric model obtained from treating each pixel as a particle, and a switched-linear system where an action-dependent linear map is used.

Built on

Parametric correspondence and chamfer matching: Two new techniques for image matching
Barrow, H.G., Tenenbaum, J.M., Bolles, R.C., Wolf, H.C.: · 1977
Earlier work this paper cites.
Mechanics and planning of manipulator pushing operations
Mason, M.T.: · 1986
Earlier work this paper cites.
Estimation of mass and center of mass of graspless and shape-unknown object
Yong Yu, Fukuda, K., Tsujio, S.: · 1999
Earlier work this paper cites.
Nonlinear Optimal Control via Occupation Measures and LMI-Relaxations
Lasserre, J.B., Henrion, D., Prieur, C., Trélat, E.: · 2008
Earlier work this paper cites.
Distributed optimization and statistical learning via the alternating direction method of multipliers
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: · 2011
Earlier work this paper cites.
Stacked convolutional auto-encoders for hierarchical feature extraction
Masci, J., Meier, U., Cireşan, D., Schmidhuber, J.: · 2011
Earlier work this paper cites.
Cvxopt: A python package for convex optimization, version 1.1.6 (2013)
Andersen, M., Dahl, J., Vandenberghe, L.: · 2013
Earlier work this paper cites.
Difftaichi: Differentiable programming for physical simulation
Original
Hu, Y., Anderson, L., Li, T., Q, S., Carr, N., Ragan-Kelley, J., Durand, F.: · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Kingma, D.P., Ba, J.: · 2014
Earlier work this paper cites.
Analysis and control of linear switched systems
Colaneri, P.: · 2015
Earlier work this paper cites.

Similar

Learning visual feature spaces for robotic manipulation with deep spatial autoencoders
Original
Finn, C., Tan, X.Y., Duan, Y., Darrell, T., Levine, S., Abbeel, P.: · 2015
Cited alongside, same era.
Learning visual predictive models of physics for playing billiards
Original
Fragkiadaki, K., Agrawal, P., Levine, S., Malik, J.: · 2015
Cited alongside, same era.
A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: · 2015
Cited alongside, same era.
Learning to poke by poking: Experiential learning of intuitive physics
Original
Agrawal, P., Nair, A., Abbeel, P., Malik, J., Levine, S.: · 2016
Cited alongside, same era.
Unsupervised learning for physical interaction through video prediction
Finn, C., Goodfellow, I., Levine, S.: · 2016
Cited alongside, same era.
Deep visual foresight for planning robot motion
Original
Finn, C., Levine, S.: · 2016
Cited alongside, same era.
Scalable identification of stable positive systems
Umenberger, J., Manchester, I.R.: · 2016
Cited alongside, same era.
Markov Chains: From Theory to Implementation and Experimentation
Gagniuc, P.: · 2017
Cited alongside, same era.
Imagenet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., Hinton, G.E.: · 2017
Cited alongside, same era.

Then

Visual interaction networks
Original
Watters, N., Tacchetti, A., Weber, T., Pascanu, R., Battaglia, P.W., Zoran, D.: · 2017
Later among the works it cites.
Robotic cleaning through dirt rearrangement planning with learned transition models
Elliott, S., Cakmak, M.: · 2018
Later among the works it cites.
Friction variability in planar pushing data: Anisotropic friction and data-collection bias
Ma, D., Rodriguez, A.: · 2018
Later among the works it cites.
Learning synergies between pushing and grasping with self-supervised deep reinforcement learning
Original
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., Funkhouser, T.A.: · 2018
Later among the works it cites.
Unsupervised learning of object structure and dynamics from videos
Minderer, M., Sun, C., Villegas, R., Cole, F., Murphy, K., Lee, H.: · 2019
Later among the works it cites.
Keto: Learning keypoint representations for tool manipulation
Original
Qin, Z., Fang, K., Zhu, Y., Li, F.F., Savarese, S.: · 2019
Later among the works it cites.
Planning robot motion using deep visual prediction
Original
Sarkar, M., Pradhan, P., Ghose, D.: · 2019
Later among the works it cites.
Learning to manipulate object collections using grounded state representations
Wilson, M., Hermans, T.: · 2019
Later among the works it cites.
Object-centric forward modeling for model predictive control
Original
Ye, Y., Gandhi, D., Gupta, A., Tulsiani, S.: · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…