2020

iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes

Shen, Bokui, Xia, Fei, Li, Chengshu et al.

Understand

We present iGibson 1.0, a novel simulation environment to develop robotic solutions for interactive tasks in large-scale realistic scenes.

Our environment contains 15 fully interactive home-sized scenes with 108 rooms populated with rigid and articulated objects.
The scenes are replicas of real-world homes, with distribution and the layout of objects aligned to those of the real world.
iGibson 1.0 integrates several key features to facilitate the study of interactive tasks: i) generation of high-quality virtual sensor signals (RGB, depth, segmentation, LiDAR, flow and so on), ii) domain randomization to change the materials of the objects (both visual and physical) and/or their shapes, iii) integrated sampling-based motion planners to generate collision-free trajectories for robot bases and arms, and iv) intuitive human-iGibson interface that enables efficient collection of human demonstrations.

Built on

C. Schlick, “An inexpensive brdf model for physically-based rendering,” in
1994
Earlier work this paper cites.
S. M. Lavalle, “Rapidly-exploring random trees: A new tool for path planning,” Department of Computer Science; Iowa State University, Tech. Rep., 1998
1998
Earlier work this paper cites.
R. Bohlin and L. E. Kavraki, “Path planning using lazy prm,” in
2000
Earlier work this paper cites.
S. M. LaValle,
2006
Earlier work this paper cites.
M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, “Ros: an open-source robot operating system,” in
2009
Earlier work this paper cites.
K. Hauser and V. Ng-Thow-Hing, “Fast smoothing of manipulator trajectories using optimal bounded-acceleration shortcuts,” in
2010
Earlier work this paper cites.
E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in
2012
Earlier work this paper cites.
E. Coumans
2013
Earlier work this paper cites.
D. Merkel, “Docker: lightweight linux containers for consistent development and deployment,”
2014
Earlier work this paper cites.
A. X. Chang
Original
2015
Earlier work this paper cites.
A. H. Qureshi and Y. Ayaz, “Intelligent bidirectional rapidly-exploring random trees for optimal motion planning in complex cluttered environments,”
2015
Earlier work this paper cites.
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in
2015
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
2015
Earlier work this paper cites.
T. P. Lillicrap
2016
Earlier work this paper cites.
S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,”
2016
Earlier work this paper cites.
G. Brockman
Original
2016
Earlier work this paper cites.
L. Porzi, S. R. Bulo, A. Penate-Sanchez, E. Ricci, and F. Moreno-Noguer, “Learning depth-aware deep representations for robotic perception,”
2016
Earlier work this paper cites.
C. Choi and H. I. Christensen, “Rgb-d object pose estimation in unstructured environments,”
2016
Earlier work this paper cites.
F. Sadeghi and S. Levine, “Cad2rl: Real single-image flight without a single real image,”
2016
Earlier work this paper cites.
H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in
2016
Earlier work this paper cites.
P. Mirowski
2017
Earlier work this paper cites.

Similar

Y. Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-Fei, and A. Farhadi, “Target-driven visual navigation in indoor scenes using deep reinforcement learning,” in
2017
Cited alongside, same era.
E. Kolve
Original
2017
Cited alongside, same era.
A. Chang
2017
Cited alongside, same era.
J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in
2017
Cited alongside, same era.
E. Parisotto and R. Salakhutdinov, “Neural map: Structured memory for deep reinforcement learning,”
2018
Cited alongside, same era.
C. R. Garrett
2018
Cited alongside, same era.
F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese, “Gibson env: Real-world perception for embodied agents,” in
2018
Cited alongside, same era.
J. Lee, M. X. Grey, S. Ha, T. Kunz, S. Jain, Y. Ye, S. S. Srinivasa, M. Stilman, and C. K. Liu, “Dart: Dynamic animation and robotics toolkit,”
2018
Cited alongside, same era.
J. Liang, V. Makoviychuk, A. Handa, N. Chentanez, M. Macklin, and D. Fox, “Gpu-accelerated robotic simulation for distributed reinforcement learning,”
2018
Cited alongside, same era.
Caelan Reed Garrett, “PyBullet Planning.”
2018
Cited alongside, same era.
P. Anderson
Original
2018
Cited alongside, same era.
A. Mandlekar
2018
Cited alongside, same era.

Then

N. Hirose, F. Xia, R. Martín-Martín, A. Sadeghian, and S. Savarese, “Deep visual mpc-policy learning for navigation,”
2019
Later among the works it cites.
K. Mo, S. Zhu, A. X. Chang, L. Yi, S. Tripathi, L. J. Guibas, and H. Su, “Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding,” in
2019
Later among the works it cites.
V. Casser, S. Pirk, R. Mahjourian, and A. Angelova, “Depth prediction without the sensors: Leveraging structure for unsupervised learning from monocular videos,” in
2019
Later among the works it cites.
Y. Hu, J. Hugonot, P. Fua, and M. Salzmann, “Segmentation-driven 6d object pose estimation,” in
2019
Later among the works it cites.
S. James
2019
Later among the works it cites.
S. Qiao, H. Wang, C. Liu, W. Shen, and A. Yuille, “Weight standardization,”
Original
2019
Later among the works it cites.
S. James, Z. Ma, D. R. Arrojo, and A. J. Davison, “Rlbench: The robot learning benchmark & learning environment,”
2020
Closest in time.
T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine, “Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,” in
2020
Closest in time.
F. Xiang
2020
Closest in time.
Y. Zhu, J. Wong, A. Mandlekar, and R. Martín-Martín, “robosuite: A modular simulation framework and benchmark for robot learning,”
Original
2020
Closest in time.
“Unity,”
2020
Closest in time.
Z. Erickson
2020
Closest in time.
X. Meng, N. Ratliff, Y. Xiang, and D. Fox, “Scaling local control to large-scale topological navigation,” in
2020
Closest in time.
“Nadrin/pbr,”
2020
Closest in time.
C. Xie, Y. Xiang, A. Mousavian, and D. Fox, “The best of both modes: Separately leveraging rgb and depth for unseen object instance segmentation,” in
2020
Closest in time.
I. Nematollahi, O. Mees, L. Hermann, and W. Burgard, “Hindsight for foresight: Unsupervised structured dynamics models from physical interaction,”
2020
Closest in time.
M. Yan, Q. Sun, I. Frosio, S. Tyree, and J. Kautz, “How to close sim-real gap? transfer with segmentation!”
Original
2020
Closest in time.
M. Andrychowicz
2020
Closest in time.
F. Xia, C. Li, R. Martín-Martín, O. Litany, A. Toshev, and S. Savarese, “Relmogen: Leveraging motion generation in reinforcement learning for mobile manipulation,”
2021
Closest in time.
Y. Lee, E. S. Hu, Z. Yang, A. Yin, and J. J. Lim, “IKEA furniture assembly environment for long-horizon complex manipulation tasks,”
2021
Closest in time.
C. Pérez-D’Arpino, C. Liu, P. Goebel, R. Martín-Martín, and S. Savarese, “Robot navigation in constrained pedestrian environments using reinforcement learning,”
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…