Built on

B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene Parsing through ADE20K Dataset,” in IEEE CVPR
2017
Earlier work this paper cites.
M. Grinvald, F. Furrer, T. Novkovic, J. J. Chung, C. Cadena, R. Siegwart, and J. Nieto, “Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery,” IEEE Robotics and Automation Letters
2019
Earlier work this paper cites.
M. Strecke and J. Stuckler, “EM-Fusion: Dynamic Object-Level SLAM With Probabilistic Data Association,” in IEEE/CVF ICCV
2019
Earlier work this paper cites.
G. Narita, T. Seno, T. Ishikawa, and Y. Kaji, “PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things,” in IROS
2019
Earlier work this paper cites.
B. Xu, W. Li, D. Tzoumanikas, M. Bloesch, A. Davison, and S. Leutenegger, “MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM,” in IEEE ICRA
2019
Earlier work this paper cites.
I. Armeni, Z.-Y. He, J. Gwak, A. R. Zamir, M. Fischer, J. Malik, and S. Savarese, “3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera,” in IEEE/CVF ICCV
2019
Earlier work this paper cites.
A. Rosinol, M. Abate, Y. Chang, and L. Carlone, “Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping,” in IEEE ICRA
2020
Earlier work this paper cites.
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis,” in ECCV
2020
Earlier work this paper cites.
A. Zareian, K. D. Rosa, D. H. Hu, and S.-F. Chang, “Open-Vocabulary Object Detection Using Captions,” in IEEE/CVF CVPR
2021
Earlier work this paper cites.
X. Gu, T.-Y. Lin, W. Kuo, and Y. Cui, “Open-vocabulary Object Detection via Vision and Language Knowledge Distillation,” arXiv preprint arXiv:2104.13921
Original
2021
Earlier work this paper cites.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al
2021
Earlier work this paper cites.
V. Blukis, R. Knepper, and Y. Artzi, “Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following,” in Conference on Robot Learning
2021
Earlier work this paper cites.
M. Grinvald, F. Tombari, R. Siegwart, and J. Nieto, “TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction,” in IEEE ICRA
2021
Earlier work this paper cites.
S.-C. Wu, J. Wald, K. Tateno, N. Navab, and F. Tombari, “SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences,” in IEEE/CVF CVPR
2021
Earlier work this paper cites.
S. Zhi, T. Laidlow, S. Leutenegger, and A. J. Davison, “In-Place Scene Labelling and Understanding with Implicit Scene Representation,” in IEEE/CVF International Conference on Computer Vision
2021
Earlier work this paper cites.
E. Sucar, S. Liu, J. Ortiz, and A. J. Davison, “iMAP: Implicit Mapping and Positioning in Real-Time,” in IEEE/CVF ICCV
2021
Earlier work this paper cites.
P. Wang, L. Liu, Y. Liu, C. Theobalt, T. Komura, and W. Wang, “NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction,” arXiv preprint arXiv:2106.10689
Original
2021
Earlier work this paper cites.

Similar

S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, T. Funkhouser, et al
Original
2022
Cited alongside, same era.
S. Zhi, E. Sucar, A. Mouton, I. Haughton, T. Laidlow, and A. J. Davison, “iLabel: Revealing Objects in Neural Fields,” IEEE Robotics and Automation Letters
2022
Cited alongside, same era.
K. Mazur, E. Sucar, and A. J. Davison, “Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding,” arXiv preprint arXiv:2210.03043
Original
2022
Cited alongside, same era.
G. Ghiasi, X. Gu, Y. Cui, and T.-Y. Lin, “Scaling Open-Vocabulary Image Segmentation with Image-Level Labels,” in ECCV
2022
Cited alongside, same era.
B. Li, K. Q. Weinberger, S. Belongie, V. Koltun, and R. Ranftl, “Language-driven Semantic Segmentation,” arXiv preprint arXiv:2201.03546
Original

Then

D. Shah, B. Osinski, B. Ichter, and S. Levine, “LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action,” arXiv preprint arXiv:2207.04429
Original
2022
Later among the works it cites.
Z. Wang, M. Li, M. Wu, M.-F. Moens, and T. Tuytelaars, “Find a Way Forward: a Language-Guided Semantic Map Navigator,” arXiv preprint arXiv:2203.03183
Original
2022
Later among the works it cites.
A. Majumdar, G. Aggarwal, B. Devnani, J. Hoffman, and D. Batra, “ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings,” arXiv preprint arXiv:2206.12403
Original
2022
Later among the works it cites.
R. Ding, J. Yang, C. Xue, W. Zhang, S. Bai, and X. Qi, “Language-driven Open-Vocabulary 3D Scene Understanding,” arXiv preprint arXiv:2211.16312
Original
2022
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2023

Neural Implicit Vision-Language Feature Fields

Blomqvist, Kenneth, Milano, Francesco, Chung, Jen Jen et al.

Understand

Recently, groundbreaking results have been presented on open-vocabulary semantic image segmentation.

Such methods segment each pixel in an image into arbitrary categories provided at run-time in the form of text prompts, as opposed to a fixed set of classes defined at training time.
In this work, we present a zero-shot volumetric open-vocabulary semantic scene segmentation method.
Our method builds on the insight that we can fuse image features from a vision-language model into a neural implicit representation.

Built on

B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene Parsing through ADE20K Dataset,” in IEEE CVPR
2017
Earlier work this paper cites.
M. Grinvald, F. Furrer, T. Novkovic, J. J. Chung, C. Cadena, R. Siegwart, and J. Nieto, “Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery,” IEEE Robotics and Automation Letters
2019
Earlier work this paper cites.
M. Strecke and J. Stuckler, “EM-Fusion: Dynamic Object-Level SLAM With Probabilistic Data Association,” in IEEE/CVF ICCV
2019
Earlier work this paper cites.
G. Narita, T. Seno, T. Ishikawa, and Y. Kaji, “PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things,” in IROS
2019
Earlier work this paper cites.
B. Xu, W. Li, D. Tzoumanikas, M. Bloesch, A. Davison, and S. Leutenegger, “MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM,” in IEEE ICRA
2019
Earlier work this paper cites.
I. Armeni, Z.-Y. He, J. Gwak, A. R. Zamir, M. Fischer, J. Malik, and S. Savarese, “3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera,” in IEEE/CVF ICCV
2019
Earlier work this paper cites.
A. Rosinol, M. Abate, Y. Chang, and L. Carlone, “Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping,” in IEEE ICRA
2020
Earlier work this paper cites.
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis,” in ECCV
2020
Earlier work this paper cites.
A. Zareian, K. D. Rosa, D. H. Hu, and S.-F. Chang, “Open-Vocabulary Object Detection Using Captions,” in IEEE/CVF CVPR
2021
Earlier work this paper cites.
X. Gu, T.-Y. Lin, W. Kuo, and Y. Cui, “Open-vocabulary Object Detection via Vision and Language Knowledge Distillation,” arXiv preprint arXiv:2104.13921
Original
2021
Earlier work this paper cites.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al
2021
Earlier work this paper cites.
V. Blukis, R. Knepper, and Y. Artzi, “Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following,” in Conference on Robot Learning
2021
Earlier work this paper cites.
M. Grinvald, F. Tombari, R. Siegwart, and J. Nieto, “TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction,” in IEEE ICRA
2021
Earlier work this paper cites.
S.-C. Wu, J. Wald, K. Tateno, N. Navab, and F. Tombari, “SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences,” in IEEE/CVF CVPR
2021
Earlier work this paper cites.
S. Zhi, T. Laidlow, S. Leutenegger, and A. J. Davison, “In-Place Scene Labelling and Understanding with Implicit Scene Representation,” in IEEE/CVF International Conference on Computer Vision
2021
Earlier work this paper cites.
E. Sucar, S. Liu, J. Ortiz, and A. J. Davison, “iMAP: Implicit Mapping and Positioning in Real-Time,” in IEEE/CVF ICCV
2021
Earlier work this paper cites.
P. Wang, L. Liu, Y. Liu, C. Theobalt, T. Komura, and W. Wang, “NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction,” arXiv preprint arXiv:2106.10689
Original
2021
Earlier work this paper cites.

Similar

S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, T. Funkhouser, et al
Original
2022
Cited alongside, same era.
S. Zhi, E. Sucar, A. Mouton, I. Haughton, T. Laidlow, and A. J. Davison, “iLabel: Revealing Objects in Neural Fields,” IEEE Robotics and Automation Letters
2022
Cited alongside, same era.
K. Mazur, E. Sucar, and A. J. Davison, “Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding,” arXiv preprint arXiv:2210.03043
Original
2022
Cited alongside, same era.
G. Ghiasi, X. Gu, Y. Cui, and T.-Y. Lin, “Scaling Open-Vocabulary Image Segmentation with Image-Level Labels,” in ECCV
2022
Cited alongside, same era.
B. Li, K. Q. Weinberger, S. Belongie, V. Koltun, and R. Ranftl, “Language-driven Semantic Segmentation,” arXiv preprint arXiv:2201.03546
Original

Then

D. Shah, B. Osinski, B. Ichter, and S. Levine, “LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action,” arXiv preprint arXiv:2207.04429
Original
2022
Later among the works it cites.
Z. Wang, M. Li, M. Wu, M.-F. Moens, and T. Tuytelaars, “Find a Way Forward: a Language-Guided Semantic Map Navigator,” arXiv preprint arXiv:2203.03183
Original
2022
Later among the works it cites.
A. Majumdar, G. Aggarwal, B. Devnani, J. Hoffman, and D. Batra, “ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings,” arXiv preprint arXiv:2206.12403
Original
2022
Later among the works it cites.
R. Ding, J. Yang, C. Xue, W. Zhang, S. Bai, and X. Qi, “Language-driven Open-Vocabulary 3D Scene Understanding,” arXiv preprint arXiv:2211.16312
Original
2022
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2022

Cited alongside, same era.

H. Zhang, P. Zhang, X. Hu, Y. Chen, L. Li, X. Dai, L. Wang, L. Yuan, J. Hwang, and J. Gao, “GLIPv2: Unifying Localization and Vision-Language Understanding,” in NeurIPS

2022

Cited alongside, same era.

X. Zou, Z.-Y. Dou, J. Yang, Z. Gan, L. Li, C. Li, X. Dai, H. Behl, J. Wang, L. Yuan, et al

Original

2022

Cited alongside, same era.

K. Ranasinghe, B. McKinzie, S. Ravi, Y. Yang, A. Toshev, and J. Shlens, “Perceptual Grouping in Vision-Language Models,” arXiv preprint arXiv:2210.09996

Original

2022

Cited alongside, same era.

M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, et al

Original

2022

Cited alongside, same era.

C. Huang, O. Mees, A. Zeng, and W. Burgard, “Visual Language Maps for Robot Navigation,” arXiv preprint arXiv:2210.05714

Original

2022

Cited alongside, same era.

C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, and Y. Su, “LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models,” arXiv preprint arXiv:2212.04088

Original

2022

Cited alongside, same era.

B. Chen, F. Xia, B. Ichter, K. Rao, K. Gopalakrishnan, M. Ryoo, A. Stone, and D. Kappler, “Open-vocabulary Queryable Scene Representations for Real World Planning,” arxiv preprint arXiv:2209.09874

Original

2022

Cited alongside, same era.

L. Schmid, J. Delmerico, J. L. Schönberger, J. Nieto, M. Pollefeys, R. Siegwart, and C. Cadena, “Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency,” in IEEE ICRA

2022