Fetching the paper…
Reading the bibliography…
In recent years, 2D Vision-Language Models (VLMs) have made significant strides in image-text understanding tasks.
Matterport3d: Learning from rgb-d data in indoor environments
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang · 2017
Earlier work this paper cites.
Deep hough voting for 3d object detection in point clouds
Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas · 2019
Earlier work this paper cites.
The replica dataset: A digital replica of indoor spaces
Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al · 2019
Earlier work this paper cites.
Rio: 3d object instance re-localization in changing indoor environments
Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias Nießner · 2019
Earlier work this paper cites.
Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes
Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas · 2020
Earlier work this paper cites.
Scanrefer: 3d object localization in rgb-d scans using natural language
Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner · 2020
Earlier work this paper cites.
Pointgroup: Dual-set point grouping for 3d instance segmentation
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, and Jiaya Jia · 2020
Earlier work this paper cites.
Structured3d: A large photo-realistic dataset for structured 3d modeling
Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou · 2020
Earlier work this paper cites.
Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data
Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al · 2021
Earlier work this paper cites.
Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut · 2021
Earlier work this paper cites.
Scan2cap: Context-aware dense captioning in rgb-d scans
Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X Chang · 2021
Earlier work this paper cites.
Transrefer3d: Entity-and-relation aware transformer for fine-grained 3d visual grounding
Dailan He, Yusheng Zhao, Junyu Luo, Tianrui Hui, Shaofei Huang, Aixi Zhang, and Si Liu · 2021
Earlier work this paper cites.
Text-guided graph neural networks for referring 3d instance segmentation
Pao-Hsiang Huang, Hsin-Hsi Lee, Hwann-Tzong Chen, and Tyng-Luh Liu · 2021
Earlier work this paper cites.
An end-to-end transformer model for 3d object detection
Ishan Misra, Rohit Girdhar, and Armand Joulin · 2021
Earlier work this paper cites.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Earlier work this paper cites.
Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai
Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al · 2021
Earlier work this paper cites.
Sat: 2d semantics assisted training for 3d visual grounding
Zhengyuan Yang, Songyang Zhang, Liwei Wang, and Jiebo Luo · 2021
Earlier work this paper cites.
Instancerefer: Cooperative holistic understanding for visual grounding on point clouds through instance multi-level contextual referring
Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Zhen Li, and Shuguang Cui · 2021
Earlier work this paper cites.
3dvg-transformer: Relation modeling for visual grounding on point clouds
Lichen Zhao, Daigang Cai, Lu Sheng, and Dong Xu · 2021
Earlier work this paper cites.
Flamingo: a visual language model for few-shot learning
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al · 2022
Earlier work this paper cites.
Scanqa: 3d question answering for spatial scene understanding
Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe · 2022
Earlier work this paper cites.
Look around and refer: 2d synthetic semantics knowledge distillation for 3d visual grounding
Eslam Bakr, Yasmeen Alsaedy, and Mohamed Elhoseiny · 2022
Earlier work this paper cites.
3djcg: A unified framework for joint dense captioning and visual grounding on 3d point clouds
Daigang Cai, Lichen Zhao, Jing Zhang, Lu Sheng, and Dong Xu · 2022
Earlier work this paper cites.
Procthor: Large-scale embodied ai using procedural generation
Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi · 2022
Earlier work this paper cites.
Semantic abstraction: Open-world 3d scene understanding from 2d vision-language models
Huy Ha and Shuran Song · 2022
Earlier work this paper cites.
Bottom up top down detection transformers for language grounding in images and point clouds
Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, and Katerina Fragkiadaki · 2022
Cited alongside, same era.
Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi · 2022
Cited alongside, same era.
3d-sps: Single-stage 3d visual grounding via referred point progressive selection
Junyu Luo, Jiahui Fu, Xianghao Kong, Chen Gao, Haibing Ren, Hao Shen, Huaxia Xia, and Si Liu · 2022
Cited alongside, same era.
Multiscan: Scalable rgbd scanning for 3d environments with articulated objects
Yongsen Mao, Yiming Zhang, Hanxiao Jiang, Angel Chang, and Manolis Savva · 2022
Cited alongside, same era.
Laion-5b: An open large-scale dataset for training next generation image-text models
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al · 2022
Cited alongside, same era.
Eda: Explicit text-decoupling and dense alignment for 3d visual grounding
Yanmin Wu, Xinhua Cheng, Renrui Zhang, Zesen Cheng, and Jian Zhang · 2023
Later among the works it cites.
Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding
Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese · 2023
Later among the works it cites.
Regionplc: Regional point-language contrastive learning for open-world 3d scene understanding
Jihan Yang, Runyu Ding, Zhe Wang, and Xiaojuan Qi · 2023
Later among the works it cites.
Mmbench: Is your multi-modal model an all-around player?
Yuanhan Zhang et. al Yuan Liu, Haodong Duan · 2023
Later among the works it cites.
Sam3d: Segment anything in 3d scenes
Xiaoyang Wu et.al Yunhan Yang · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Softgroup for 3d instance segmentation on point clouds
Thang Vu, Kookhoi Kim, Tung M Luu, Thanh Nguyen, and Chang D Yoo · 2022
Cited alongside, same era.
3d question answering
Shuquan Ye, Dongdong Chen, Songfang Han, and Jing Liao · 2022
Cited alongside, same era.
X-trans2cap: Cross-modal knowledge transfer using transformer for 3d dense captioning
Zhihao Yuan, Xu Yan, Yinghong Liao, Yao Guo, Guanbin Li, Shuguang Cui, and Zhen Li · 2022
Cited alongside, same era.
Pointclip: Point cloud understanding by clip
Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li · 2022
Cited alongside, same era.
Instructblip: Towards general-purpose vision-language models with instruction tuning
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi · 2023
Cited alongside, same era.
Pla: Language-driven open-vocabulary 3d scene understanding
Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi · 2023
Cited alongside, same era.
Clip goes 3d: Leveraging prompt tuning for language grounded 3d recognition
Deepti Hegde, Jeya Maria Jose Valanarasu, and Vishal Patel · 2023
Cited alongside, same era.
Scanents3d: Exploiting phrase-to-3d-object correspondences for improved visio-linguistic models in 3d scenes
Ahmed Abdelreheem, Kyle Olszewski, Hsin-Ying Lee, Peter Wonka, and Panos Achlioptas · 2024
Later among the works it cites.
Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Deyao Zhu, Jian Ding, and Mohamed Elhoseiny · 2024
Later among the works it cites.
Hourvideo: 1-hour video-language understanding
Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristóbal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, and Li Fei-Fei · 2024
Later among the works it cites.
Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al · 2024
Later among the works it cites.
Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang · 2024
Later among the works it cites.
Sceneverse: Scaling 3d vision-language learning for grounded scene understanding
Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang · 2024
Later among the works it cites.
Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation
Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva · 2024
Later among the works it cites.
Lisa: Reasoning segmentation via large language model
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia · 2024
Later among the works it cites.
Llava-onevision: Easy visual task transfer
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li · 2024
Later among the works it cites.
Lexicon3d: Probing visual foundation models for complex 3d scene understanding
Yunze Man, Shuhong Zheng, Zhipeng Bao, Martial Hebert, Liang-Yan Gui, and Yu-Xiong Wang · 2024
Later among the works it cites.
Gpt-4o: A large language model by openai
OpenAI · 2024
Later among the works it cites.
Gemma 2: Improving open language models at a practical size
Gemma Team and Google DeepMind · 2024
Later among the works it cites.
Four ways to improve verbo-visual fusion for dense 3d visual grounding
Ozan Unal, Christos Sakaridis, Suman Saha, and Luc Van Gool · 2024
Later among the works it cites.
xAI, 2024
2024
Later among the works it cites.
Minicpm-v: A gpt-4v level mllm on your phone
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al · 2024
Later among the works it cites.
Agent3d-zero: An agent for zero-shot 3d understanding
Sha Zhang, Di Huang, Jiajun Deng, Shixiang Tang, Wanli Ouyang, Tong He, and Yanyong Zhang · 2024
Later among the works it cites.
Imagine while reasoning in space: Multimodal visualization-of-thought
Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei · 2025
Closest in time.
Thinking in Space: How Multimodal Large Language Models See, Remember and Recall Spaces
Jihan Yang, Shusheng Yang, Anjali Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie · 2025
Closest in time.
Video-3d llm: Learning position-aware video representation for 3d scene understanding
Duo Zheng, Shijia Huang, and Liwei Wang · 2025
Closest in time.