Fetching the paper…
Reading the bibliography…
Despite large-scale pretraining endowing models with language and vision reasoning capabilities, improving their spatial reasoning capability remains challenging due to the lack of data grounded in the 3D world.
Wordseye: An automatic text-to-scene conversion system
Bob Coyne and Richard Sproat · 2001
Earlier work this paper cites.
Real-time automatic 3d scene generation from natural language voice and text descriptions
Lee M Seversky and Lijun Yin · 2006
Earlier work this paper cites.
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Example-based synthesis of 3d object arrangements
Matthew Fisher, Daniel Ritchie, Manolis Savva, Thomas Funkhouser, and Pat Hanrahan · 2012
Earlier work this paper cites.
Learning spatial knowledge for text to 3d scene generation
Angel Chang, Manolis Savva, and Christopher D Manning · 2014
Earlier work this paper cites.
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick · 2014
Earlier work this paper cites.
The clutterpalette: An interactive tool for detailing indoor scenes
Lap-Fai Yu, Sai-Kit Yeung, and Demetri Terzopoulos · 2015
Earlier work this paper cites.
Sceneseer: 3d scene design with natural language
Angel X Chang, Mihail Eric, Manolis Savva, and Christopher D Manning · 2017
Earlier work this paper cites.
Language-driven synthesis of 3d scenes from scene databases
Rui Ma, Akshay Gadi Patil, Matthew Fisher, Manyi Li, Sören Pirk, Binh-Son Hua, Sai-Kit Yeung, Xin Tong, Leonidas Guibas, and Hao Zhang · 2018
Earlier work this paper cites.
Object hallucination in image captioning
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko · 2018
Earlier work this paper cites.
Horizonnet: Learning room layout with 1d representation and pano stretch data augmentation
Cheng Sun, Chi-Wei Hsiao, Min Sun, and Hwann-Tzong Chen · 2019
Earlier work this paper cites.
Rel3D: A minimally contrastive benchmark for grounding spatial relations in 3d
Ankit Goyal, Kaiyu Yang, Dawei Yang, and Jia Deng · 2020
Earlier work this paper cites.
Atiss: Autoregressive transformers for indoor scene synthesis
Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler · 2021
Earlier work this paper cites.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Earlier work this paper cites.
Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding
Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind · 2021
Earlier work this paper cites.
Procthor: Large-scale embodied ai using procedural generation
Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi · 2022
Earlier work this paper cites.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Cited alongside, same era.
Objaverse: A universe of annotated 3d objects
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi · 2023
Cited alongside, same era.
Diffusion360: Seamless 360 degree panoramic image generation based on diffusion models
Mengyang Feng, Jinlin Liu, Miaomiao Cui, and Xuansong Xie · 2023
Cited alongside, same era.
Text2Room: Extracting textured 3D meshes from 2D text-to-image models
Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner · 2023
Cited alongside, same era.
3D-LLM: Injecting the 3D world into large language models
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan · 2023
Training language models to self-correct via reinforcement learning
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al · 2024
Later among the works it cites.
InstructScene: Instruction-driven 3D indoor scene synthesis with semantic graph prior
Chenguo Lin and Yadong Mu · 2024
Later among the works it cites.
Material palette: Extraction of materials from a single image
Ivan Lopes, Fabio Pizzati, and Raoul de Charette · 2024
Later among the works it cites.
RoboCasa: Large-scale simulation of everyday tasks for generalist robots
Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Large language models cannot self-correct reasoning yet
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou · 2023
Cited alongside, same era.
Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4v
Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao · 2023
Cited alongside, same era.
SceneWiz3D: Towards text-guided 3d scene composition
Qihang Zhang, Chaoyang Wang, Aliaksandr Siarohin, Peiye Zhuang, Yinghao Xu, Ceyuan Yang, Dahua Lin, Bolei Zhou, Sergey Tulyakov, and Hsin-Ying Lee · 2023
Cited alongside, same era.
Open-universe indoor scene generation using llm program synthesis and uncurated object databases
Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, R Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, and Daniel Ritchie · 2024
Cited alongside, same era.
I-design: Personalized llm interior designer
Ata Çelen, Guo Han, Konrad Schindler, Luc Van Gool, Iro Armeni, Anton Obukhov, and Xi Wang · 2024
Cited alongside, same era.
Spatialrgpt: Grounded spatial reasoning in vision language model
An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu · 2024
Cited alongside, same era.
Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al · 2024
Cited alongside, same era.
NVIDIA, Maciej Bala, Yin Cui, Yifan Ding, Yunhao Ge, Zekun Hao, Jon Hasselgren, Jacob Huffman, Jingyi Jin, J.P. Lewis, Zhaoshuo Li, Chen-Hsuan Lin, Yen-Chen Lin, Tsung-Yi Lin, Ming-Yu Liu, Alice Luo, Qianli Ma, Jacob Munkberg, Stella Shi, Fangyin Wei, Donglai Xiang, Jiashu Xu, Xiaohui Zeng, and Qinsheng Zhang · 2024
Later among the works it cites.
Advances in data-driven analysis and synthesis of 3d indoor scenes
Akshay Gadi Patil, Supriya Gadi Patil, Manyi Li, Matthew Fisher, Manolis Savva, and Hao Zhang · 2024
Later among the works it cites.
Compositional 3d scene generation using locally conditioned diffusion
Ryan Po and Gordon Wetzstein · 2024
Later among the works it cites.
Lay-a-scene: Personalized 3d object arrangement using text-to-image priors
Ohad Rahamim, Hilit Segev, Idan Achituve, Yuval Atzmon, Yoni Kasten, and Gal Chechik · 2024
Later among the works it cites.
Grounded sam: Assembling open-world models for diverse visual tasks, 2024
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang · 2024
Later among the works it cites.
Controlroom3d: Room generation using semantic proxy rooms
Jonas Schult, Sam Tsai, Lukas Höllein, Bichen Wu, Jialiang Wang, Chih-Yao Ma, Kunpeng Li, Xiaofang Wang, Felix Wimbauer, Zijian He, et al · 2024
Later among the works it cites.
Florence-2: Advancing a unified representation for a variety of vision tasks
Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan · 2024
Later among the works it cites.
Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan · 2024
Later among the works it cites.
RLF-V: Towards trustworthy MLLMs via behavior alignment from fine-grained correctional human feedback
Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al · 2024
Later among the works it cites.
Clay: A controllable large-scale generative model for creating high-quality 3d assets
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu · 2024
Later among the works it cites.
GALA3D: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting
Xiaoyu Zhou, Xingjian Ran, Yajiao Xiong, Jinlin He, Zhiwei Lin, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang · 2024
Later among the works it cites.
DreamScene360: Unconstrained text-to-3D scene generation with panoramic gaussian splatting
Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, and Achuta Kadambi · 2025
Closest in time.