Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Multi-concept customization of text-to-image diffusion

Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. 2023 · 1941

Earlier work this paper cites.

3dsnet: Unsupervised shape-to-shape 3d style transfer

Original

Mattia Segu, Margarita Grinvald, Roland Siegwart, and Federico Tombari. 2020 · 2011

Earlier work this paper cites.

Analogy-driven 3d style transfer

Chongyang Ma, Haibin Huang, Alla Sheffer, Evangelos Kalogerakis, and Rui Wang. 2014 · 2014

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019

Earlier work this paper cites.

Improving image captioning with better use of caption

Zhan Shi, Xu Zhou, Xipeng Qiu, and Xiaodan Zhu. 2020 · 2020

Earlier work this paper cites.

Alfworld: Aligning text and embodied environments for interactive learning

Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Cote, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2020 · 2020

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021 · 2021

Earlier work this paper cites.

Zero-shot text-to-image generation

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021 · 2021

Earlier work this paper cites.

Training-free structured diffusion guidance for compositional text-to-image synthesis

Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. 2022 · 2022

Earlier work this paper cites.

Toist: Task oriented instance segmentation transformer with noun-pronoun distillation

Pengfei Li, Beiwen Tian, Yongliang Shi, Xiaoxue Chen, Hao Zhao, Guyue Zhou, and Ya-Qin Zhang. 2022 · 2022

Earlier work this paper cites.

Dreamfusion: Text-to-3d using 2d diffusion

Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. 2022 · 2022

Earlier work this paper cites.

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022 · 2022

Earlier work this paper cites.

Scaling autoregressive models for content-rich text-to-image generation

Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, Benton C. Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. 2022 · 2022

Earlier work this paper cites.

Training diffusion models with reinforcement learning

Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. 2023 · 2023

Earlier work this paper cites.

Instructpix2pix: Learning to follow image editing instructions

Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2023 · 2023

Earlier work this paper cites.

Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models

Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. 2023 · 2023

Earlier work this paper cites.

Chatcot: Tool-augmented chain-of-thought reasoning on chat-based large language models

Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Wayne Xin Zhao, and Ji-Rong Wen. 2023c · 2023

Earlier work this paper cites.

Sdfusion: Multimodal 3d shape completion, reconstruction, and generation

Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alexander G Schwing, and Liang-Yan Gui. 2023 · 2023

Earlier work this paper cites.

Mindagent: Emergent gaming interaction

Original

Ran Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-Chun Zhu, Demetri Terzopoulos, Li Fei-Fei, et al. 2023 · 2023

Earlier work this paper cites.

Critic: Large language models can self-correct with tool-interactive critiquing

Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Nan Duan, Weizhu Chen, et al. 2023 · 2023

Earlier work this paper cites.

threestudio: A unified framework for 3d content generation

Yuan-Chen Guo, Ying-Tian Liu, Ruizhi Shao, Christian Laforte, Vikram Voleti, Guan Luo, Chia-Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, and Song-Hai Zhang. 2023 · 2023

Earlier work this paper cites.

Visual programming: Compositional visual reasoning without training

Tanmay Gupta and Aniruddha Kembhavi. 2023 · 2023

Earlier work this paper cites.

An embodied generalist agent in 3d world

Original

Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. 2023 · 2023

Earlier work this paper cites.

Adapt: Action-aware driving caption transformer

Bu Jin, Xinyu Liu, Yupeng Zheng, Pengfei Li, Hao Zhao, Tong Zhang, Yuhang Zheng, Guyue Zhou, and Jingjing Liu. 2023 · 2023

Cited alongside, same era.

Imagic: Text-based real image editing with diffusion models

Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. 2023 · 2023

Cited alongside, same era.

Deepfloyd if

DeepFloyd Lab. 2023 · 2023

Cited alongside, same era.

Platypus: Quick, cheap, and powerful refinement of llms

Ariel Lee, Cole Hunter, and Nataniel Ruiz. 2023 · 2023

Cited alongside, same era.

Understanding embodied reference with touch-line transformer

Yang Li, Xiaoxue Chen, Hao Zhao, Jiangtao Gong, Guyue Zhou, Federico Rossano, and Yixin Zhu. 2023 · 2023

Cited alongside, same era.

Magic3d: High-resolution text-to-3d content creation

Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. 2023 · 2023

Unsupervised road anomaly detection with language anchors

Beiwen Tian, Mingdao Liu, Huan-ang Gao, Pengfei Li, Hao Zhao, and Guyue Zhou. 2023 · 2023

Later among the works it cites.

Textmesh: Generation of realistic 3d meshes from text prompts

Original

Christina Tsalicoglou, Fabian Manhardt, Alessio Tonioni, Michael Niemeyer, and Federico Tombari. 2023 · 2023

Later among the works it cites.

Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation

Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. 2023 · 2023

Later among the works it cites.

Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation

Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. 2023 · 2023

Later among the works it cites.

The rise and potential of large language model based agents: A survey

Original

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023 · 2023