Fetching the paper…

Reading the bibliography…

2023

Generative AI meets 3D: A Survey on Text-to-3D in AIGC Era

Li, Chenghao, Zhang, Chaoning, Cho, Joseph et al.

Original
PDF
Post on X

Understand

Generative AI has made significant progress in recent years, with text-guided content generation being the most practical as it facilitates interaction between human instructions and AI-generated content (AIGC).

Thanks to advancements in text-to-image and 3D modeling technologies, like neural radiance field (NeRF), text-to-3D has emerged as a nascent yet highly active research field.
Our work conducts a comprehensive survey on this topic and follows up on subsequent research progress in the overall field, aiming to help readers interested in this direction quickly catch up with its rapid development.
First, we introduce 3D data representations, including both Structured and non-Structured data.

Generative AI meets 3D: A Survey on Text-to-3D in AIGC Era · Around

Built on

L. W. Peng and S. M. Shamsuddin, “3d object reconstruction and representation using neural networks,” in Proceedings of the 2nd international conference on Computer graphics and interactive techniques in Australasia and South East Asia , 2004, pp. 139–147
2004
Earlier work this paper cites.
J. F. Blinn, “What is a pixel?” IEEE computer graphics and applications , vol. 25, no. 5, pp. 82–87, 2005
2005
Earlier work this paper cites.
B. Cyganek and J. P. Siebert, An introduction to 3D computer vision techniques and algorithms . John Wiley & Sons, 2011
2011
Earlier work this paper cites.
Y. Furukawa, C. Hernández et al. , “Multi-view stereo: A tutorial,” Foundations and Trends® in Computer Graphics and Vision , vol. 9, no. 1-2, pp. 1–148, 2015
2015
Earlier work this paper cites.
F. Bogo, A. Kanazawa, C. Lassner, P. Gehler, J. Romero, and M. J. Black, “Keep it smpl: Automatic estimation of 3d human pose and shape from a single image,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14 . Springer, 2016, pp. 561–578
2016
Earlier work this paper cites.
C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 652–660
2017
Earlier work this paper cites.
E. Ahmed, A. Saint, A. E. R. Shabayek, K. Cherenkova, R. Das, G. Gusev, D. Aouada, and B. Ottersten, “A survey on deep learning advances on different 3d data representations,” arXiv preprint arXiv:1808.01462 , 2018
Original
2018
Earlier work this paper cites.
L. Minto, P. Zanuttigh, and G. Pagnutti, “Deep learning for 3d shape classification based on volumetric density and surface approximation clues.” in VISIGRAPP (5: VISAPP) , 2018, pp. 317–324
2018
Earlier work this paper cites.
B. Zhao, X. Wu, Z.-Q. Cheng, H. Liu, Z. Jie, and J. Feng, “Multi-view image generation from a single-view,” in Proceedings of the 26th ACM international conference on Multimedia , 2018, pp. 383–391
2018
Earlier work this paper cites.
A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748 , 2018
Original
2018
Earlier work this paper cites.
K. Chen, C. B. Choy, M. Savva, A. X. Chang, T. Funkhouser, and S. Savarese, “Text2shape: Generating shapes from natural language by learning joint embeddings,” in Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III 14 . Springer, 2019, pp. 100–116
2019
Earlier work this paper cites.
P. Achlioptas, J. Fan, R. Hawkins, N. Goodman, and L. J. Guibas, “Shapeglot: Learning language for shape differentiation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 8938–8947
2019
Earlier work this paper cites.
J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, “Deepsdf: Learning continuous signed distance functions for shape representation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 165–174
2019
Earlier work this paper cites.
R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, and D. Cohen-Or, “Meshcnn: a network with an edge,” ACM Transactions on Graphics (TOG) , vol. 38, no. 4, pp. 1–12, 2019
2019
Earlier work this paper cites.
L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger, “Occupancy networks: Learning 3d reconstruction in function space,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 4460–4470
2019
Earlier work this paper cites.
C. Wang, M. Cheng, F. Sohel, M. Bennamoun, and J. Li, “Normalnet: A voxel-based cnn for 3d object classification and retrieval,” Neurocomputing , vol. 323, pp. 139–147, 2019
2019
Earlier work this paper cites.
W. Liu, J. Sun, W. Li, T. Hu, and P. Wang, “Deep learning on point clouds and its application: A survey,” Sensors , vol. 19, no. 19, p. 4188, 2019
2019
Earlier work this paper cites.
G. Pavlakos, V. Choutas, N. Ghorbani, T. Bolkart, A. A. Osman, D. Tzionas, and M. J. Black, “Expressive body capture: 3d hands, face, and body from a single image,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 10 975–10 985
2019
Earlier work this paper cites.
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in Neural Information Processing Systems , vol. 33, pp. 6840–6851, 2020
2020
Earlier work this paper cites.
Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3d point clouds: A survey,” IEEE transactions on pattern analysis and machine intelligence , vol. 43, no. 12, pp. 4338–4364, 2020
2020
Earlier work this paper cites.
Y. Jin, D. Jiang, and M. Cai, “3d reconstruction using deep learning: a survey,” Communications in Information and Systems , vol. 20, no. 4, pp. 389–413, 2020
2020
Earlier work this paper cites.
W. Cao, Z. Yan, Z. He, and Z. He, “A comprehensive survey on geometric deep learning,” IEEE Access , vol. 8, pp. 35 929–35 949, 2020
2020
Earlier work this paper cites.
K. Rematas and V. Ferrari, “Neural voxel renderer: Learning an accurate and controllable rendering tool,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 5417–5427
2020
Earlier work this paper cites.
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE transactions on neural networks and learning systems , vol. 32, no. 1, pp. 4–24, 2020
2020
Earlier work this paper cites.
Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv preprint arXiv:2011.13456 , 2020
Original
2020
Earlier work this paper cites.
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 9729–9738
2020
Earlier work this paper cites.
R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, “Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,” IEEE transactions on pattern analysis and machine intelligence , vol. 44, no. 3, pp. 1623–1637, 2020
2020
Earlier work this paper cites.
B. Kye, N. Han, E. Kim, Y. Park, and S. Jo, “Educational applications of metaverse: possibilities and limitations,” Journal of educational evaluation for health professions , vol. 18, 2021
2021
Earlier work this paper cites.
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM , vol. 65, no. 1, pp. 99–106, 2021
2021
Earlier work this paper cites.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763
2021
Earlier work this paper cites.
H. Zhou, W. Zhang, K. Chen, W. Li, and N. Yu, “Three-dimensional mesh steganography and steganalysis: a review,” IEEE Transactions on Visualization and Computer Graphics , 2021
2021
Earlier work this paper cites.
T. Shen, J. Gao, K. Yin, M.-Y. Liu, and S. Fidler, “Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis,” Advances in Neural Information Processing Systems , vol. 34, pp. 6087–6101, 2021
2021
Earlier work this paper cites.
P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021
2021
Earlier work this paper cites.
A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” arXiv preprint arXiv:2112.10741 , 2021
Original
2021
Earlier work this paper cites.
R. Ranftl, A. Bochkovskiy, and V. Koltun, “Vision transformers for dense prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 12 179–12 188
2021
Earlier work this paper cites.
M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 9650–9660
2021
Earlier work this paper cites.
L.-H. Lee, T. Braud, P. Zhou, L. Wang, D. Xu, Z. Lin, A. Kumar, C. Bermejo, and P. Hui, “All one needs to know about metaverse: A complete survey on technological singularity, virtual ecosystem, and research agenda,” arXiv preprint arXiv:2110.05352 , 2021
Original
2021
Earlier work this paper cites.
A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125 , 2022
Original
2022
Earlier work this paper cites.
Q. Yang, Y. Zhao, H. Huang, Z. Xiong, J. Kang, and Z. Zheng, “Fusing blockchain and ai with metaverse: A survey,” IEEE Open Journal of the Computer Society , vol. 3, pp. 122–136, 2022
2022
Earlier work this paper cites.
Z. Shi, S. Peng, Y. Xu, Y. Liao, and Y. Shen, “Deep generative models on 3d representations: A survey,” arXiv preprint arXiv:2210.15663 , 2022
Original
2022
Earlier work this paper cites.
H. Jeon, H. Youn, S. Ko, and T. Kim, “Blockchain and ai meet in the metaverse,” Advances in the Convergence of Blockchain and Artificial Intelligence , vol. 73, no. 10.5772, 2022
2022
Earlier work this paper cites.
R. Fu, X. Zhan, Y. Chen, D. Ritchie, and S. Sridhar, “Shapecrafter: A recursive text-conditioned 3d shape generation model,” arXiv preprint arXiv:2207.09446 , 2022
Original
2022
Earlier work this paper cites.
C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman et al. , “Laion-5b: An open large-scale dataset for training next generation image-text models,” arXiv preprint arXiv:2210.08402 , 2022
Original
2022
Earlier work this paper cites.
A. Jain, B. Mildenhall, J. T. Barron, P. Abbeel, and B. Poole, “Zero-shot text-guided object generation with dream fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 867–876
2022
Earlier work this paper cites.
J. Xu, X. Wang, W. Cheng, Y.-P. Cao, Y. Shan, X. Qie, and S. Gao, “Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models,” arXiv preprint arXiv:2212.14704 , 2022
Original
2022
Earlier work this paper cites.
C. Wang, M. Chai, M. He, D. Chen, and J. Liao, “Clip-nerf: Text-and-image driven manipulation of neural radiance fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 3835–3844
2022
Earlier work this paper cites.
B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, “Dreamfusion: Text-to-3d using 2d diffusion,” arXiv preprint arXiv:2209.14988 , 2022
Original
2022
Earlier work this paper cites.
H. Wang, X. Du, J. Li, R. A. Yeh, and G. Shakhnarovich, “Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation,” arXiv preprint arXiv:2212.00774 , 2022
Original
2022
Earlier work this paper cites.
Y. Dong, S. Ruan, H. Su, C. Kang, X. Wei, and J. Zhu, “Viewfool: Evaluating the robustness of visual recognition to adversarial viewpoints,” arXiv preprint arXiv:2210.03895 , 2022
Original
2022
Earlier work this paper cites.
H. Wang and J. Zhang, “A survey of deep learning-based mesh processing,” Communications in Mathematics and Statistics , vol. 10, no. 1, pp. 163–194, 2022
2022
Earlier work this paper cites.
T. Müller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” ACM transactions on graphics (TOG) , vol. 41, no. 4, pp. 1–15, 2022
2022
Earlier work this paper cites.
E. R. Chan, C. Z. Lin, M. A. Chan, K. Nagano, B. Pan, S. De Mello, O. Gallo, L. J. Guibas, J. Tremblay, S. Khamis et al. , “Efficient geometry-aware 3d generative adversarial networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 16 123–16 133
2022
Earlier work this paper cites.
J. Gao, T. Shen, Z. Wang, W. Chen, K. Yin, D. Li, O. Litany, Z. Gojcic, and S. Fidler, “Get3d: A generative model of high quality 3d textured shapes learned from images,” Advances In Neural Information Processing Systems , vol. 35, pp. 31 841–31 854, 2022
2022
Earlier work this paper cites.
B. Zhang, M. Nießner, and P. Wonka, “3dilg: Irregular latent grids for 3d generative modeling,” Advances in Neural Information Processing Systems , vol. 35, pp. 21 871–21 885, 2022
2022
Earlier work this paper cites.
K. Gao, Y. Gao, H. He, D. Lu, L. Xu, and J. Li, “Nerf: Neural radiance field in 3d vision, a comprehensive review,” arXiv preprint arXiv:2210.00379 , 2022
Original
2022
Earlier work this paper cites.
M. A. Bautista, P. Guo, S. Abnar, W. Talbott, A. Toshev, Z. Chen, L. Dinh, S. Zhai, H. Goh, D. Ulbricht et al. , “Gaudi: A neural architect for immersive 3d scene generation,” Advances in Neural Information Processing Systems , vol. 35, pp. 25 102–25 116, 2022
2022
Earlier work this paper cites.
C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al. , “Photorealistic text-to-image diffusion models with deep language understanding,” Advances in Neural Information Processing Systems , vol. 35, pp. 36 479–36 494, 2022
2022
Earlier work this paper cites.
N. Mohammad Khalid, T. Xie, E. Belilovsky, and T. Popa, “Clip-mesh: Generating textured meshes from text using pretrained image-text models,” in SIGGRAPH Asia 2022 Conference Papers , 2022, pp. 1–8
2022
Earlier work this paper cites.
C.-H. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng, X. Huang, K. Kreis, S. Fidler, M.-Y. Liu, and T.-Y. Lin, “Magic3d: High-resolution text-to-3d content creation,” arXiv preprint arXiv:2211.10440 , 2022
Original
2022
Earlier work this paper cites.
A. Sanghi, R. Fu, V. Liu, K. Willis, H. Shayani, A. Khasahmadi, S. Sridhar, and D. Ritchie, “Clip-sculptor: Zero-shot generation of high-fidelity and diverse shapes from natural language,” arXiv preprint arXiv:2211.01427 , Nov 2022
Original
2022
Earlier work this paper cites.
G. Metzer, E. Richardson, O. Patashnik, R. Giryes, and D. Cohen-Or, “Latent-nerf for shape-guided generation of 3d shapes and textures,” arXiv preprint arXiv:2211.07600 , 2022
Original
2022
Earlier work this paper cites.
O. Michel, R. Bar-On, R. Liu, S. Benaim, and R. Hanocka, “Text2mesh: Text-driven neural stylization for meshes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 13 492–13 502
2022
Earlier work this paper cites.
G. Tevet, B. Gordon, A. Hertz, A. H. Bermano, and D. Cohen-Or, “Motionclip: Exposing human motion generation to clip space,” in Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII . Springer, 2022, pp. 358–374
2022
Earlier work this paper cites.
F. Hong, M. Zhang, L. Pan, Z. Cai, L. Yang, and Z. Liu, “Avatarclip: Zero-shot text-driven generation and animation of 3d avatars,” arXiv preprint arXiv:2205.08535 , 2022
Original
2022
Earlier work this paper cites.
U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni et al. , “Make-a-video: Text-to-video generation without text-video data,” arXiv preprint arXiv:2209.14792 , 2022
Original
2022
Earlier work this paper cites.
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 10 684–10 695
2022
Earlier work this paper cites.
Y. Chen, R. Chen, J. Lei, Y. Zhang, and K. Jia, “Tango: Text-driven photorealistic and robust 3d stylization via lighting decomposition,” arXiv preprint arXiv:2210.11277 , 2022
Original
2022
Earlier work this paper cites.
T. Brooks, A. Holynski, and A. A. Efros, “Instructpix2pix: Learning to follow image editing instructions,” arXiv preprint arXiv:2211.09800 , 2022
Original
2022
Earlier work this paper cites.
C. Zhang, C. Zhang, C. Li, Y. Qiao, S. Zheng, S. K. Dam, M. Zhang, J. U. Kim, S. T. Kim, J. Choi et al. , “One small step for generative ai, one giant leap for agi: A complete survey on chatgpt in aigc era,” arXiv preprint arXiv:2304.06488 , 2023
Original
2023
Earlier work this paper cites.
C. Zhang, C. Zhang, M. Zhang, and I. S. Kweon, “Text-to-image diffusion model in generative ai: A survey,” arXiv preprint arXiv:2303.07909 , 2023
Original
2023
Earlier work this paper cites.

Similar

C. Zhang, C. Zhang, S. Zheng, Y. Qiao, C. Li, M. Zhang, S. K. Dam, C. M. Thwal, Y. L. Tun, L. L. Huy et al. , “A complete survey on generative ai (aigc): Is chatgpt from gpt-4 to gpt-5 all you need?” arXiv preprint arXiv:2303.11717 , 2023
Original
2023
Cited alongside, same era.
H. Wang, H. Ning, Y. Lin, W. Wang, S. Dhelim, F. Farha, J. Ding, and M. Daneshmand, “A survey on the metaverse: The state-of-the-art, technologies, applications, and challenges,” IEEE Internet of Things Journal , vol. 10, no. 16, pp. 14 671–14 688, 2023
2023
Cited alongside, same era.
Z. Lv, “Generative artificial intelligence in the metaverse era,” Cognitive Robotics , vol. 3, pp. 208–217, 2023
2023
Cited alongside, same era.
W. Xia and J.-H. Xue, “A survey on deep generative 3d-aware image synthesis,” ACM Computing Surveys , vol. 56, no. 4, pp. 1–34, 2023
2023
Cited alongside, same era.

Then

C. Li, C. Zhang, A. Waghwase, L.-H. Lee, F. Rameau, Y. Yang, S.-H. Bae, and C. S. Hong, “Generative ai meets 3d: A survey on text-to-3d in aigc era,” arXiv preprint arXiv:2305.06131 , 2023
Original
2023
Closest in time.
R. Fridman, A. Abecasis, Y. Kasten, and T. Dekel, “Scenescape: Text-driven consistent scene generation,” arXiv preprint arXiv:2302.01133 , 2023
Original
2023
Closest in time.
R. Po and G. Wetzstein, “Compositional 3d scene generation using locally conditioned diffusion,” arXiv preprint arXiv:2303.12218 , 2023
Original
2023
Closest in time.
U. Singer, S. Sheynin, A. Polyak, O. Ashual, I. Makarov, F. Kokkinos, N. Goyal, A. Vedaldi, D. Parikh, J. Johnson et al. , “Text-to-4d dynamic scene generation,” arXiv preprint arXiv:2301.11280 , 2023
Original
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

H. Wu, C. Wen, S. Shi, X. Li, and C. Wang, “Virtual sparse convolution for multimodal 3d object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 21 653–21 662

2023

Cited alongside, same era.

D. Hu, Z. Zhang, T. Hou, T. Liu, H. Fu, and M. Gong, “Multiscale representation for real-time anti-aliasing neural rendering,” arXiv preprint arXiv:2304.10075 , 2023

2023

Cited alongside, same era.

X. Yu, M. Xu, Y. Zhang, H. Liu, C. Ye, Y. Wu, Z. Yan, C. Zhu, Z. Xiong, T. Liang et al. , “Mvimgnet: A large-scale dataset of multi-view images,” arXiv preprint arXiv:2303.06042 , 2023

2023

Cited alongside, same era.

B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,” 2023

2023

Cited alongside, same era.

J. R. Shue, E. R. Chan, R. Po, Z. Ankner, J. Wu, and G. Wetzstein, “3d neural field generation using triplane diffusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 20 875–20 886

2023

Cited alongside, same era.

B. Han, Y. Liu, and Y. Shen, “Zero3d: Semantic-driven multi-category 3d shape generation,” arXiv preprint arXiv:2301.13591 , 2023

2023

Cited alongside, same era.

L. Zhou, A. Shih, C. Meng, and S. Ermon, “Dreampropeller: Supercharge text-to-3d generation with parallel sampling,” arXiv preprint arXiv:2311.17082 , 2023

2023

Cited alongside, same era.

S. Hong, D. Ahn, and S. Kim, “Debiasing scores and prompts of 2d diffusion for robust text-to-3d generation,” arXiv preprint arXiv:2303.15413 , 2023

2023

Cited alongside, same era.

Y. Lin, H. Bai, S. Li, H. Lu, X. Lin, H. Xiong, and L. Wang, “Componerf: Text-guided multi-object compositional nerf with editable 3d scene layout,” arXiv preprint arXiv:2303.13843 , 2023

2023

Closest in time.

D. Cohen-Bar, E. Richardson, G. Metzer, R. Giryes, and D. Cohen-Or, “Set-the-scene: Global-local training for generating controllable nerf scenes,” arXiv preprint arXiv:2303.13450 , 2023

2023

Closest in time.

M. Gao, Y. Xu, Y. Zhao, T. Hou, C. Zhao, and M. Gong, “Clip3dstyler: Language guided 3d arbitrary neural style transfer,” arXiv preprint arXiv:2305.15732 , 2023

2023

Closest in time.

E. Richardson, G. Metzer, Y. Alaluf, R. Giryes, and D. Cohen-Or, “Texture: Text-guided texturing of 3d shapes,” arXiv preprint arXiv:2302.01721 , 2023

2023

Closest in time.

T. Cao, K. Kreis, S. Fidler, N. Sharp, and K. Yin, “Texfusion: Synthesizing 3d textures with text-guided image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 4169–4181

2023

Closest in time.

D. Z. Chen, Y. Siddiqui, H.-Y. Lee, S. Tulyakov, and M. Nießner, “Text2tex: Text-driven texture synthesis via diffusion models,” arXiv preprint arXiv:2303.11396 , 2023

2023

Closest in time.

S. Aneja, J. Thies, A. Dai, and M. Nießner, “Clipface: Text-guided editing of textured 3d morphable models,” in ACM SIGGRAPH 2023 Conference Proceedings , 2023, pp. 1–11

2023

Closest in time.

R. Shao, J. Sun, C. Peng, Z. Zheng, B. Zhou, H. Zhang, and Y. Liu, “Control4d: Dynamic portrait editing by learning 4d gan from 2d diffusion-based editor,” arXiv preprint arXiv:2305.20082 , 2023

2023

Closest in time.

J. Zhuang, C. Wang, L. Lin, L. Liu, and G. Li, “Dreameditor: Text-driven 3d scene editing with neural fields,” in SIGGRAPH Asia 2023 Conference Papers , 2023, pp. 1–10

2023

Closest in time.

J. Park, G. Kwon, and J. C. Ye, “Ed-nerf: Efficient text-guided editing of 3d scene using latent space nerf,” arXiv preprint arXiv:2310.02712 , 2023

2023

Closest in time.

J. Fang, J. Wang, X. Zhang, L. Xie, and Q. Tian, “Gaussianeditor: Editing 3d gaussians delicately with text instructions,” arXiv preprint arXiv:2311.16037 , 2023

2023

Closest in time.

H. Kamata, Y. Sakuma, A. Hayakawa, M. Ishii, and T. Narihira, “Instruct 3d-to-3d: Text instruction guided 3d-to-3d conversion,” arXiv preprint arXiv:2303.15780 , 2023

2023

Closest in time.

J. Xu, X. Wang, Y.-P. Cao, W. Cheng, Y. Shan, and S. Gao, “Instructp2p: Learning to edit 3d point clouds with text instructions,” arXiv preprint arXiv:2306.07154 , 2023

2023

Closest in time.

A. Haque, M. Tancik, A. A. Efros, A. Holynski, and A. Kanazawa, “Instruct-nerf2nerf: Editing 3d scenes with instructions,” arXiv preprint arXiv:2303.12789 , 2023

2023

Closest in time.

D. Wang, T. Zhang, A. Abboud, and S. Süsstrunk, “Inpaintnerf360: Text-guided 3d inpainting on unbounded neural radiance fields,” arXiv preprint arXiv:2305.15094 , 2023

2023

Closest in time.

X. Cheng, T. Yang, J. Wang, Y. Li, L. Zhang, J. Zhang, and L. Yuan, “Progressive3d: Progressively local editing for text-to-3d content creation with complex semantic prompts,” arXiv preprint arXiv:2310.11784 , 2023

2023

Closest in time.

A. Mikaeili, O. Perel, D. Cohen-Or, and A. Mahdavi-Amiri, “Sked: Sketch-guided text-based 3d editing,” arXiv preprint arXiv:2303.10735 , 2023

2023

Closest in time.

W. Gao, N. Aigerman, T. Groueix, V. G. Kim, and R. Hanocka, “Textdeformer: Geometry manipulation using text guidance,” arXiv preprint arXiv:2304.13348 , 2023

2023

Closest in time.

E. Sella, G. Fiebelman, P. Hedman, and H. Averbuch-Elor, “Vox-e: Text-guided voxel editing of 3d objects,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 430–440

2023

Closest in time.

Y. He, Y. Bai, M. Lin, W. Zhao, Y. Hu, J. Sheng, R. Yi, J. Li, and Y.-J. Liu, “T3 bench: Benchmarking current progress in text-to-3d generation,” arXiv preprint arXiv:2310.02977 , 2023

2023

Closest in time.

H. Zohny, J. McMillan, and M. King, “Ethics of generative ai,” pp. 79–80, 2023

2023

Closest in time.

K. Wach, C. D. Duong, J. Ejdys, R. Kazlauskaitė, P. Korzynski, G. Mazurek, J. Paliszkiewicz, and E. Ziemba, “The dark side of generative artificial intelligence: A critical analysis of controversies and risks of chatgpt,” Entrepreneurial Business and Economics Review , vol. 11, no. 2, pp. 7–30, 2023

2023

Closest in time.

J. Liu, X. Huang, T. Huang, L. Chen, Y. Hou, S. Tang, Z. Liu, W. Ouyang, W. Zuo, J. Jiang et al. , “A comprehensive survey on 3d content generation,” arXiv preprint arXiv:2402.01166 , 2024

2024

Closest in time.

X. Li, Q. Zhang, D. Kang, W. Cheng, Y. Gao, J. Zhang, Z. Liang, J. Liao, Y.-P. Cao, and Y. Shan, “Advances in 3d generation: A survey,” arXiv preprint arXiv:2401.17807 , 2024

2024

Closest in time.

C. Jiang, “A survey on text-to-3d contents generation in the wild,” arXiv preprint arXiv:2405.09431 , 2024

2024

Closest in time.

H. Lee, M. Savva, and A. X. Chang, “Text-to-3d shape generation,” in Computer Graphics Forum . Wiley Online Library, 2024, p. e15061

2024

Closest in time.

S. Chen, X. Chen, A. Pang, X. Zeng, W. Cheng, Y. Fu, F. Yin, Y. Wang, Z. Wang, C. Zhang, J. Yu, G. Yu, B. Fu, and T. Chen, “Meshxl: Neural coordinate field for generative 3d foundation models,” arXiv preprint arXiv:2405.20853 , 2024

2024

Closest in time.

R. Yan, K. Wu, and K. Ma, “Flow score distillation for diverse text-to-3d generation,” arXiv preprint arXiv:2405.10988 , 2024

2024

Closest in time.

F. Hong, J. Tang, Z. Cao, M. Shi, T. Wu, Z. Chen, T. Wang, L. Pan, D. Lin, and Z. Liu, “3dtopia: Large text-to-3d generation model with hybrid diffusion priors,” arXiv preprint arXiv:2403.02234 , 2024

2024

Closest in time.

Z. Wu, P. Zhou, X. Yi, X. Yuan, and H. Zhang, “Consistent3d: Towards consistent high-fidelity text-to-3d generation with deterministic sampling prior,” arXiv preprint arXiv:2401.09050 , 2024

2024

Closest in time.

S. Kim, K. Li, X. Deng, Y. Shi, M. Cho, and P. Wang, “Enhancing 3d fidelity of text-to-3d using cross-view correspondences,” arXiv preprint arXiv:2404.10603 , 2024

2024

Closest in time.

X. Miao, H. Duan, V. Ojha, J. Song, T. Shah, Y. Long, and R. Ranjan, “Dreamer xl: Towards high-resolution text-to-3d generation via trajectory score matching,” arXiv preprint arXiv:2405.11252 , 2024

2024

Closest in time.

Y. Zhang, X. Miao, H. Duan, B. Wei, T. Shah, Y. Long, and R. Ranjan, “Exactdreamer: High-fidelity text-to-3d content creation via exact score matching,” arXiv preprint arXiv:2405.15914 , 2024

2024

Closest in time.

X. Li, J. Mo, Y. Wang, C. Parameshwara, X. Fei, A. Swaminathan, C. Taylor, Z. Tu, P. Favaro, and S. Soatto, “Grounded compositional and diverse text-to-3d with pretrained multi-view diffusion model,” arXiv preprint arXiv:2404.18065 , 2024

Original

2024

Closest in time.

J. Wu, X. Gao, X. Liu, Z. Shen, C. Zhao, H. Feng, J. Liu, and E. Ding, “Hd-fusion: Detailed text-to-3d generation leveraging multiple noise estimation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 3202–3211

2024

Closest in time.

D. Wang, T. Zhang, A. Abboud, and S. Süsstrunk, “Innerf360: Text-guided 3d-consistent object inpainting on 360-degree neural radiance fields,” arXiv preprint arXiv:2305.15094 , 2024

2024

Closest in time.

S. Dong, L. Ding, Z. Huang, Z. Wang, T. Xue, and D. Xu, “Interactive3d: Create what you want by interactive 3d generation,” arXiv preprint arXiv:2404.16510 , 2024

2024

Closest in time.

Y. Chen, C. Zhang, X. Yang, Z. Cai, G. Yu, L. Yang, and G. Lin, “It3d: Improved text-to-3d generation with explicit view synthesis,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 2, 2024, pp. 1237–1244

2024

Closest in time.

Z. Wang, C. Lu, Y. Wang, F. Bao, C. Li, H. Su, and J. Zhu, “Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.

Y. Ma, Z. Lin, J. Ji, Y. Fan, X. Sun, and R. Ji, “X-oscar: A progressive framework for high-quality text-guided 3d animatable avatar generation,” arXiv preprint arXiv:2405.00954 , 2024

2024

Closest in time.

H. Zhang, B. Chen, H. Yang, L. Qu, X. Wang, L. Chen, C. Long, F. Zhu, D. Du, and M. Zheng, “Avatarverse: High-quality & stable 3d avatar creation from text and pose,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 7, 2024, pp. 7124–7132

2024

Closest in time.

N. Kolotouros, T. Alldieck, A. Zanfir, E. Bazavan, M. Fieraru, and C. Sminchisescu, “Dreamhuman: Animatable 3d avatars from text,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.

Y. Huang, J. Wang, A. Zeng, H. Cao, X. Qi, Y. Shi, Z.-J. Zha, and L. Zhang, “Dreamwaltz: Make a scene with complex 3d animatable avatars,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.

K. He, K. Yao, Q. Zhang, J. Yu, L. Liu, and L. Xu, “Dresscode: Autoregressively sewing and generating garments from text guidance,” arXiv preprint arXiv:2401.16465 , 2024

2024

Closest in time.

X. Han, Y. Cao, K. Han, X. Zhu, J. Deng, Y.-Z. Song, T. Xiang, and K.-Y. K. Wong, “Headsculpt: Crafting 3d head avatars with text,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.

Z. Zhou, F. Ma, H. Fan, and Y. Yang, “Headstudio: Text to animatable head avatars with 3d gaussian splatting,” arXiv preprint arXiv:2402.06149 , 2024

2024

Closest in time.

R. Shao, Y. Pang, Z. Zheng, J. Sun, and Y. Liu, “Human4dit: Free-view human video generation with 4d diffusion transformer,” arXiv preprint arXiv:2405.17405 , 2024

2024

Closest in time.

J. Gong, S. Ji, L. G. Foo, K. Chen, H. Rahmani, and J. Liu, “Laga: Layered 3d avatar generation and customization via gaussian splatting,” arXiv preprint arXiv:2405.12663 , 2024

2024

Closest in time.

J. Tang, Y. Zeng, K. Fan, X. Wang, B. Dai, K. Chen, and L. Ma, “Make-it-vivid: Dressing your animatable biped cartoon characters from text,” arXiv preprint arXiv:2403.16897 , 2024

2024

Closest in time.

C. Wang, J. Huang, R. Zhang, Q. Wang, H. Yang, H. Huang, C. Ma, and W. Xu, “Text-driven diverse facial texture generation via progressive latent-space refinement,” arXiv preprint arXiv:2404.09540 , 2024

2024

Closest in time.

Y. Wu, H. Xu, X. Tang, X. Chen, S. Tang, Z. Zhang, C. Li, and X. Jin, “Portrait3d: Text-guided high-quality 3d portrait generation using pyramid representation and gans prior,” arXiv preprint arXiv:2404.10394 , 2024

2024

Closest in time.

P. Li, C. Tang, Q. Huang, and Z. Li, “Art3d: 3d gaussian splatting for text-guided artistic scenes generation,” arXiv preprint arXiv:2405.10508 , 2024

2024

Closest in time.

Y.-C. Lee, Y.-T. Chen, A. Wang, T.-H. Liao, B. Y. Feng, and J.-B. Huang, “Vividdream: Generating 3d scene with ambient dynamics,” arXiv preprint arXiv:2405.20334 , 2024

2024

Closest in time.

O. Rahamim, H. Segev, I. Achituve, Y. Atzmon, Y. Kasten, and G. Chechik, “Lay-a-scene: Personalized 3d object arrangement using text-to-image priors,” arXiv preprint arXiv:2406.00687 , 2024

2024

Closest in time.

Y. Ma, D. Zhan, and Z. Jin, “Fastscene: Text-driven fast 3d indoor scene generation via panoramic gaussian splatting,” arXiv preprint arXiv:2405.05768 , 2024

2024

Closest in time.

X. Zhou, X. Ran, Y. Xiong, J. He, Z. Lin, Y. Wang, D. Sun, and M.-H. Yang, “Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting,” arXiv preprint arXiv:2402.07207 , 2024

2024

Closest in time.

S. Bahmani, X. Liu, Y. Wang, I. Skorokhodov, V. Rong, Z. Liu, X. Liu, J. J. Park, S. Tulyakov, G. Wetzstein et al. , “Tc4d: Trajectory-conditioned text-to-4d generation,” arXiv preprint arXiv:2403.17920 , 2024

2024

Closest in time.

S. Bahmani, I. Skorokhodov, V. Rong, G. Wetzstein, L. Guibas, P. Wonka, S. Tulyakov, J. J. Park, A. Tagliasacchi, and D. B. Lindell, “4d-fy: Text-to-4d generation using hybrid score distillation sampling,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 7996–8006

2024

Closest in time.

H. Xu, Y. Wu, X. Tang, J. Zhang, Y. Zhang, Z. Zhang, C. Li, and X. Jin, “Fusiondeformer: text-guided mesh deformation using diffusion models,” The Visual Computer , pp. 1–12, 2024

2024

Closest in time.

Y. Li, Y. Dou, Y. Shi, Y. Lei, X. Chen, Y. Zhang, P. Zhou, and B. Ni, “Focaldreamer: Text-driven 3d editing via focal-fusion assembly,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 4, 2024, pp. 3279–3287

2024

Closest in time.

F.-L. Liu, H. Fu, Y.-K. Lai, and L. Gao, “Sketchdream: Sketch-based text-to-3d generation and editing,” arXiv preprint arXiv:2405.06461 , 2024

2024

Closest in time.

J. Zhuang, D. Kang, Y.-P. Cao, G. Li, L. Lin, and Y. Shan, “Tip-editor: An accurate 3d editor following both text-prompts and image-prompts,” arXiv preprint arXiv:2401.14828 , 2024

2024

Closest in time.

Y.-C. Chen, S. Ling, Z. Chen, V. G. Kim, M. Gadelha, and A. Jacobson, “Text-guided controllable mesh refinement for interactive 3d modeling,” arXiv preprint arXiv:2406.01592 , 2024

2024

Closest in time.

T. Wu, G. Yang, Z. Li, K. Zhang, Z. Liu, L. Guibas, D. Lin, and G. Wetzstein, “Gpt-4v (ision) is a human-aligned evaluator for text-to-3d generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 22 227–22 238

2024

Closest in time.

W. Li, J. Liu, R. Chen, Y. Liang, X. Chen, P. Tan, and X. Long, “Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner,” arXiv preprint arXiv:2405.14979 , 2024

Original

2024

Closest in time.

Z. Qi, M. Yu, R. Dong, and K. Ma, “Vpp: Efficient conditional 3d generation via voxel-point progressive representation,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.