2023

A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT

Cao, Yihan, Li, Siyu, Liu, Yixin et al.

Understand

Recently, ChatGPT, along with DALL-E-2 and Codex,has been gaining significant attention from society.

As a result, many individuals have become interested in related resources and are seeking to uncover the background and secrets behind its impressive performance.
In fact, ChatGPT and other Generative AI (GAI) techniques belong to the category of Artificial Intelligence Generated Content (AIGC), which involves the creation of digital content, such as images, music, and natural language, through AI models.
The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace.

Built on

P. S. Heckbert, “Survey of texture mapping,” IEEE computer graphics and applications
1986
Earlier work this paper cites.
P. J. Burt and E. H. Adelson, “The laplacian pyramid as a compact image code,” in Readings in computer vision
1987
Earlier work this paper cites.
K. Knill and S. Young, “Hidden Markov Models in Speech and Language Processing,” in Corpus-Based Methods in Language and Speech Processing
1997
Earlier work this paper cites.
A. A. Efros and T. K. Leung, “Texture synthesis by non-parametric sampling,” in Proceedings of the seventh IEEE international conference on computer vision
1999
Earlier work this paper cites.
K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics
2002
Earlier work this paper cites.
Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin, “A neural probabilistic language model,” J. Mach. Learn. Res
2003
Earlier work this paper cites.
C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out
2004
Earlier work this paper cites.
R. Coulom, “Whole-history rating: A bayesian rating system for players of time-varying strength,” in Computers and Games: 6th International Conference, CG 2008, Beijing, China, September 29-October 1, 2008. Proceedings 6
2008
Earlier work this paper cites.
D. A. Reynolds et al
2009
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition
2009
Earlier work this paper cites.
T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur, “Recurrent neural network based language model.,” in Interspeech
2010
Earlier work this paper cites.
A. Graves and A. Graves, “Long short-term memory,” Supervised sequence labelling with recurrent neural networks
2012
Earlier work this paper cites.
C. W. Fox and S. J. Roberts, “A tutorial on variational bayesian inference,” Artificial intelligence review
2012
Earlier work this paper cites.
T.-C. Zorila, V. Kandia, and Y. Stylianou, “Speech-in-noise intelligibility improvement based on spectral shaping and dynamic range compression,” in Thirteenth Annual Conference of the International Speech Communication Association
2012
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114
Original
2013
Earlier work this paper cites.
L. Banarescu, C. Bonial, S. Cai, M. Georgescu, K. Griffitt, U. Hermjakob, K. Knight, P. Koehn, M. Palmer, and N. Schneider, “Abstract meaning representation for sembanking,” in Proceedings of the 7th linguistic annotation workshop and interoperability with discourse
2013
Earlier work this paper cites.
S. Wang, D. Lo, and L. Jiang, “An empirical study on developer interactions in stackoverflow,” in Proceedings of the 28th annual ACM symposium on applied computing
2013
Earlier work this paper cites.
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” Advances in neural information processing systems
2013
Earlier work this paper cites.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” 2014
2014
Earlier work this paper cites.
L. Dinh, D. Krueger, and Y. Bengio, “Nice: Non-linear independent components estimation,” arXiv preprint arXiv:1410.8516
Original
2014
Earlier work this paper cites.
M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784
Original
2014
Earlier work this paper cites.
Y. Gil, M. Greaves, J. Hendler, and H. Hirsh, “Amplify scientific discovery with artificial intelligence,” Science
2014
Earlier work this paper cites.
J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)
2014
Earlier work this paper cites.
E. L. Denton, S. Chintala, R. Fergus, et al
2015
Earlier work this paper cites.
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434
Original
2015
Earlier work this paper cites.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems
2015
Earlier work this paper cites.
A. M. Dai and Q. V. Le, “Semi-supervised sequence learning,” Advances in neural information processing systems
2015
Earlier work this paper cites.
M. Kandlhofer, G. Steinbauer, S. Hirschmugl-Gaisch, and P. Huber, “Artificial intelligence and computer science in education: From kindergarten to university,” in 2016 IEEE Frontiers in Education Conference (FIE)
2016
Earlier work this paper cites.
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450
Original
2016
Earlier work this paper cites.
J. Donahue, P. Krähenbühl, and T. Darrell, “Adversarial feature learning,” arXiv preprint arXiv:1605.09782
Original
2016
Earlier work this paper cites.
I. Durugkar, I. Gemp, and S. Mahadevan, “Generative multi-adversarial networks,” arXiv preprint arXiv:1611.01673
Original
2016
Earlier work this paper cites.
M.-Y. Liu and O. Tuzel, “Coupled generative adversarial networks,” Advances in neural information processing systems
2016
Earlier work this paper cites.
X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Interpretable representation learning by information maximizing generative adversarial nets,” Advances in neural information processing systems
2016
Earlier work this paper cites.
S. Nowozin, B. Cseke, and R. Tomioka, “f-gan: Training generative neural samplers using variational divergence minimization,” Advances in neural information processing systems
2016
Earlier work this paper cites.
T. Che, Y. Li, A. P. Jacob, Y. Bengio, and W. Li, “Mode regularized generative adversarial networks,” arXiv preprint arXiv:1612.02136
Original
2016
Earlier work this paper cites.
L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein, “Unrolled generative adversarial networks,” arXiv preprint arXiv:1611.02163
Original
2016
Earlier work this paper cites.
M. D. Hoffman and M. J. Johnson, “Elbo surgery: yet another way to carve up the variational evidence lower bound,” in Workshop in Advances in Approximate Bayesian Inference, NIPS
2016
Earlier work this paper cites.
L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using real nvp,” arXiv preprint arXiv:1605.08803
Original
2016
Earlier work this paper cites.
K. Choi, G. Fazekas, B. McFee, K. Cho, and M. Sandler, “Towards music captioning: Generating music playlist descriptions,” arXiv preprint arXiv:1608.04868
Original
2016
Earlier work this paper cites.
X. Li, A. Taheri, L. Tu, and K. Gimpel, “Commonsense knowledge base completion,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
2016
Earlier work this paper cites.
L. Dong and M. Lapata, “Language to logical form with neural attention,” arXiv preprint arXiv:1601.01280
Original
2016
Earlier work this paper cites.
R. Jia and P. Liang, “Data recombination for neural semantic parsing,” arXiv preprint arXiv:1606.03622
Original
2016
Earlier work this paper cites.
J. T. Rolfe, “Discrete variational autoencoders,” arXiv preprint arXiv:1609.02200
Original
2016
Earlier work this paper cites.
P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep Reinforcement Learning from Human Preferences,” in Advances in Neural Information Processing Systems
2017
Earlier work this paper cites.
R. Dey and F. M. Salem, “Gate-variants of gated recurrent unit (gru) neural networks,” in 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS)
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems
2017
Earlier work this paper cites.
P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems
2017
Earlier work this paper cites.
T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progressive growing of gans for improved quality, stability, and variation,” arXiv preprint arXiv:1710.10196
Original
2017
Earlier work this paper cites.
T. Nguyen, T. Le, H. Vu, and D. Phung, “Dual discriminator generative adversarial nets,” Advances in neural information processing systems
2017
Earlier work this paper cites.
Q. Hoang, T. D. Nguyen, T. Le, and D. Phung, “Multi-generator generative adversarial nets,” arXiv preprint arXiv:1708.02556
Original
2017
Earlier work this paper cites.
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, “Improved training of wasserstein gans,” Advances in neural information processing systems
2017
Earlier work this paper cites.
X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. Paul Smolley, “Least squares generative adversarial networks,” in Proceedings of the IEEE international conference on computer vision
2017
Earlier work this paper cites.
A. Van Den Oord, O. Vinyals, et al
2017
Earlier work this paper cites.
G. Papamakarios, T. Pavlakou, and I. Murray, “Masked autoregressive flow for density estimation,” Advances in neural information processing systems
2017
Earlier work this paper cites.
G. Zheng, Y. Yang, and J. Carbonell, “Convolutional normalizing flows,” arXiv preprint arXiv:1711.02255
Original
2017
Earlier work this paper cites.
A. N. Gomez, M. Ren, R. Urtasun, and R. B. Grosse, “The reversible residual network: Backpropagation without storing activations,” Advances in neural information processing systems
2017
Earlier work this paper cites.
H. Zhang, T. Xu, H. Li, S. Zhang, X. Wang, X. Huang, and D. N. Metaxas, “Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks,” in Proceedings of the IEEE international conference on computer vision
2017
Earlier work this paper cites.
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” arXiv preprint arXiv:1710.10903
Original
2017
Earlier work this paper cites.
P. Yin and G. Neubig, “A syntactic neural model for general-purpose code generation,” arXiv preprint arXiv:1704.01696
Original
2017
Earlier work this paper cites.
M. Allamanis, M. Brockschmidt, and M. Khademi, “Learning to represent programs with graphs,” arXiv preprint arXiv:1711.00740
Original
2017
Earlier work this paper cites.
P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” Transactions of the association for computational linguistics
2017
Earlier work this paper cites.
Publisher: Journal of Advertising Research _eprint: https://www.journalofadvertisingresearch.com/content/58/3/263.full.pdf
J. Kietzmann, J. Paschen, and E. Treen, “Artificial Intelligence in Advertising,” Journal of Advertising Research · 2018
Earlier work this paper cites.
U. Khandelwal, H. He, P. Qi, and D. Jurafsky, “Sharp nearby, fuzzy far away: How neural language models use context,” 2018
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805
Original
2018
Earlier work this paper cites.
A. Sergeev and M. Del Balso, “Horovod: fast and easy distributed deep learning in tensorflow,” 2018
2018
Earlier work this paper cites.
A. Radford and K. Narasimhan, “Improving language understanding by generative pre-training,” 2018
2018
Earlier work this paper cites.
A. Brock, J. Donahue, and K. Simonyan, “Large scale gan training for high fidelity natural image synthesis,” arXiv preprint arXiv:1809.11096
Original
2018
Earlier work this paper cites.
D. Ulyanov, A. Vedaldi, and V. Lempitsky, “It takes (only) two: Adversarial generator-encoder networks,” in Proceedings of the AAAI Conference on Artificial Intelligence
2018
Earlier work this paper cites.
A. Ghosh, V. Kulharia, V. P. Namboodiri, P. H. Torr, and P. K. Dokania, “Multi-agent diverse generative adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition
2018
Earlier work this paper cites.
Y. Lu, Y.-W. Tai, and C.-K. Tang, “Attribute-guided face generation using conditional cyclegan,” in Proceedings of the European conference on computer vision (ECCV)
2018
Earlier work this paper cites.
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normalization for generative adversarial networks,” arXiv preprint arXiv:1802.05957
Original
2018
Earlier work this paper cites.
A. Jolicoeur-Martineau, “The relativistic discriminator: a key element missing from standard gan,” arXiv preprint arXiv:1807.00734
Original
2018
Earlier work this paper cites.
J. Tomczak and M. Welling, “Vae with a vampprior,” in International Conference on Artificial Intelligence and Statistics
2018
Earlier work this paper cites.
C.-W. Huang, D. Krueger, A. Lacoste, and A. Courville, “Neural autoregressive flows,” in International Conference on Machine Learning
2018
Earlier work this paper cites.
J.-H. Jacobsen, A. Smeulders, and E. Oyallon, “i-revnet: Deep invertible networks,” arXiv preprint arXiv:1802.07088
Original
2018
Earlier work this paper cites.
E. Haber, L. Ruthotto, E. Holtham, and S.-H. Jun, “Learning across scales—multiscale methods for convolution neural networks,” in Thirty-second AAAI conference on artificial intelligence
2018
Earlier work this paper cites.
T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, “Attngan: Fine-grained text to image generation with attentional generative adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition
2018
Earlier work this paper cites.
H. Liang, H. Wang, J. Wang, S. You, Z. Sun, J.-M. Wei, and Z. Yang, “Jtav: Jointly learning social media content representation by fusing textual, acoustic, and visual features,” arXiv preprint arXiv:1806.01483
Original
2018
Earlier work this paper cites.
B. Distiawan, J. Qi, R. Zhang, and W. Wang, “Gtr-lstm: A triple encoder for sentence generation from rdf data,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
2018
Earlier work this paper cites.
L. Song, Y. Zhang, Z. Wang, and D. Gildea, “A graph-to-sequence model for amr-to-text generation,” 2018
2018
Earlier work this paper cites.
C. Lyu and I. Titov, “Amr parsing as graph prediction with latent alignment,” arXiv preprint arXiv:1805.05286
Original
2018
Earlier work this paper cites.
B. Chen, L. Sun, and X. Han, “Sequence-to-action: End-to-end semantic graph generation for semantic parsing,” arXiv preprint arXiv:1809.00773
Original
2018
Earlier work this paper cites.
H. Dai, Y. Tian, B. Dai, S. Skiena, and L. Song, “Syntax-directed variational autoencoder for structured data,” arXiv preprint arXiv:1802.08786
Original
2018
Earlier work this paper cites.
H. Zhang, Y. Li, F. Ma, J. Gao, and L. Su, “Texttruth: an unsupervised approach to discover trustworthy information from multi-sourced text data,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
2018
Earlier work this paper cites.
Y. Qi and J. Xiao, “Fintech: Ai powers financial services to improve people’s lives,” Communications of the ACM
2018
Earlier work this paper cites.
M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” arXiv
2018
Earlier work this paper cites.
A. Gokaslan and V. Cohen, “Openwebtext corpus,” 2019
2019
Earlier work this paper cites.
J. Agnese, J. Herrera, H. Tao, and X. Zhu, “A survey and taxonomy of adversarial neural networks for text-to-image synthesis,” 2019
2019
Earlier work this paper cites.
Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” Advances in Neural Information Processing Systems
2019
Earlier work this paper cites.
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” arXiv preprint arXiv:1910.13461
Original
2019
Earlier work this paper cites.
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692
Original
2019
Earlier work this paper cites.
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le, “Xlnet: Generalized autoregressive pretraining for language understanding,” Advances in neural information processing systems
2019
Earlier work this paper cites.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019
2019
Earlier work this paper cites.
M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro, “Megatron-lm: Training multi-billion parameter language models using model parallelism,” 2019
2019
Earlier work this paper cites.
H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena, “Self-attention generative adversarial networks,” in International conference on machine learning
2019
Earlier work this paper cites.

Similar

T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
2019
Cited alongside, same era.
Q. Mao, H.-Y. Lee, H.-Y. Tseng, S. Ma, and M.-H. Yang, “Mode seeking generative adversarial networks for diverse image synthesis,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
2019
Cited alongside, same era.
L. Maaløe, M. Fraccaro, V. Liévin, and O. Winther, “Biva: A very deep hierarchy of latent variables for generative modeling,” Advances in neural information processing systems
2019
Cited alongside, same era.
P. Ghosh, M. S. Sajjadi, A. Vergari, M. Black, and B. Schölkopf, “From variational to deterministic autoencoders,” arXiv preprint arXiv:1903.12436
Original
2019
Cited alongside, same era.
A. Razavi, A. Van den Oord, and O. Vinyals, “Generating diverse high-fidelity images with vq-vae-2,” Advances in neural information processing systems
2019
Cited alongside, same era.
E. Hoogeboom, R. Van Den Berg, and M. Welling, “Emerging convolutions for generative normalizing flows,” in International Conference on Machine Learning
2019
Cited alongside, same era.
L. H. Li, M. Yatskar, D. Yin, C.-J. Hsieh, and K.-W. Chang, “Visualbert: A simple and performant baseline for vision and language,” arXiv preprint arXiv:1908.03557
Original
2019
Cited alongside, same era.
W. Su, X. Zhu, Y. Cao, B. Li, L. Lu, F. Wei, and J. Dai, “Vl-bert: Pre-training of generic visual-linguistic representations,” arXiv preprint arXiv:1908.08530
Original
2019
Cited alongside, same era.
H. Tan and M. Bansal, “Lxmert: Learning cross-modality encoder representations from transformers,” arXiv preprint arXiv:1908.07490
Original
2019
Cited alongside, same era.
Red Hook, NY, USA: Curran Associates Inc., 2019
J. Lu, D. Batra, D. Parikh, and S. Lee, ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks · 2019
Cited alongside, same era.
Y. Zhang, R. J. Weiss, H. Zen, Y. Wu, Z. Chen, R. Skerry-Ryan, Y. Jia, A. Rosenberg, and B. Ramabhadran, “Learning to speak fluently in a foreign language: Multilingual speech synthesis and cross-language voice cloning,” arXiv preprint arXiv:1907.04448
Original
2019
Cited alongside, same era.
Y. Yu, S. Tang, F. Raposo, and L. Chen, “Deep cross-modal correlation learning for audio and lyrics in music retrieval,” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)
2019
Cited alongside, same era.

Then

O. Patashnik, Z. Wu, E. Shechtman, D. Cohen-Or, and D. Lischinski, “Styleclip: Text-driven manipulation of stylegan imagery,” in Proceedings of the IEEE/CVF International Conference on Computer Vision
2021
Later among the works it cites.
A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” 2021
2021
Later among the works it cites.
M. Chen, X. Tan, B. Li, Y. Liu, T. Qin, S. Zhao, and T.-Y. Liu, “Adaspeech: Adaptive text to speech for custom voice,” arXiv preprint arXiv:2103.00993
Original
2021
Later among the works it cites.
A. Ferraro, X. Favory, K. Drossos, Y. Kim, and D. Bogdanov, “Enriched music representations with multiple cross-modal contrastive learning,” IEEE Signal Processing Letters
2021
Later among the works it cites.
I. Manco, E. Benetos, E. Quinton, and G. Fazekas, “Muscaps: Generating captions for music audio,” in 2021 International Joint Conference on Neural Networks (IJCNN)
2021
Later among the works it cites.
X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190
Original
2021
Later among the works it cites.
C. Edwards, C. Zhai, and H. Ji, “Text2mol: Cross-modal molecule retrieval with natural language queries,” pp. 595–607, 01 2021
2021
Later among the works it cites.
Y. Wang, W. Wang, S. Joty, and S. C. Hoi, “Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” arXiv preprint arXiv:2109.00859
Original
2021
Later among the works it cites.
W. U. Ahmad, S. Chakraborty, B. Ray, and K.-W. Chang, “Unified pre-training for program understanding and generation,” arXiv preprint arXiv:2103.06333
Original
2021
Later among the works it cites.
S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang, et al
Original
2021
Later among the works it cites.
arXiv:2107.03374 [cs]
Original
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating Large Language Models Trained on Code,” July 2021 · 2021
Later among the works it cites.
O. Evans, O. Cotton-Barratt, L. Finnveden, A. Bales, A. Balwit, P. Wills, L. Righetti, and W. Saunders, “Truthful ai: Developing and governing ai that does not lie,” arXiv preprint arXiv:2110.06674
Original
2021
Later among the works it cites.
R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, et al
Original
2021
Later among the works it cites.
S. Lin, J. Hilton, and O. Evans, “Truthfulqa: Measuring how models mimic human falsehoods,” arXiv preprint arXiv:2109.07958
Original
2021
Later among the works it cites.
E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the dangers of stochastic parrots: Can language models be too big?,” in Proceedings of the 2021 ACM conference on fairness, accountability, and transparency
2021
Later among the works it cites.
R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al
Original
2021
Later among the works it cites.
Z. Kenton, T. Everitt, L. Weidinger, I. Gabriel, V. Mikulik, and G. Irving, “Alignment of language agents,” arXiv preprint arXiv:2103.14659
Original
2021
Later among the works it cites.
J. Dhamala, T. Sun, V. Kumar, S. Krishna, Y. Pruksachatkun, K.-W. Chang, and R. Gupta, “Bold: Dataset and metrics for measuring biases in open-ended language generation,” in Proceedings of the 2021 ACM conference on fairness, accountability, and transparency
2021
Later among the works it cites.
P. P. Liang, C. Wu, L.-P. Morency, and R. Salakhutdinov, “Towards understanding and mitigating social biases in language models,” in International Conference on Machine Learning
2021
Later among the works it cites.
N. Carlini, Y. Liu, H. Daume III, U. Erlingsson, T. Kohno, and D. Song, “Extracting training data from large language models,” in 30th USENIX Security Symposium (USENIX Security 21)
2021
Later among the works it cites.
G. Somepalli, V. Singla, M. Goldblum, J. Geiping, and T. Goldstein, “Diffusion art or digital forgery? investigating data replication in diffusion models,” arXiv preprint arXiv:2212.03860
Original
2021
Later among the works it cites.
W. Kim, B. Son, and I. Kim, “Vilt: Vision-and-language transformer without convolution or region supervision,” in International Conference on Machine Learning
2021
Later among the works it cites.
Y. Huo, M. Zhang, G. Liu, H. Lu, Y. Gao, G. Yang, J. Wen, H. Zhang, B. Xu, W. Zheng, et al
Original
2021
Later among the works it cites.
L. Yunjiu, W. Wei, and Y. Zheng, “Artificial intelligence-generated and human expert-designed vocabulary tests: A comparative study,” SAGE Open
2022
Later among the works it cites.
“Chatgpt: Optimizing language models for dialogue,” Nov. 2022
2022
Later among the works it cites.
arXiv:2204.06125 [cs]
Original
A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical Text-Conditional Image Generation with CLIP Latents,” Apr. 2022 · 2022
Later among the works it cites.
arXiv:2209.03430 [cs]
Original
P. P. Liang, A. Zadeh, and L.-P. Morency, “Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions,” Sept. 2022 · 2022
Later among the works it cites.
arXiv:2203.02155 [cs]
Original
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” Mar. 2022 · 2022
Later among the works it cites.
arXiv:2112.10752 [cs]
Original
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” Apr. 2022 · 2022
Later among the works it cites.
J. Li, T. Tang, W. X. Zhao, J.-Y. Nie, and J.-R. Wen, “Pretrained language models for text generation: A survey,” 2022
2022
Later among the works it cites.
M. Suzuki and Y. Matsuo, “A survey of multimodal deep generative models,” Advanced Robotics
2022
Later among the works it cites.
S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, T. Mihaylov, M. Ott, S. Shleifer, K. Shuster, D. Simig, P. S. Koura, A. Sridhar, T. Wang, and L. Zettlemoyer, “Opt: Open pre-trained transformer language models,” 2022
2022
Later among the works it cites.
A. Glaese, N. McAleese, M. Trębacz, J. Aslanides, V. Firoiu, T. Ewalds, M. Rauh, L. Weidinger, M. Chadwick, P. Thacker, et al
Original
2022
Later among the works it cites.
R. Ramamurthy, P. Ammanabrolu, K. Brantley, J. Hessel, R. Sifa, C. Bauckhage, H. Hajishirzi, and Y. Choi, “Is reinforcement learning (not) for natural language processing?: Benchmarks, baselines, and building blocks for natural language policy optimization,” arXiv preprint arXiv:2210.01241
Original
2022
Later among the works it cites.
Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al
Original
2022
Later among the works it cites.
J. J. Dai, D. Ding, D. Shi, S. Huang, J. Wang, X. Qiu, K. Huang, G. Song, Y. Wang, Q. Gong, et al
2022
Later among the works it cites.
J. Ni, T. Young, V. Pandelea, F. Xue, and E. Cambria, “Recent advances in deep learning based dialogue systems: A systematic survey,” Artificial intelligence review
2022
Later among the works it cites.
T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé, et al
Original
2022
Later among the works it cites.
S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti, E. Zhang, R. Child, R. Y. Aminabadi, J. Bernauer, X. Song, M. Shoeybi, Y. He, M. Houston, S. Tiwary, and B. Catanzaro, “Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model,” 2022
2022
Later among the works it cites.
Z. Li, Z. Wang, M. Tan, R. Nallapati, P. Bhatia, A. Arnold, B. Xiang, and D. Roth, “Dq-bart: Efficient sequence-to-sequence model via joint distillation and quantization,” arXiv preprint arXiv:2203.11239
Original
2022
Later among the works it cites.
T. Salimans and J. Ho, “Progressive distillation for fast sampling of diffusion models,” arXiv preprint arXiv:2202.00512
Original
2022
Later among the works it cites.
H. Zheng, P. He, W. Chen, and M. Zhou, “Truncated diffusion probabilistic models,” stat
2022
Later among the works it cites.
Z. Lyu, X. Xu, C. Yang, D. Lin, and B. Dai, “Accelerating diffusion models via early stop of the diffusion process,” arXiv preprint arXiv:2205.12524
Original
2022
Later among the works it cites.
G. Franzese, S. Rossi, L. Yang, A. Finamore, D. Rossi, M. Filippone, and P. Michiardi, “How much is enough? a study on diffusion times in score-based generative models,” 2022
2022
Later among the works it cites.
F. Bao, C. Li, J. Zhu, and B. Zhang, “Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models,” arXiv preprint arXiv:2201.06503
Original
2022
Later among the works it cites.
D. Watson, W. Chan, J. Ho, and M. Norouzi, “Learning fast samplers for diffusion models by differentiating through sample quality,” in International Conference on Learning Representations
2022
Later among the works it cites.
A. Bansal, E. Borgnia, H.-M. Chu, J. S. Li, H. Kazemi, F. Huang, M. Goldblum, J. Geiping, and T. Goldstein, “Cold diffusion: Inverting arbitrary image transforms without noise,” arXiv preprint arXiv:2208.09392
Original
2022
Later among the works it cites.
K. Pandey, A. Mukherjee, P. Rai, and A. Kumar, “Diffusevae: Efficient, controllable and high-fidelity generation from low-dimensional latents,” 2022
2022
Later among the works it cites.
J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in International Conference on Machine Learning
2022
Later among the works it cites.
J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al
Original
2022
Later among the works it cites.
J. Merullo, L. Castricato, C. Eickhoff, and E. Pavlick, “Linearly mapping from image to text space,” arXiv preprint arXiv:2209.15162
Original
2022
Later among the works it cites.
C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. Denton, S. K. S. Ghasemipour, B. K. Ayan, S. S. Mahdavi, R. G. Lopes, et al
Original
2022
Later among the works it cites.
I. Manco, E. Benetos, E. Quinton, and G. Fazekas, “Learning music audio representations via weak language supervision,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
2022
Later among the works it cites.
B. Elizalde, S. Deshmukh, M. A. Ismail, and H. Wang, “Clap: Learning audio concepts from natural language supervision,” arXiv preprint arXiv:2206.04769
Original
2022
Later among the works it cites.
Y. Lu, Q. Liu, D. Dai, X. Xiao, H. Lin, X. Han, L. Sun, and H. Wu, “Unified structure generation for universal information extraction,” arXiv preprint arXiv:2203.12277
Original
2022
Later among the works it cites.
I. Melnyk, P. Dognin, and P. Das, “Knowledge graph generation from text,” arXiv preprint arXiv:2211.10511
Original
2022
Later among the works it cites.
B. Su, D. Du, Z. Yang, Y. Zhou, J. Li, A. Rao, H. Sun, Z. Lu, and J.-R. Wen, “A molecular multimodal foundation model associating molecule graphs with natural language,” 2022
2022
Later among the works it cites.
C. Edwards, T. Lai, K. Ros, G. Honke, K. Cho, and H. Ji, “Translation between molecules and natural language,” 2022
2022
Later among the works it cites.
E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong, “Codegen: An open large language model for code with multi-turn program synthesis,” 2022
2022
Later among the works it cites.
S. K. Lahiri, A. Naik, G. Sakkas, P. Choudhury, C. von Veh, M. Musuvathi, J. P. Inala, C. Wang, and J. Gao, “Interactive code generation via test-driven user-intent formalization,” 2022
2022
Later among the works it cites.
K. Shuster, J. Xu, M. Komeili, D. Ju, E. M. Smith, S. Roller, M. Ung, M. Chen, K. Arora, J. Lane, et al
Original
2022
Later among the works it cites.
" O’Reilly Media, Inc.", 2022
L. Tunstall, L. Von Werra, and T. Wolf, Natural language processing with transformers · 2022
Later among the works it cites.
A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al
Original
2022
Later among the works it cites.
A. Fawzi, M. Balog, A. Huang, T. Hubert, B. Romera-Paredes, M. Barekatain, A. Novikov, F. J. R Ruiz, J. Schrittwieser, G. Swirszcz, et al
2022
Later among the works it cites.
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al
Original
2022
Later among the works it cites.
N. Lee, W. Ping, P. Xu, M. Patwary, M. Shoeybi, and B. Catanzaro, “Factuality enhanced language models for open-ended text generation,” arXiv preprint arXiv:2206.04624
Original
2022
Later among the works it cites.
R. Thoppilan, D. De Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y. Du, et al
Original
2022
Later among the works it cites.
D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, et al
Original
2022
Later among the works it cites.
Y. Wu, N. Yu, Z. Li, M. Backes, and Y. Zhang, “Membership inference attacks against text-to-image generation models,” arXiv preprint arXiv:2210.00968
Original
2022
Later among the works it cites.
H. Hu, Z. Salcic, L. Sun, G. Dobbie, P. S. Yu, and X. Zhang, “Membership inference attacks on machine learning: A survey,” ACM Computing Surveys (CSUR)
2022
Later among the works it cites.
N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramèr, and C. Zhang, “Quantifying memorization across neural language models,” arXiv preprint arXiv:2202.07646
Original
2022
Later among the works it cites.
R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, and R. Stojnic, “Galactica: A large language model for science,” arXiv preprint arXiv:2211.09085
Original
2022
Later among the works it cites.
O. Ostapenko, T. Lesort, P. Rodríguez, M. R. Arefin, A. Douillard, I. Rish, and L. Charlin, “Continual learning with foundation models: An empirical study of latent replay,” in Conference on Lifelong Learning Agents
2022
Later among the works it cites.
J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” arXiv preprint arXiv:2201.11903
Original
2022
Later among the works it cites.
A. Madaan, S. Zhou, U. Alon, Y. Yang, and G. Neubig, “Language models of code are few-shot commonsense learners,” arXiv preprint arXiv:2210.07128
Original
2022
Later among the works it cites.
J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al
Original
2022
Later among the works it cites.
Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, L. Li, and Z. Sui, “A survey on in-context learning,” 2023
2023
Closest in time.
B. Zhu, J. Jiao, and M. I. Jordan, “Principled reinforcement learning with human feedback from pairwise or k k -wise comparisons,” arXiv preprint arXiv:2301.11270
Original
2023
Closest in time.
X. Amatriain, “Transformer models: an introduction and catalog,” arXiv preprint arXiv:2302.07730
Original
2023
Closest in time.
J. Y. Koh, R. Salakhutdinov, and D. Fried, “Grounding language models to images for multimodal generation,” arXiv preprint arXiv:2301.13823
Original
2023
Closest in time.
P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Computing Surveys
2023
Closest in time.
C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He, et al
Original
2023
Closest in time.
A. Borji, “A categorical archive of chatgpt failures,” 2023
2023
Closest in time.
“Could chatgpt become a monster misinformation superspreader?,” Jan 2023
2023
Closest in time.
A. Mahadevan, “This newspaper doesn’t exist: How chatgpt can launch fake news sites in minutes,” Feb 2023
2023
Closest in time.
“Why can’t ai draw realistic human hands?,” Jan 2023
2023
Closest in time.
J. Duan, F. Kong, S. Wang, X. Shi, and K. Xu, “Are diffusion models vulnerable to membership inference attacks?,” arXiv preprint arXiv:2302.01316
Original
2023
Closest in time.
H. Hu and J. Pang, “Membership inference of diffusion models,” arXiv preprint arXiv:2301.09956
Original
2023
Closest in time.
T. Matsumoto, T. Miura, and N. Yanai, “Membership inference attacks against diffusion models,” arXiv preprint arXiv:2302.03262
Original
2023
Closest in time.
N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V. Sehwag, F. Tramèr, B. Balle, D. Ippolito, and E. Wallace, “Extracting training data from diffusion models,” arXiv preprint arXiv:2301.13188
Original
2023
Closest in time.
Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola, “Multimodal chain-of-thought reasoning in language models,” arXiv preprint arXiv:2302.00923
Original
2023
Closest in time.
A. Aghajanyan, L. Yu, A. Conneau, W.-N. Hsu, K. Hambardzumyan, S. Zhang, S. Roller, N. Goyal, O. Levy, and L. Zettlemoyer, “Scaling laws for generative mixed-modal language models,” arXiv preprint arXiv:2301.03728
Original
2023
Closest in time.
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al
Original
2023
Closest in time.
J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” arXiv preprint arXiv:2301.12597
Original
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…