Fetching the paper…
Reading the bibliography…
This work introduces self-infilling code generation, a general framework that incorporates infilling operations into auto-regressive decoding.
Scheduled sampling for sequence prediction with recurrent neural networks
Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N · 2015
Earlier work this paper cites.
Sequence level training with recurrent neural networks
Ranzato, M., Chopra, S., Auli, M., and Zaremba, W · 2015
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al · 2018
Earlier work this paper cites.
Kermit: Generative insertion-based modeling for sequences
Chan, W., Kitaev, N., Guu, K., Stern, M., and Uszkoreit, J · 2019
Earlier work this paper cites.
Insertion-based decoding with automatically inferred generation order
Gu, J., Liu, Q., and Cho, K · 2019
Earlier work this paper cites.
RoBERTa: A robustly optimized BERT pretraining approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Earlier work this paper cites.
Insertion transformer: Flexible sequence generation via insertion operations
Stern, M., Chan, W., Kiros, J., and Uszkoreit, J · 2019
Earlier work this paper cites.
Non-monotonic sequential text generation
Welleck, S., Brantley, K., Iii, H. D., and Cho, K · 2019
Earlier work this paper cites.
Xlnet: Generalized autoregressive pretraining for language understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R., and Le, Q. V · 2019
Earlier work this paper cites.
Zhu, W., Hu, Z., and Xing, E · 2019
Earlier work this paper cites.
Structural language models of code
Alon, U., Sadaka, R., Levy, O., and Yahav, E · 2020
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Earlier work this paper cites.
Enabling language models to fill in the blanks
Donahue, C., Lee, M., and Liang, P · 2020
Earlier work this paper cites.
The curious case of neural text degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y · 2020
Earlier work this paper cites.
Spanbert: Improving pre-training by representing and predicting spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., and Levy, O · 2020
Earlier work this paper cites.
BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L · 2020
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2020
Earlier work this paper cites.
Shen, T., Quach, V., Barzilay, R., and Jaakkola, T · 2020
Earlier work this paper cites.
Htlm: Hyper-text pre-training and prompting of language models
Aghajanyan, A., Okhonko, D., Lewis, M., Joshi, M., Xu, H., Ghosh, G., and Zettlemoyer, L · 2021
Earlier work this paper cites.
Program synthesis with large language models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al · 2021
Cited alongside, same era.
Training verifiers to solve math word problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al · 2021
Cited alongside, same era.
CM3: A causal masked multimodal model of the internet
Aghajanyan, A., Huang, B., Ross, C., Karpukhin, V., Xu, H., Goyal, N., Okhonko, D., Joshi, M., Ghosh, G., Lewis, M., et al · 2022
Cited alongside, same era.
Efficient training of language models to fill in the middle
Bavarian, M., Jun, H., Tezak, N., Schulman, J., McLeavey, C., Tworek, J., and Chen, M · 2022
Cited alongside, same era.
Wizardcoder: Empowering code large language models with evol-instruct
Luo, Z., Xu, C., Zhao, P., Sun, Q., Geng, X., Hu, W., Tao, C., Ma, J., Lin, Q., and Jiang, D · 2023
Closest in time.
Self-refine: Iterative refinement with self-feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al · 2023
Closest in time.
A guidance language for controlling large language models., 2023
Microsoft · 2023
Closest in time.
Octopack: Instruction tuning code large language models
Muennighoff, N., Liu, Q., Zebaze, A., Zheng, Q., Hui, B., Zhuo, T. Y., Singh, S., Tang, X., von Werra, L., and Longpre, S · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Glm: General language model pretraining with autoregressive blank infilling
Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., and Tang, J · 2022
Cited alongside, same era.
Large language models are zero-shot reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y · 2022
Cited alongside, same era.
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al · 2022
Cited alongside, same era.
New GPT-3 Capabilities: Edit and Insert
OpenAI, Bavarian, M., Jiang, A., Jun, H., and Pondé, H · 2022
Cited alongside, same era.
Transcending scaling laws with 0.1% extra compute
Tay, Y., Wei, J., Chung, H. W., Tran, V. Q., So, D. R., Shakeri, S., Garcia, X., Zheng, H. S., Rao, J., Chowdhery, A., et al · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al · 2022
Cited alongside, same era.
Retrieval-augmented multimodal language modeling
Yasunaga, M., Aghajanyan, A., Shi, W., James, R., Leskovec, J., Liang, P., Lewis, M., Zettlemoyer, L., and Yih, W.-t · 2022
Cited alongside, same era.
Glm-130b: An open bilingual pre-trained model
Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al · 2022
Cited alongside, same era.
Nguyen, A., Karampatziakis, N., and Chen, W · 2023
Closest in time.
Lever: Learning to verify language-to-code generation with execution
Ni, A., Iyer, S., Radev, D., Stoyanov, V., Yih, W.-t., Wang, S., and Lin, X. V · 2023
Closest in time.
OpenAI · 2023
Closest in time.
Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., and Wang, W. Y · 2023
Closest in time.
Code llama: Open foundation models for code
Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Remez, T., Rapin, J., et al · 2023
Closest in time.
Film: Fill-in language models for any-order generation
Shen, T., Peng, H., Shen, R., Fu, Y., Harchaoui, Z., and Choi, Y · 2023
Closest in time.
UL2: Unifying language learning paradigms
Tay, Y., Dehghani, M., Tran, V. Q., Garcia, X., Wei, J., Wang, X., Chung, H. W., Bahri, D., Schuster, T., Zheng, S., Zhou, D., Houlsby, N., and Metzler, D · 2023
Closest in time.
Efficient guided generation for llms
Willard, B. T. and Louf, R · 2023
Closest in time.
Text2reward: Automated dense reward function generation for reinforcement learning
Xie, T., Zhao, S., Wu, C. H., Liu, Y., Luo, Q., Zhong, V., Yang, Y., and Yu, T · 2023
Closest in time.
Scaling autoregressive multi-modal models: Pretraining and instruction tuning
Yu, L., Shi, B., Pasunuru, R., Muller, B., Golovneva, O., Wang, T., Babu, A., Tang, B., Karrer, B., Sheynin, S., et al · 2023
Closest in time.
Coder reviewer reranking for code generation
Zhang, T., Yu, T., Hashimoto, T., Lewis, M., Yih, W.-t., Fried, D., and Wang, S · 2023
Closest in time.
Outline, then details: Syntactically guided coarse-to-fine code generation
Zheng, W., Sharan, S. P., Jaiswal, A. K., Wang, K., Xi, Y., Xu, D., and Wang, Z · 2023
Closest in time.
Codegemma: Open code models based on gemma, 2024
CodeGemma Team, G. L · 2024
Closest in time.
Deepseek-coder: When the large language model meets programming–the rise of code intelligence
Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y., et al · 2024
Closest in time.
Starcoder 2 and the stack v2: The next generation
Lozhkov, A., Li, R., Allal, L. B., Cassano, F., Lamy-Poirier, J., Tazi, N., Tang, A., Pykhtar, D., Liu, J., Wei, Y., et al · 2024
Closest in time.
A survey of neural code intelligence: Paradigms, advances and beyond
Sun, Q., Chen, Z., Xu, F., Cheng, K., Ma, C., Yin, Z., Wang, J., Han, C., Zhu, R., Yuan, S., et al · 2024
Closest in time.
Repoformer: Selective retrieval for repository-level code completion
Wu, D., Ahmad, W. U., Zhang, D., Ramanathan, M. K., and Ma, X · 2024
Closest in time.
Lemur: Harmonizing natural language and code for language agents
Xu, Y., Su, H., Xing, C., Mi, B., Liu, Q., Shi, W., Hui, B., Zhou, F., Liu, Y., Xie, T., Cheng, Z., Zhao, S., Kong, L., Wang, B., Xiong, C., and Yu, T · 2024
Closest in time.