Fetching the paper…
Reading the bibliography…
Large language models (LLMs) pretrained on vast source code have achieved prominent progress in code intelligence.
ORANGE: a method for evaluating automatic evaluation metrics for machine translation
C. Lin and F. J. Och · 2004
Earlier work this paper cites.
Mining source code repositories at massive scale using language modeling
M. Allamanis and C. Sutton · 2013
Earlier work this paper cites.
Towards a big data curated benchmark of inter-project code clones
J. Svajlenko, J. F. Islam, I. Keivanloo, C. K. Roy, and M. M. Mia · 2014
Earlier work this paper cites.
Probabilistic model for code with decision trees
V. Raychev, P. Bielik, and M. T. Vechev · 2016
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
MathQA: Towards interpretable math word problem solving with operation-based formalisms
A. Amini, S. Gabriel, S. Lin, R. Koncel-Kedziorski, Y. Choi, and H. Hajishirzi · 2019
Earlier work this paper cites.
BERT: pre-training of deep bidirectional transformers for language understanding
J. Devlin, M. Chang, K. Lee, and K. Toutanova · 2019
Earlier work this paper cites.
Unified language model pre-training for natural language understanding and generation
L. Dong, N. Yang, W. Wang, F. Wei, X. Liu, Y. Wang, J. Gao, M. Zhou, and H. Hon · 2019
Earlier work this paper cites.
Codesearchnet challenge: Evaluating the state of semantic code search
H. Husain, H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt · 2019
Earlier work this paper cites.
Billion-scale similarity search with GPUs
J. Johnson, M. Douze, and H. Jégou · 2019
Earlier work this paper cites.
Roberta: A robustly optimized BERT pretraining approach
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov · 2019
Earlier work this paper cites.
Decoupled weight decay regularization
I. Loshchilov and F. Hutter · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al · 2019
Earlier work this paper cites.
Codebert: A pre-trained model for programming and natural languages
Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou · 2020
Earlier work this paper cites.
Momentum contrast for unsupervised visual representation learning
K. He, H. Fan, Y. Wu, S. Xie, and R. B. Girshick · 2020
Earlier work this paper cites.
Dense passage retrieval for open-domain question answering
V. Karpukhin, B. Oguz, S. Min, P. S. H. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih · 2020
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu · 2020
Earlier work this paper cites.
Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters
J. Rasley, S. Rajbhandari, O. Ruwase, and Y. He · 2020
Earlier work this paper cites.
Recipes for safety in open-domain chatbots
J. Xu, D. Ju, M. Li, Y.-L. Boureau, J. Weston, and E. Dinan · 2020
Earlier work this paper cites.
Unified pre-training for program understanding and generation
W. U. Ahmad, S. Chakraborty, B. Ray, and K. Chang · 2021
Earlier work this paper cites.
Program synthesis with large language models
J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al · 2021
Earlier work this paper cites.
Gpt-neo: Large scale autoregressive language modeling with mesh-tensorflow, march 2021
S. Black, G. Leo, P. Wang, C. Leahy, and S. Biderman · 2021
Cited alongside, same era.
Evaluating large language models trained on code
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al · 2021
Cited alongside, same era.
Training verifiers to solve math word problems
K. Cobbe, V. Kosaraju, M. Bavarian, J. Hilton, R. Nakano, C. Hesse, and J. Schulman · 2021
Cited alongside, same era.
Graphcodebert: Pre-training code representations with data flow
D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. B. Clement, D. Drain, N. Sundaresan, J. Yin, D. Jiang, and M. Zhou · 2021
Cited alongside, same era.
Measuring coding challenge competence with apps
D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song, and J. Steinhardt · 2021
Coderl: Mastering code generation through pretrained models and deep reinforcement learning
H. Le, Y. Wang, A. D. Gotmare, S. Savarese, and S. C. H. Hoi · 2022
Later among the works it cites.
Solving quantitative reasoning problems with language models
A. Lewkowycz, A. J. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra · 2022
Later among the works it cites.
P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks
X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, and J. Tang · 2022
Later among the works it cites.
Learning from self-sampled correct and partially-correct programs
A. Ni, J. P. Inala, C. Wang, O. Polozov, C. Meek, D. R. Radev, and J. Gao · 2022
Later among the works it cites.
Spt-code: Sequence-to-sequence pre-training for learning source code representations
C. Niu, C. Li, V. Ng, J. Ge, L. Huang, and B. Luo · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Cosqa: 20, 000+ web queries for code search and question answering
J. Huang, D. Tang, L. Shou, M. Gong, K. Xu, D. Jiang, M. Zhou, and N. Duan · 2021
Cited alongside, same era.
GeDi: Generative discriminator guided sequence generation
B. Krause, A. D. Gotmare, B. McCann, N. S. Keskar, S. Joty, R. Socher, and N. F. Rajani · 2021
Cited alongside, same era.
The power of scale for parameter-efficient prompt tuning
B. Lester, R. Al-Rfou, and N. Constant · 2021
Cited alongside, same era.
Align before fuse: Vision and language representation learning with momentum distillation
J. Li, R. R. Selvaraju, A. Gotmare, S. R. Joty, C. Xiong, and S. C. Hoi · 2021
Cited alongside, same era.
X. Liu, Y. Zheng, Z. Du, M. Ding, Y. Qian, Z. Yang, and J. Tang · 2021
Cited alongside, same era.
Codexglue: A machine learning benchmark dataset for code understanding and generation
S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. B. Clement, D. Drain, D. Jiang, D. Tang, G. Li, L. Zhou, L. Shou, L. Zhou, M. Tufano, M. Gong, M. Zhou, N. Duan, N. Sundaresan, S. K. Deng, S. Fu, and S. Liu · 2021
Cited alongside, same era.
Retrieval augmented code generation and summarization
M. R. Parvez, W. U. Ahmad, S. Chakraborty, B. Ray, and K. Chang · 2021
Cited alongside, same era.
Training language models to follow instructions with human feedback
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Gray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe · 2022
Later among the works it cites.
Alexatm 20b: Few-shot learning using a large-scale multilingual seq2seq model
S. Soltan, S. Ananthakrishnan, J. FitzGerald, R. Gupta, W. Hamza, H. Khan, C. Peris, S. Rawls, A. Rosenbaum, A. Rumshisky, C. S. Prakash, M. Sridhar, F. Triefenbach, A. Verma, G. Tür, and P. Natarajan · 2022
Later among the works it cites.
Vl-adapter: Parameter-efficient transfer learning for vision-and-language tasks
Y.-L. Sung, J. Cho, and M. Bansal · 2022
Later among the works it cites.
Ml-enhanced code completion improves developer productivity, 2022
M. Tabachnyk and S. Nikolov · 2022
Later among the works it cites.
Unifying language learning paradigms
Y. Tay, M. Dehghani, V. Q. Tran, X. Garcia, D. Bahri, T. Schuster, H. S. Zheng, N. Houlsby, and D. Metzler · 2022
Later among the works it cites.
CODE-MVP: Learning to represent source code from multiple views with contrastive pre-training
X. Wang, Y. Wang, Y. Wan, J. Wang, P. Zhou, L. Li, H. Wu, and J. Liu · 2022
Later among the works it cites.
Code alpaca: An instruction-following llama model for code generation
S. Chaudhary · 2023
Closest in time.
Codet: Code generation with generated tests
B. Chen, F. Zhang, A. Nguyen, D. Zan, Z. Lin, J.-G. Lou, and W. Chen · 2023
Closest in time.
Starcoder: may the source be with you!
R. Li, L. B. Allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim, Q. Liu, E. Zheltonozhskii, T. Y. Zhuo, T. Wang, O. Dehaene, M. Davaadorj, J. Lamy-Poirier, J. Monteiro, O. Shliazhko, N. Gontier, N. Meade, A. Zebaze, M. Yee, L. K. Umapathi, J. Zhu, B. Lipkin, M. Oblokulov, Z. Wang, R. M. V, J. Stillerman, S. S. Patel, D. Abulkhanov, M. Zocca, M. Dey, Z. Zhang, N. Fahmy, U. Bhattacharyya, W. Yu, S. Singh, S. Luccioni, P. Villegas, M. Kunakov, F. Zhdanov, M. Romero, T. Lee, N. Timor, J. Ding, C. Schlesinger, H. Schoelkopf, J. Ebert, T. Dao, M. Mishra, A. Gu, J. Robinson, C. J. Anderson, B. Dolan-Gavitt, D. Contractor, S. Reddy, D. Fried, D. Bahdanau, Y. Jernite, C. M. Ferrandis, S. Hughes, T. Wolf, A. Guha, L. von Werra, and H. de Vries · 2023
Closest in time.
Meet in the middle: A new pre-training paradigm
A. Nguyen, N. Karampatziakis, and W. Chen · 2023
Closest in time.
OpenAI · 2023
Closest in time.
Combining parameter-efficient modules for task-level generalisation
E. M. Ponti, A. Sordoni, Y. Bengio, and S. Reddy · 2023
Closest in time.
replit-code-v1-3b, 2023
replit · 2023
Closest in time.
Stanford alpaca: An instruction-following llama model
R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto · 2023
Closest in time.
Llama: Open and efficient foundation language models
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al · 2023
Closest in time.
Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x, 2023
Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, Z. Wang, L. Shen, A. Wang, Y. Li, T. Su, Z. Yang, and J. Tang · 2023
Closest in time.