Fetching the paper…
Reading the bibliography…
Recent advancements in open-source code large language models (LLMs) have been driven by fine-tuning on the data generated from powerful closed-source LLMs, which are expensive to obtain.
A statistical interpretation of term specificity and its application in retrieval , 132–142
Sparck Jones, K. 1988 · 1988
Earlier work this paper cites.
On the resemblance and containment of documents
Broder, A. 1997 · 1997
Earlier work this paper cites.
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
Shazeer, N. M.; and Stern, M. 2018 · 2018
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020 · 2020
Earlier work this paper cites.
Program synthesis with large language models
Austin, J.; Odena, A.; Nye, M.; Bosma, M.; Michalewski, H.; Dohan, D.; Jiang, E.; Cai, C.; Terry, M.; Le, Q.; et al. 2021 · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; Pinto, H. P. d. O.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. 2021 · 2021
Earlier work this paper cites.
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation
Wang, Y.; Wang, W.; Joty, S.; and Hoi, S. C. 2021 · 2021
Earlier work this paper cites.
Finetuned language models are zero-shot learners
Wei, J.; Bosma, M.; Zhao, V. Y.; Guu, K.; Yu, A. W.; Lester, B.; Du, N.; Dai, A. M.; and Le, Q. V. 2021 · 2021
Earlier work this paper cites.
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; Schuh, P.; Shi, K.; Tsvyashchenko, S.; Maynez, J.; Rao, A.; Barnes, P.; Tay, Y.; Shazeer, N. M.; Prabhakaran, V.; Reif, E.; Du, N.; Hutchinson, B. C.; Pope, R.; Bradbury, J.; Austin, J.; Isard, M.; Gur-Ari, G.; Yin, P.; Duke, T.; Levskaya, A.; Ghemawat, S.; Dev, S.; Michalewski, H.; García, X.; Misra, V.; Robinson, K.; Fedus, L.; Zhou, D.; Ippolito, D.; Luan, D.; Lim, H.; Zoph, B.; Spiridonov, A.; Sepassi, R.; Dohan, D.; Agrawal, S.; Omernick, M.; Dai, A. M.; Pillai, T. S.; Pellat, M.; Lewkowycz, A.; Moreira, E.; Child, R.; Polozov, O.; Lee, K.; Zhou, Z.; Wang, X.; Saeta, B.; Díaz, M.; Firat, O.; Catasta, M.; Wei, J.; Meier-Hellstern, K. S.; Eck, D.; Dean, J.; Petrov, S.; and Fiedel, N. 2022 · 2022
Earlier work this paper cites.
Competition-level code generation with AlphaCode
Li, Y.; Choi, D.; Chung, J.; Kushman, N.; Schrittwieser, J.; Leblond, R.; Tom; Eccles; Keeling, J.; Gimeno, F.; Lago, A. D.; Hubert, T.; Choy, P.; de, C.; d’Autume, M.; Babuschkin, I.; Chen, X.; Huang, P.-S.; Welbl, J.; Gowal, S.; Alexey; Cherepanov; Molloy, J.; Mankowitz, D. J.; Robson, E. S.; Kohli, P.; de, N.; Freitas; Kavukcuoglu, K.; and Vinyals, O. 2022 · 2022
Earlier work this paper cites.
Codegen: An open large language model for code with multi-turn program synthesis
Nijkamp, E.; Pang, B.; Hayashi, H.; Tu, L.; Wang, H.; Zhou, Y.; Savarese, S.; and Xiong, C. 2022 · 2022
Earlier work this paper cites.
Chatgpt: Optimizing language models for dialogue
OpenAI. 2022 · 2022
Earlier work this paper cites.
One Embedder, Any Task: Instruction-Finetuned Text Embeddings
Su, H.; Shi, W.; Kasai, J.; Wang, Y.; Hu, Y.; Ostendorf, M.; tau Yih, W.; Smith, N. A.; Zettlemoyer, L.; and Yu, T. 2022 · 2022
Earlier work this paper cites.
Self-instruct: Aligning language models with self-generated instructions
Wang, Y.; Kordi, Y.; Mishra, S.; Liu, A.; Smith, N. A.; Khashabi, D.; and Hajishirzi, H. 2022 · 2022
Earlier work this paper cites.
Bai, J.; Bai, S.; Chu, Y.; Cui, Z.; Dang, K.; Deng, X.; Fan, Y.; Ge, W.; Han, Y.; Huang, F.; Hui, B.; Ji, L.; Li, M.; Lin, J.; Lin, R.; Liu, D.; Liu, G.; Lu, C.; Lu, K.; Ma, J.; Men, R.; Ren, X.; Ren, X.; Tan, C.; Tan, S.; Tu, J.; Wang, P.; Wang, S.; Wang, W.; Wu, S.; Xu, B.; Xu, J.; Yang, A.; Yang, H.; Yang, J.; Yang, S.; Yao, Y.; Yu, B.; Yuan, H.; Yuan, Z.; Zhang, J.; Zhang, X.; Zhang, Y.; Zhang, Z.; Zhou, C.; Zhou, J.; Zhou, X.; and Zhu, T. 2023 · 2023
Earlier work this paper cites.
MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation
Cassano, F.; Gouwar, J.; Nguyen, D.; Nguyen, S. D.; Phipps-Costin, L.; Pinckney, D.; Yee, M.-H.; Zi, Y.; Anderson, C. J.; Feldman, M. Q.; Guha, A.; Greenberg, M.; and Jangda, A. 2023 · 2023
Earlier work this paper cites.
Code Alpaca: An Instruction-following LLaMA model for code generation
Chaudhary, S. 2023 · 2023
Cited alongside, same era.
Visual programming: Compositional visual reasoning without training
Gupta, T.; and Kembhavi, A. 2023 · 2023
Cited alongside, same era.
Irugalbandara, C.; Mahendra, A.; Daynauth, R.; Arachchige, T. K.; Flautner, K.; Tang, L.; Kang, Y.; and Mars, J. 2023 · 2023
Cited alongside, same era.
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I. 2023 · 2023
Cited alongside, same era.
DS-1000: A natural and reliable benchmark for data science code generation
Lai, Y.; Li, C.; Wang, Y.; Zhang, T.; Zhong, R.; Zettlemoyer, L.; Yih, W.-t.; Fried, D.; Wang, S.; and Yu, T. 2023 · 2023
Cited alongside, same era.
Gemini: a family of highly capable multimodal models
Team, G.; Anil, R.; Borgeaud, S.; Wu, Y.; Alayrac, J.-B.; Yu, J.; Soricut, R.; Schalkwyk, J.; Dai, A. M.; Hauth, A.; et al. 2023 · 2023
Later among the works it cites.
The evolved code alpaca dataset
theblackcat102. 2023 · 2023
Later among the works it cites.
Voyager: An open-ended embodied agent with large language models
Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, Y.; Fan, L.; and Anandkumar, A. 2023 · 2023
Later among the works it cites.
Magicoder: Source code is all you need
Wei, Y.; Wang, Z.; Liu, J.; Ding, Y.; and Zhang, L. 2023 · 2023
Later among the works it cites.
Wizardlm: Empowering large language models to follow complex instructions
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
StarCoder: may the source be with you!
Li, R.; Allal, L. B.; Zi, Y.; Muennighoff, N.; Kocetkov, D.; Mou, C.; Marone, M.; Akiki, C.; Li, J.; Chim, J.; Liu, Q.; Zheltonozhskii, E.; Zhuo, T. Y.; Wang, T.; Dehaene, O.; Davaadorj, M.; Lamy-Poirier, J.; Monteiro, J.; Shliazhko, O.; Gontier, N.; Meade, N.; Zebaze, A.; Yee, M.-H.; Umapathi, L. K.; Zhu, J.; Lipkin, B.; Oblokulov, M.; Wang, Z.; Murthy, R.; Stillerman, J.; Patel, S. S.; Abulkhanov, D.; Zocca, M.; Dey, M.; Zhang, Z.; Fahmy, N.; Bhattacharyya, U.; Yu, W.; Singh, S.; Luccioni, S.; Villegas, P.; Kunakov, M.; Zhdanov, F.; Romero, M.; Lee, T.; Timor, N.; Ding, J.; Schlesinger, C.; Schoelkopf, H.; Ebert, J.; Dao, T.; Mishra, M.; Gu, A.; Robinson, J.; Anderson, C. J.; Dolan-Gavitt, B.; Contractor, D.; Reddy, S.; Fried, D.; Bahdanau, D.; Jernite, Y.; Ferrandis, C. M.; Hughes, S. M.; Wolf, T.; Guha, A.; von Werra, L.; and de Vries, H. 2023 · 2023
Cited alongside, same era.
Code as policies: Language model programs for embodied control
Liang, J.; Huang, W.; Xia, F.; Xu, P.; Hausman, K.; Ichter, B.; Florence, P.; and Zeng, A. 2023 · 2023
Cited alongside, same era.
Liu, J.; Xia, C.; Wang, Y.; and Zhang, L. 2023 · 2023
Cited alongside, same era.
Wizardcoder: Empowering code large language models with evol-instruct
Luo, Z.; Xu, C.; Zhao, P.; Sun, Q.; Geng, X.; Hu, W.; Tao, C.; Ma, J.; Lin, Q.; and Jiang, D. 2023 · 2023
Cited alongside, same era.
Eureka: Human-level reward design via coding large language models
Ma, Y. J.; Liang, W.; Wang, G.; Huang, D.-A.; Bastani, O.; Jayaraman, D.; Zhu, Y.; Fan, L.; and Anandkumar, A. 2023 · 2023
Cited alongside, same era.
GitHub Copilot – Your AI pair programmer
Microsoft. 2023 · 2023
Cited alongside, same era.
OctoPack: Instruction Tuning Code Large Language Models
Muennighoff, N.; Liu, Q.; Liu, Q.; Zebaze, A.; Zheng, Q.; Hui, B.; Zhuo, T. Y.; Singh, S.; Tang, X.; von Werra, L.; and Longpre, S. 2023 · 2023
Cited alongside, same era.
Xu, C.; Sun, Q.; Zheng, K.; Geng, X.; Zhao, P.; Feng, J.; Tao, C.; and Jiang, D. 2023 · 2023
Later among the works it cites.
Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation
Yu, Z.; Zhang, X.; Shang, N.; Huang, Y.; Xu, C.; Zhao, Y.; Hu, W.; and Yin, Q. 2023 · 2023
Later among the works it cites.
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X
Zheng, Q.; Xia, X.; Zou, X.; Dong, Y.; Wang, S.; Xue, Y.; Wang, Z.; Shen, L.; Wang, A.; Li, Y.; Su, T.; Yang, Z.; and Tang, J. 2023 · 2023
Later among the works it cites.
DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence
Guo, D.; Zhu, Q.; Yang, D.; Xie, Z.; Dong, K.; Zhang, W.; Chen, G.; Bi, X.; Wu, Y.; Li, Y. K.; Luo, F.; Xiong, Y.; and Liang, W. 2024 · 2024
Closest in time.
StarCoder 2 and The Stack v2: The Next Generation
Lozhkov, A.; Li, R.; Allal, L. B.; Cassano, F.; Lamy-Poirier, J.; Tazi, N.; Tang, A.; Pykhtar, D.; Liu, J.; Wei, Y.; et al. 2024 · 2024
Closest in time.
AI models collapse when trained on recursively generated data
Shumailov, I.; Shumaylov, Z.; Zhao, Y.; Papernot, N.; Anderson, R. J.; and Gal, Y. 2024 · 2024
Closest in time.
Learning Performance-Improving Code Edits
Shypula, A.; Madaan, A.; Zeng, Y.; Alon, U.; Gardner, J. R.; Yang, Y.; Hashemi, M.; Neubig, G.; Ranganathan, P.; Bastani, O.; and Yazdanbakhsh, A. 2024 · 2024
Closest in time.
AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data
Song, Z.; Wang, Y.; Zhang, W.; Liu, K.; Lyu, C.; Song, D.; Guo, Q.; Yan, H.; Lin, D.; Chen, K.; and Zhao, C. 2024 · 2024
Closest in time.
Tang, H.; Key, D.; and Ellis, K. 2024 · 2024
Closest in time.
StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation
Wei, Y.; Cassano, F.; Liu, J.; Ding, Y.; Jain, N.; de Vries, H.; von Werra, L.; Guha, A.; and Zhang, L. 2024 · 2024
Closest in time.
AutoMathText: Autonomous Data Selection with Language Models for Mathematical Texts
Zhang, Y.; Luo, Y.; Yuan, Y.; and Yao, A. C.-C. 2024 · 2024
Closest in time.
Zhu, E.; Markovtsev, V.; Astafiev, A.; Khan, A.; Ha, C.; Łukasiewicz, W.; Foster, A.; Sinusoidal36; Thakur, S.; Ortolani, S.; Titusz; Letal, V.; Bentley, Z.; fpug; hguhlich; long2ice; oisincar; Assa, R.; Ibraimoski, S.; Kumar, R.; TianHuan, Q.; Rosenthal, M. J.; Joshi, K.; Mann, K.; JonR; Halliwell, J.; and Oriekhov, A. 2024 · 2024
Closest in time.