Fetching the paper…
Reading the bibliography…
The rapid development of large language models has revolutionized code intelligence in software development.
Neural machine translation of rare words with subword units
R. Sennrich, B. Haddow, and A. Birch · 2015
Earlier work this paper cites.
Think you have solved question answering? try arc, the ai2 reasoning challenge
P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord · 2018
Earlier work this paper cites.
Decoupled weight decay regularization, 2019
I. Loshchilov and F. Hutter · 2019
Earlier work this paper cites.
Pipedream: Generalized pipeline parallelism for dnn training
D. Narayanan, A. Harlap, A. Phanishayee, V. Seshadri, N. R. Devanur, G. R. Ganger, P. B. Gibbons, and M. Zaharia · 2019
Earlier work this paper cites.
Hellaswag: Can a machine really finish your sentence?
R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi · 2019
Earlier work this paper cites.
Measuring massive multitask language understanding
D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt · 2020
Earlier work this paper cites.
A diverse corpus for evaluating and developing english math word problem solvers
S.-Y. Miao, C.-C. Liang, and K.-Y. Su · 2020
Earlier work this paper cites.
Zero: Memory optimizations toward training trillion parameter models
S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He · 2020
Earlier work this paper cites.
Program synthesis with large language models, 2021
J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, and C. Sutton · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al · 2021
Earlier work this paper cites.
Training verifiers to solve math word problems
K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al · 2021
Earlier work this paper cites.
Measuring mathematical problem solving with the math dataset
D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt · 2021
Earlier work this paper cites.
Are nlp models really able to solve simple math word problems?
A. Patel, S. Bhattamishra, and N. Goyal · 2021
Earlier work this paper cites.
Winogrande: An adversarial winograd schema challenge at scale
K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi · 2021
Earlier work this paper cites.
Y. Wang, W. Wang, S. Joty, and S. C. Hoi · 2021
Earlier work this paper cites.
Efficient training of language models to fill in the middle
M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen · 2022
Cited alongside, same era.
Glm: General language model pretraining with autoregressive blank infilling
Z. Du, Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, and J. Tang · 2022
Cited alongside, same era.
Incoder: A generative model for code infilling and synthesis
D. Fried, A. Aghajanyan, J. Lin, S. Wang, E. Wallace, F. Shi, R. Zhong, W.-t. Yih, L. Zettlemoyer, and M. Lewis · 2022
Cited alongside, same era.
The stack: 3 tb of permissively licensed source code
D. Kocetkov, R. Li, L. Jia, C. Mou, Y. Jernite, M. Mitchell, C. M. Ferrandis, S. Hughes, T. Wolf, D. Bahdanau, et al · 2022
Cited alongside, same era.
Deduplicating training data makes language models better
K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison-Burch, and N. Carlini · 2022
Cited alongside, same era.
Tora: A tool-integrated reasoning agent for mathematical problem solving
Z. Gou, Z. Shao, Y. Gong, Y. Yang, M. Huang, N. Duan, W. Chen, et al · 2023
Later among the works it cites.
Hai-llm: An efficient and lightweight tool for training large models
High-Flyer · 2023
Later among the works it cites.
Things i’m learning while training superhot
kaiokendev · 2023
Later among the works it cites.
Reducing activation recomputation in large transformer models
V. A. Korthikanti, J. Casper, S. Lym, L. McAfee, M. Andersch, M. Shoeybi, and B. Catanzaro · 2023
Later among the works it cites.
Ds-1000: A natural and reliable benchmark for data science code generation
Y. Lai, C. Li, Y. Wang, T. Zhang, R. Zhong, L. Zettlemoyer, W.-t. Yih, D. Fried, S. Wang, and T. Yu · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning
P. Lu, L. Qiu, K.-W. Chang, Y. N. Wu, S.-C. Zhu, T. Rajpurohit, P. Clark, and A. Kalyan · 2022
Cited alongside, same era.
Codegen: An open large language model for code with multi-turn program synthesis
E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong · 2022
Cited alongside, same era.
Challenging big-bench tasks and whether chain-of-thought can solve them
M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, , and J. Wei · 2022
Cited alongside, same era.
Santacoder: don’t reach for the stars!
L. B. Allal, R. Li, D. Kocetkov, C. Mou, C. Akiki, C. M. Ferrandis, N. Muennighoff, M. Mishra, A. Gu, M. Dey, et al · 2023
Cited alongside, same era.
Multipl-e: a scalable and polyglot approach to benchmarking neural code generation
F. Cassano, J. Gouwar, D. Nguyen, S. Nguyen, L. Phipps-Costin, D. Pinckney, M.-H. Yee, Y. Zi, C. J. Anderson, M. Q. Feldman, et al · 2023
Cited alongside, same era.
Extending context window of large language models via positional interpolation
S. Chen, S. Wong, L. Chen, and Y. Tian · 2023
Cited alongside, same era.
Flashattention-2: Faster attention with better parallelism and work partitioning, 2023
T. Dao · 2023
Cited alongside, same era.
R. Li, L. B. Allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim, et al · 2023
Later among the works it cites.
Codegen2: Lessons for training llms on programming and natural languages, 2023
E. Nijkamp, H. Hayashi, C. Xiong, S. Savarese, and Y. Zhou · 2023
Later among the works it cites.
Gpt-4 technical report, 2023
OpenAI · 2023
Later among the works it cites.
Exploring the limits of transfer learning with a unified text-to-text transformer, 2023
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu · 2023
Later among the works it cites.
Code llama: Open foundation models for code
B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin, et al · 2023
Later among the works it cites.
Roformer: Enhanced transformer with rotary position embedding, 2023
J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu · 2023
Later among the works it cites.
Stanford alpaca: An instruction-following llama model
R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al · 2023
Later among the works it cites.
Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x
Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, L. Shen, Z. Wang, A. Wang, Y. Li, et al · 2023
Later among the works it cites.
Deepseek llm: Scaling open-source language models with longtermism
DeepSeek-AI · 2024
Closest in time.