Fetching the paper…
Reading the bibliography…
In this paper, we introduce a new task for code completion that focuses on handling long code input and propose a sparse Transformer model, called LongCoder, to address this task.
Mining source code repositories at massive scale using language modeling
Allamanis, M. and Sutton, C · 2013
Earlier work this paper cites.
Mining idioms from source code
Allamanis, M. and Sutton, C · 2014
Earlier work this paper cites.
On the localness of software
Tu, Z., Su, Z., and Devanbu, P. T · 2014
Earlier work this paper cites.
PHOG: probabilistic model for code
Bielik, P., Raychev, V., and Vechev, M. T · 2016
Earlier work this paper cites.
On the naturalness of software
Hindle, A., Barr, E. T., Gabel, M., Su, Z., and Devanbu, P. T · 2016
Earlier work this paper cites.
Probabilistic model for code with decision trees
Raychev, V., Bielik, P., and Vechev, M. T · 2016
Earlier work this paper cites.
Are deep neural networks the best choice for modeling source code?
Hellendoorn, V. J. and Devanbu, P. T · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I · 2017
Earlier work this paper cites.
The adverse effects of code duplication in machine learning models of code
Allamanis, M · 2019
Earlier work this paper cites.
Generating long sequences with sparse transformers
Child, R., Gray, S., Radford, A., and Sutskever, I · 2019
Earlier work this paper cites.
BERT: pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M., Lee, K., and Toutanova, K · 2019
Earlier work this paper cites.
Unified language model pre-training for natural language understanding and generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., and Hon, H · 2019
Earlier work this paper cites.
Codesearchnet challenge: Evaluating the state of semantic code search
Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., and Brockschmidt, M · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Cited alongside, same era.
Longformer: The long-document transformer
Beltagy, I., Peters, M. E., and Cohan, A · 2020
Cited alongside, same era.
Language models are few-shot learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D · 2020
Cited alongside, same era.
Transformers are rnns: Fast autoregressive transformers with linear attention
Katharopoulos, A., Vyas, A., Pappas, N., and Fleuret, F · 2020
Rethinking attention with performers
Choromanski, K. M., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlós, T., Hawkins, P., Davis, J. Q., Mohiuddin, A., Kaiser, L., Belanger, D. B., Colwell, L. J., and Weller, A · 2021
Later among the works it cites.
Long-range modeling of source code files with ewash: Extended window access by syntax hierarchy
Clement, C. B., Lu, S., Liu, X., Tufano, M., Drain, D., Duan, N., Sundaresan, N., and Svyatkovskiy, A · 2021
Later among the works it cites.
Codexglue: A machine learning benchmark dataset for code understanding and generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C. B., Drain, D., Jiang, D., Tang, D., Li, G., Zhou, L., Shou, L., Zhou, L., Tufano, M., Gong, M., Zhou, M., Duan, N., Sundaresan, N., Deng, S. K., Fu, S., and Liu, S · 2021
Later among the works it cites.
Incoder: A generative model for code infilling and synthesis
Fried, D., Aghajanyan, A., Lin, J., Wang, S., Wallace, E., Shi, F., Zhong, R., Yih, W.-t., Zettlemoyer, L., and Lewis, M · 2022
Later among the works it cites.
Unixcoder: Unified cross-modal pre-training for code representation
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Reformer: The efficient transformer
Kitaev, N., Kaiser, L., and Levskaya, A · 2020
Cited alongside, same era.
Multi-task learning based pre-trained language model for code completion
Liu, F., Li, G., Zhao, Y., and Jin, Z · 2020
Cited alongside, same era.
Intellicode compose: code generation using transformer
Svyatkovskiy, A., Deng, S. K., Fu, S., and Sundaresan, N · 2020
Cited alongside, same era.
Linformer: Self-attention with linear complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., and Ma, H · 2020
Cited alongside, same era.
Big bird: Transformers for longer sequences
Zaheer, M., Guruganesh, G., Dubey, K. A., Ainslie, J., Alberti, C., Ontañón, S., Pham, P., Ravula, A., Wang, Q., Yang, L., and Ahmed, A · 2020
Cited alongside, same era.
Evaluating large language models trained on code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al · 2021
Cited alongside, same era.
Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., and Yin, J · 2022
Later among the works it cites.
Competition-level code generation with alphacode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al · 2022
Later among the works it cites.
A conversational paradigm for program synthesis
Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., and Xiong, C · 2022
Later among the works it cites.
cosformer: Rethinking softmax in attention
Qin, Z., Sun, W., Deng, H., Li, D., Wei, Y., Lv, B., Yan, J., Kong, L., and Zhong, Y · 2022
Later among the works it cites.
Efficient transformers: A survey
Tay, Y., Dehghani, M., Bahri, D., and Metzler, D · 2022
Later among the works it cites.
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., and Hellendoorn, V. J · 2022
Later among the works it cites.
Repobench: Benchmarking repository-level code auto-completion systems
Liu, T., Xu, C., and McAuley, J · 2023
Closest in time.