Fetching the paper…
Reading the bibliography…
Pretrained language models for code token embeddings are used in code search, code clone detection, and other code-related tasks.
1907
Earlier work this paper cites.
1908
Earlier work this paper cites.
R. Hadsell, S. Chopra, and Y. LeCun, “Dimensionality reduction by learning an invariant mapping,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06) , vol. 2, 2006, pp. 1735–1742
2006
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
2018
Earlier work this paper cites.
2019
Earlier work this paper cites.
A. Conneau and G. Lample, “Cross-lingual language model pretraining,” in Proceedings of the 33rd International Conference on Neural Information Processing Systems , 2019, pp. 7059–7069
2019
Earlier work this paper cites.
D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, L. Shujie, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu et al. , “Graphcodebert: Pre-training code representations with data flow,” in International Conference on Learning Representations , 2020
2020
Earlier work this paper cites.
A. Kanade, P. Maniatis, G. Balakrishnan, and K. Shi, “Learning and evaluating contextual embedding of source code,” in International conference on machine learning . PMLR, 2020, pp. 5110–5121
2020
Earlier work this paper cites.
Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang et al. , “Codebert: A pre-trained model for programming and natural languages,” in Findings of the Association for Computational Linguistics: EMNLP 2020 , 2020, pp. 1536–1547
2020
Earlier work this paper cites.
W.-C. Chang, F. X. Yu, Y.-W. Chang, Y. Yang, and S. Kumar, “Pre-training tasks for embedding-based large-scale retrieval,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=rkg-mA4FDr
2020
Cited alongside, same era.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020
2020
Cited alongside, same era.
T. Wang and P. Isola, “Understanding contrastive representation learning through alignment and uniformity on the hypersphere,” in International Conference on Machine Learning . PMLR, 2020, pp. 9929–9939
2020
Cited alongside, same era.
T. Gao, X. Yao, and D. Chen, “SimCSE: Simple contrastive learning of sentence embeddings,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 6894–6910
D. Guo, S. Lu, N. Duan, Y. Wang, M. Zhou, and J. Yin, “Unixcoder: Unified cross-modal pre-training for code representation,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2022, pp. 7212–7225
2022
Later among the works it cites.
X. Pei, D. Liu, L. Qian, and C. Xu, “Contrastive code-comment pre-training,” in 2022 IEEE International Conference on Data Mining (ICDM) , 2022, pp. 398–407
2022
Later among the works it cites.
X. Li, Y. Gong, Y. Shen, X. Qiu, H. Zhang, B. Yao, W. Qi, D. Jiang, W. Chen, and N. Duan, “CodeRetriever: A large scale contrastive pre-training method for code search,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing . Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 2898–2910. [Online]. Available: https://aclanthology.org/2022.emnlp-main.187
2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2021
Cited alongside, same era.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in Proceedings of the 38th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18–24 Jul 2021, pp. 8748–8763
2021
Cited alongside, same era.
2021
Cited alongside, same era.
2021
Cited alongside, same era.
A. Andonian, Q. Anthony, S. Biderman, S. Black, P. Gali, L. Gao, E. Hallahan, J. Levy-Kramer, C. Leahy, L. Nestler, K. Parker, M. Pieler, S. Purohit, T. Songz, W. Phil, and S. Weinbach, “GPT-NeoX: Large Scale Autoregressive Language Modeling in PyTorch,” 8 2021. [Online]. Available: https://www.github.com/eleutherai/gpt-neox
2021
Cited alongside, same era.
B. Wang and A. Komatsuzaki, “GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model,” https://github.com/kingoflolz/mesh-transformer-jax , May 2021
2021
Cited alongside, same era.
2022
Cited alongside, same era.
X. Wang, Y. Wang, Y. Wan, J. Wang, P. Zhou, L. Li, H. Wu, and J. Liu, “CODE-MVP: Learning to represent source code from multiple views with contrastive pre-training,” in Findings of the Association for Computational Linguistics: NAACL 2022 . Seattle, United States: Association for Computational Linguistics, Jul. 2022, pp. 1066–1077. [Online]. Available: https://aclanthology.org/2022.findings-naacl.80
2022
Later among the works it cites.
X. Wang, Q. Wu, H. Zhang, C. Lyu, X. Jiang, Z. Zheng, L. Lyu, and S. Hu, “Heloc: Hierarchical contrastive learning of source code representation,” in Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension , 2022, pp. 354–365
2022
Later among the works it cites.
X. Li, D. Guo, Y. Gong, Y. Lin, Y. Shen, X. Qiu, D. Jiang, W. Chen, and N. Duan, “Soft-labeled contrastive pre-training for function-level code representation,” in Findings of the Association for Computational Linguistics: EMNLP 2022 . Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 118–129. [Online]. Available: https://aclanthology.org/2022.findings-emnlp.9
2022
Later among the works it cites.
2022
Later among the works it cites.
E. Ben Zaken, Y. Goldberg, and S. Ravfogel, “BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 1–9. [Online]. Available: https://aclanthology.org/2022.acl-short.1
2022
Later among the works it cites.
H. Zhang, Y. Gong, Y. Shen, J. Lv, N. Duan, and W. Chen, “Adversarial retriever-ranker for dense text retrieval,” in International Conference on Learning Representations , 2022. [Online]. Available: https://openreview.net/forum?id=MR7XubKUFB
2022
Later among the works it cites.