Unlock the Correlation between Supervised Fine-Tuning and Reinforcement Learning in Training Code Large Language Models

Brown, T., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems , 33, 1877–1901

1901

Earlier work this paper cites.

Aho, A. V., & Ullman, J. D. (1972). The Theory of Parsing, Translation and Compiling. Volume 1. Prentice-Hall

1972

Earlier work this paper cites.

Summers, P. D. (1977). A methodology for LISP program construction from examples. Journal of the ACM (JACM) , 24(1), 161–175

1977

Earlier work this paper cites.

Chandra, A. K., Kozen, D. C., & Stockmeyer, L. J. (1981). Alternation. Journal of the Association for Computing Machinery , 28(1), 114–133. doi:10.1145/322234.322243

1981

Earlier work this paper cites.

American Psychological Association. (1983). Publications Manual. American Psychological Association

1983

Earlier work this paper cites.

Gusfield, D. (1997). Algorithms on Strings, Trees and Sequences. Cambridge University Press

1997

Earlier work this paper cites.

Sutton, R. S., McAllester, D., Singh, S., & Mansour, Y. (1999). Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems , 12

1999

Earlier work this paper cites.

Ando, R. K., & Zhang, T. (2005). A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data. Journal of Machine Learning Research , 6, 1817–1853

2005

Earlier work this paper cites.

Andrew, G., & Gao, J. (2007). Scalable training of L1-regularized log-linear models. In Proceedings of the 24th International Conference on Machine Learning (pp. 33–40)

2007

Earlier work this paper cites.

Liang, P., Jordan, M. I., & Klein, D. (2010). Learning programs: A hierarchical Bayesian approach. In ICML (pp. 639–646)

2010

Earlier work this paper cites.

Shrivastava, A., & Li, P. (2014). In defense of minhash over simhash. In Artificial Intelligence and Statistics (pp. 886–894)

2014

Earlier work this paper cites.

Rasooli, M. S., & Tetreault, J. R. (2015). Yara Parser: A Fast and Accurate Dependency Parser. Computing Research Repository , arXiv:1503.06733. Retrieved from http://arxiv.org/abs/1503.06733

Original

2015

Earlier work this paper cites.

Schulman, J., et al. (2015). High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438

Original

2015

Earlier work this paper cites.

Yin, P., & Neubig, G. (2017). A syntactic neural model for general-purpose code generation. arXiv preprint arXiv:1704.01696

Original

2017

Earlier work this paper cites.

Schulman, J., et al. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

Original

2017

Earlier work this paper cites.

Radford, A., et al. (2019). Language models are unsupervised multitask learners. OpenAI Blog , 1(8), 9

2019

Earlier work this paper cites.

Chen, T., et al. (2020). Big self-supervised models are strong semi-supervised learners. Advances in Neural Information Processing Systems , 33, 22243–22255

2020

Earlier work this paper cites.

Raffel, C., et al. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140), 1–67

2020

Cited alongside, same era.

Chen, M., et al. (2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

Original

2021

Cited alongside, same era.

Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021). Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859

Original

2021

Cited alongside, same era.

Austin, J., et al. (2021). Program Synthesis with Large Language Models. arXiv preprint arXiv:2108.07732

Original

2021

Cited alongside, same era.

Sanh, V., et al. (2021). Multitask Prompted Training Enables Zero-Shot Task Generalization. In International Conference on Learning Representations

Achiam, J., et al. (2023). GPT-4 technical report. arXiv preprint arXiv:2303.08774

Original

2023

Later among the works it cites.

Gemini Team, et al. (2023). Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

Original

2023

Later among the works it cites.

Jiang, A. Q., et al. (2023). Mistral 7B. arXiv preprint arXiv:2310.06825

Original

2023

Later among the works it cites.

Chiang, W.-L., et al. (2023). Vicuna: An open-source chatbot impressing GPT-4 with 90Retrieved from https://vicuna.lmsys.org (accessed 14 April 2023)

2023

Later among the works it cites.

Wei, Y., et al. (2023). Magicoder: Source code is all you need. arXiv preprint arXiv:2312.02120

Original