Inductive-bias Learning: Generating Code Models with Large Language Model

C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning . PMLR, 2017, pp. 1126–1135

2017

Earlier work this paper cites.

G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” Advances in neural information processing systems , vol. 30, 2017

2017

Earlier work this paper cites.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020

2020

Earlier work this paper cites.

S. M. Xie, A. Raghunathan, P. Liang, and T. Ma, “An explanation of in-context learning as implicit bayesian inference,” arXiv preprint arXiv:2111.02080 , 2021

Original

2021

Earlier work this paper cites.

E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou, “What learning algorithm is in-context learning? investigations with linear models,” arXiv preprint arXiv:2211.15661 , 2022

Original

2022

Earlier work this paper cites.

S. Chan, A. Santoro, A. Lampinen, J. Wang, A. Singh, P. Richemond, J. McClelland, and F. Hill, “Data distributional properties drive emergent in-context learning in transformers,” Advances in Neural Information Processing Systems , vol. 35, pp. 18 878–18 891, 2022

2022

Cited alongside, same era.

D. Chijiwa, S. Yamaguchi, A. Kumagai, and Y. Ida, “Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks,” Advances in Neural Information Processing Systems , vol. 35, pp. 25 264–25 277, 2022

2022

Cited alongside, same era.

S. Garg, D. Tsipras, P. S. Liang, and G. Valiant, “What can transformers learn in-context? a case study of simple function classes,” Advances in Neural Information Processing Systems , vol. 35, pp. 30 583–30 598, 2022

2022

Cited alongside, same era.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in Neural Information Processing Systems , vol. 35, pp. 24 824–24 837, 2022

A. Mahankali, T. B. Hashimoto, and T. Ma, “One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention,” arXiv preprint arXiv:2307.03576 , 2023

Original

2023

Closest in time.

A. Sordoni, X. Yuan, M.-A. Côté, M. Pereira, A. Trischler, Z. Xiao, A. Hosseini, F. Niedtner, and N. L. Roux, “Deep language networks: Joint prompt training of stacked llms using variational inference,” arXiv preprint arXiv:2306.12509 , 2023

Original

2023

Closest in time.

J. Von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov, “Transformers learn in-context by gradient descent,” in International Conference on Machine Learning . PMLR, 2023, pp. 35 151–35 174

2023

Closest in time.

J. Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou et al. , “Larger language models do in-context learning differently,” arXiv preprint arXiv:2303.03846 , 2023

Original

2023

Closest in time.