Large Language Models (LLMs) have shown outstanding performance across wide range of downstream tasks.
This competency is attributed to their substantial parameter size and pre-training on extensive corpus.
Moreover, LLMs have exhibited enhanced reasoning capabilities in tackling complex reasoning tasks, owing to the utilization of a method named ``Chain-of-Thought (CoT) prompting''.
This method is designed to generate intermediate reasoning steps that guide the inference of the final answer.
Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA · Around
Built on
Brown T, Mann B, Ryder N, et al. Language models are few-shot learners[J]. Advances in neural information processing systems, 2020, 33: 1877-1901
Raffel C, Shazeer N, Roberts A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. The Journal of Machine Learning Research, 2020, 21(1): 5485-5551
2020
Earlier work this paper cites.
Similar
Wei J, Wang X, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[J]. Advances in Neural Information Processing Systems, 2022, 35: 24824-24837
Kojima T, Gu S S, Reid M, et al. Large language models are zero-shot reasoners[J]. Advances in neural information processing systems, 2022, 35: 22199-22213
Lu P, Mishra S, Xia T, et al. Learn to explain: Multimodal reasoning via thought chains for science question answering[J]. Advances in Neural Information Processing Systems, 2022, 35: 2507-2521
2022
Cited alongside, same era.
Ouyang L, Wu J, Jiang X, et al. Training language models to follow instructions with human feedback[J]. Advances in Neural Information Processing Systems, 2022, 35: 27730-27744