Fetching the paper…
Reading the bibliography…
Large language models (LLMs) have pushed the limits of natural language understanding and exhibited excellent problem-solving ability.
Distilling the Knowledge in a Neural Network
G. Hinton, O. Vinyals, and J. Dean · 2015
Earlier work this paper cites.
Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education
X. Zhu · 2015
Earlier work this paper cites.
Iterative Machine Teaching
W. Liu, B. Dai, A. Humayun, C. Tay, C. Yu, L. Smith, J. Rehg, and L. Song · 2017
Earlier work this paper cites.
Proximal Policy Optimization Algorithms
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov · 2017
Earlier work this paper cites.
Generalization in anti-causal learning
N. Kilbertus, G. Parascandolo, and B. Schölkopf · 2018
Earlier work this paper cites.
Feature Representation of Short Utterances Based on Knowledge Distillation for Spoken Language Identification
P. Shen, X. Lu, S. Li, and H. Kawai · 2018
Earlier work this paper cites.
T. Wang, J. Zhu, A. Torralba, and A. Efros · 2018
Earlier work this paper cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
J. Devlin, M. Chang, K. Lee, and K. Toutanova · 2019
Earlier work this paper cites.
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner · 2019
Earlier work this paper cites.
Knowledge Adaptation for Efficient Semantic Segmentation
T. He, C. Shen, Z. Tian, D. Gong, C. Sun, and Y. Yan · 2019
Earlier work this paper cites.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov · 2019
Earlier work this paper cites.
Relational Knowledge Distillation
W. Park, D. Kim, Y. Lu, and M. Cho · 2019
Earlier work this paper cites.
Language Models are Unsupervised Multitask Learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever · 2019
Earlier work this paper cites.
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
A. Talmor, J. Herzig, N. Lourie, and J. Berant · 2019
Earlier work this paper cites.
Language Models are Few-Shot Learners
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei · 2020
Earlier work this paper cites.
Improved Knowledge Distillation via Teacher Assistant
S. Mirzadeh, M. Farajtabar, A. Li, N. Levine, A. Matsukawa, and H. Ghasemzadeh · 2020
Earlier work this paper cites.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. Liu · 2020
Earlier work this paper cites.
Evaluating Large Language Models Trained on Code
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba · 2021
Earlier work this paper cites.
Training Verifiers to Solve Math Word Problems
K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman · 2021
Earlier work this paper cites.
Knowledge Distillation: A Survey
J. Gou, B. Yu, S. Maybank, and D. Tao · 2021
Earlier work this paper cites.
Measuring Mathematical Problem Solving With the MATH Dataset
D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt · 2021
Earlier work this paper cites.
Iterative Teaching by Label Synthesis
W. Liu, Z. Liu, H. Wang, L. Paull, B. Schölkopf, and A. Weller · 2021
Earlier work this paper cites.
GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model
B. Wang and A. Komatsuzaki · 2021
Cited alongside, same era.
Dataset Condensation with Gradient Matching
B. Zhao, K. Mopuri, and H. Bilen · 2021
Cited alongside, same era.
Submodularity In Machine Learning and Artificial Intelligence
J. Bilmes · 2022
Cited alongside, same era.
PaLM: Scaling Language Modeling with Pathways
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Austin, M. Isard, G. Gur-Ari, P. Yin, T. Duke, A. Levskaya, S. Ghemawat, S. Dev, H. Michalewski, X. Garcia, V. Misra, K. Robinson, L. Fedus, D. Zhou, D. Ippolito, D. Luan, H. Lim, B. Zoph, A. Spiridonov, R. Sepassi, D. Dohan, S. Agrawal, M. Omernick, A. Dai, T. Pillai, M. Pellat, A. Lewkowycz, E. Moreira, R. Child, O. Polozov, K. Lee, Z. Zhou, X. Wang, B. Saeta, M. Diaz, O. Firat, M. Catasta, J. Wei, K. Meier-Hellstern, D. Eck, J. Dean, S. Petrov, and N. Fiedel · 2022
Cited alongside, same era.
InternLM: A Multilingual Language Model with Progressively Enhanced Capabilities
InternLM · 2023
Closest in time.
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
H. Luo, Q. Sun, C. Xu, P. Zhao, J. Lou, C. Tao, X. Geng, Q. Lin, S. Chen, and D. Zhang · 2023
Closest in time.
Teaching Small Language Models to Reason
L. Magister, J. Mallinson, J. Adamek, E. Malmi, and A. Severyn · 2023
Closest in time.
When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale
M. Marion, A. Üstün, L. Pozzobon, A. Wang, M. Fadaee, and S. Hooker · 2023
Closest in time.
Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs
MosaicML · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
J. Huang, S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han · 2022
Cited alongside, same era.
Solving Quantitative Reasoning Problems with Language Models
A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra · 2022
Cited alongside, same era.
Explanations from Large Language Models Make Small Reasoners Better
S. Li, J. Chen, Y. Shen, Z. Chen, X. Zhang, Z. Li, H. Wang, J. Qian, B. Peng, Y. Mao, W. Chen, and X. Yan · 2022
Cited alongside, same era.
MetaICL: Learning to Learn In Context
S. Min, M. Lewis, L. Zettlemoyer, and H. Hajishirzi · 2022
Cited alongside, same era.
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong · 2022
Cited alongside, same era.
Training Language Models to Follow Instructions with Human Feedback
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe · 2022
Cited alongside, same era.
Galactica: A Large Language Model for Science
R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, and R. Stojnic · 2022
Cited alongside, same era.
Chain of Thought Prompting Elicits Reasoning in Large Language Models
J. Wei, X. Wang, D. Schuurmans, Maarten Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou · 2022
Cited alongside, same era.
Ariel N., Cole J., and Nataniel R · 2023
Closest in time.
G. Penedo, Q. Malartic, D. Hesslow, R. Cojocaru, A. Cappelli, H. Alobeidli, B. Pannier, E. Almazrouei, and J. Launay · 2023
Closest in time.
Iterative Teaching by Data Hallucination
Z. Qiu, W. Liu, T. Xiao, Z. Liu, U. Bhatt, Y. Luo, A. Weller, and B. Schölkopf · 2023
Closest in time.
Code Llama: Open Foundation Models for Code
B. Rozière, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin, A. Kozhevnikov, I. Evtimov, J. Bitton, M. Bhatt, C. Ferrer, A. Grattafiori, W. Xiong, A. Défossez, J. Copet, F. Azhar, H. Touvron, L. Martin, N. Usunier, T. Scialom, and G. Synnaeve · 2023
Closest in time.
Distilling Reasoning Capabilities into Smaller Language Models
K. Shridhar, A. Stolfo, and M. Sachan · 2023
Closest in time.
A Survey of Reasoning with Foundation Models
J. Sun, C. Zheng, E. Xie, Z. Liu, R. Chu, J. Qiu, et al · 2023
Closest in time.
Stanford Alpaca: An Instruction-following LLaMA Model
R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. Hashimoto · 2023
Closest in time.
Large Language Models are Better Reasoners with Self-Verification
Y. Weng, M. Zhu, F. Xia, B. Li, S. He, K. Liu, and J. Zhao · 2023
Closest in time.
Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
Z. Yuan, H. Yuan, C. Li, G. Dong, C. Tan, and C. Zhou · 2023
Closest in time.
Llemma: An Open Language Model For Mathematics
Z. Azerbayev, H. Schoelkopf, K. Paster, M. Dos, S. McAleer, A. Jiang, J. Deng, S. Biderman, and S. Welleck · 2024
Closest in time.
The Reversal Curse: LLMs Trained on “A is B” Fail to Learn “B is A”
L. Berglund, M. Tong, M. Kaufmann, M. Balesni, A. Stickland, T. Korbak, and O. Evans · 2024
Closest in time.
Let’s Verify Step by Step
H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe · 2024
Closest in time.
WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Z. Luo, C. Xu, P. Zhao, Q. Sun, X. Geng, W. Hu, C. Tao, J. Ma, Q. Lin, and D. Jiang · 2024
Closest in time.
Lego-Prover: Neural theorem proving with growing libraries
H. Xin, H. Wang, C. Zheng, L. Li, Z. Liu, Q. Cao, Y. Huang, J. Xiong, H. Shi, E. Xie, J. Yin, Z. Li, H. Liao, and X. Liang · 2024
Closest in time.
DQ-LoRE: Dual queries with low rank approximation re-ranking for in-context learning
J. Xiong, Z. Li, C. Zheng, Z. Guo, Y. Yin, E. Xie, Z. Yang, Q. Cao, H. Wang, X. Han, J. Tang, C. Li, and X. Liang · 2024
Closest in time.
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
X. Yue, X. Qu, G. Zhang, Y. Fu, W. Huang, H. Sun, Y. Su, and W. Chen · 2024
Closest in time.