Fetching the paper…
Reading the bibliography…
Pretrained models are the standard starting point for training.
Linear mode connectivity and the lottery ticket hypothesis
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, and Michael Carbin. 2020 · 1912
Earlier work this paper cites.
Don’t stop pretraining: Adapt language models to domains and tasks
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020 · 2004
Earlier work this paper cites.
Meta-learning in neural networks: A survey
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2020 · 2004
Earlier work this paper cites.
The pascal recognising textual entailment challenge
Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005 · 2005
Earlier work this paper cites.
Automatically constructing a corpus of sentential paraphrases
William B. Dolan and Chris Brockett. 2005 · 2005
Earlier work this paper cites.
The second pascal recognising textual entailment challenge
Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, and Bernardo Magnini. 2006 · 2006
Earlier work this paper cites.
The third pascal recognizing textual entailment challenge
Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and William B. Dolan. 2007 · 2007
Earlier work this paper cites.
The sixth pascal recognizing textual entailment challenge
Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. 2009 · 2009
Earlier work this paper cites.
Linear mode connectivity in multitask and continual learning
Seyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. 2021 · 2010
Earlier work this paper cites.
The winograd schema challenge
Hector J. Levesque, Ernest Davis, and L. Morgenstern. 2011 · 2011
Earlier work this paper cites.
Choice of plausible alternatives: An evaluation of commonsense causal reasoning
Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S. Gordon. 2011 · 2011
Earlier work this paper cites.
Revisiting "qualitatively characterizing neural network optimization problems"
Jonathan Frankle. 2020 · 2012
Earlier work this paper cites.
The Winograd schema challenge
Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012 · 2012
Earlier work this paper cites.
Recursive deep models for semantic compositionality over a sentiment treebank
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013 · 2013
Earlier work this paper cites.
Qualitatively characterizing neural network optimization problems
Ian J. Goodfellow and Oriol Vinyals. 2015 · 2015
Earlier work this paper cites.
Semeval-2016 task 6: Detecting stance in tweets
Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016 · 2016
Earlier work this paper cites.
SQuAD: 100,000+ questions for machine comprehension of text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017 · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter. 2017 · 2017
Earlier work this paper cites.
Emotion intensities in tweets
Saif M. Mohammad and Felipe Bravo-Marquez. 2017 · 2017
Earlier work this paper cites.
Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. 2017 · 2017
Earlier work this paper cites.
SemEval-2017 task 4: Sentiment analysis in Twitter
Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017 · 2017
Earlier work this paper cites.
SemEval-2018 Task 2: Multilingual Emoji Prediction
Francesco Barbieri, Jose Camacho-Collados, Francesco Ronzano, Luis Espinosa-Anke, Miguel Ballesteros, Valerio Basile, Viviana Patti, and Horacio Saggion. 2018 · 2018
Cited alongside, same era.
e-snli: Natural language inference with natural language explanations
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018 · 2018
Cited alongside, same era.
Adamw and super-convergence is now the fastest way to train neural nets
Sylvain Gugger and Jeremy Howard. 2018 · 2018
Cited alongside, same era.
Looking beyond the surface: A challenge set for reading comprehension over multiple sentences
Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018 · 2018
Cited alongside, same era.
On first-order meta-learning algorithms
Alex Nichol, Joshua Achiam, and John Schulman. 2018 · 2018
Cited alongside, same era.
Adversarial NLI: A new benchmark for natural language understanding
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020 · 2020
Later among the works it cites.
Intermediate-task transfer learning with pretrained language models: When and why does it work?
Yada Pruksachatkun, Jason Phang, Haokun Liu, Phu Mon Htut, Xiaoyi Zhang, Richard Yuanzhe Pang, Clara Vania, Katharina Kann, and Samuel R. Bowman. 2020 · 2020
Later among the works it cites.
Cluster & tune: Enhance bert performance in low resource text classification
Eyal Shnarch, Ariel Gera, Alon Halfon, Lena Dankin, Leshem Choshen, Ranit Aharonov, and Noam Slonim. 2020 · 2020
Later among the works it cites.
Pegasus: Pre-training with extracted gap-sentences for abstractive summarization
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020 · 2020
Later among the works it cites.
Exploring the limits of large scale pre-training
Samira Abnar, Mostafa Dehghani, Behnam Neyshabur, and Hanie Sedghi. 2021 · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jason Phang, Thibault Févry, and Samuel R. Bowman. 2018 · 2018
Cited alongside, same era.
SemEval-2018 task 3: Irony detection in English tweets
Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018 · 2018
Cited alongside, same era.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018 · 2018
Cited alongside, same era.
A broad-coverage challenge corpus for sentence understanding through inference
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018 · 2018
Cited alongside, same era.
SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter
Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019 · 2019
Cited alongside, same era.
BoolQ: Exploring the surprising difficulty of natural yes/no questions
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
The CommitmentBank: Investigating projection in naturally occurring discourse
Marie-Catherine de Marneffe, Mandy Simons, and Judith Tonhauser. 2019 · 2019
Cited alongside, same era.
Later among the works it cites.
Muppet: Massive multi-task representations with pre-finetuning
Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. 2021 · 2021
Later among the works it cites.
Ext5: Towards extreme multi-task scaling for transfer learning
Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q Tran, Dara Bahri, Jianmo Ni, et al. 2021 · 2021
Later among the works it cites.
Rethinking why intermediate-task fine-tuning works
Ting-Yun Chang and Chi-Jen Lu. 2021 · 2021
Later among the works it cites.
The grammar-learning trajectories of neural language models
Leshem Choshen, Guy Hacohen, Daphna Weinshall, and Omri Abend. 2021 · 2021
Later among the works it cites.
The devil is in the detail: Simple tricks improve systematic generalization of transformers
R’obert Csord’as, Kazuki Irie, and Jürgen Schmidhuber. 2021 · 2021
Later among the works it cites.
The role of permutation invariance in linear mode connectivity of neural networks
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. 2021 · 2021
Later among the works it cites.
Does pretraining for summarization require knowledge transfer?
Kundan Krishna, Jeffrey Bigham, and Zachary C Lipton. 2021 · 2021
Later among the works it cites.
Merging models with fisher-weighted averaging
Michael Matena and Colin Raffel. 2021 · 2021
Later among the works it cites.
Making transformers solve compositional tasks
Santiago Ontan’on, Joshua Ainslie, Vaclav Cvicek, and Zachary Kenneth Fisher. 2021 · 2021
Later among the works it cites.
On neurons invariant to sentence structural changes in neural machine translation
Gal Patel, Leshem Choshen, and Omri Abend. 2021 · 2021
Later among the works it cites.
Few-shot question answering by pretraining span selection
Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, and Omer Levy. 2021 · 2021
Later among the works it cites.
A thorough evaluation of task-specific pretraining for summarization
Sascha Rothe, Joshua Maynez, and Shashi Narayan. 2021 · 2021
Later among the works it cites.
Revisiting few-sample bert fine-tuning
Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, and Yoav Artzi. 2021 · 2021
Later among the works it cites.
Revisiting parameter-efficient tuning: Are we really there yet?
Guanzheng Chen, Fangyu Liu, Zaiqiao Meng, and Shangsong Liang. 2022 · 2022
Closest in time.
Analyzing populations of neural networks via dynamical model embedding
Jordan Cotler, Kai Sheng Tai, Felipe Hernandez, Blake Elias, and David Sussillo. 2022 · 2022
Closest in time.
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022 · 2022
Closest in time.