Fetching the paper…
Reading the bibliography…
In this work, we try to decipher the internal connection of NLP technology development in the past decades, searching for essence, which rewards us with a (potential) new learning paradigm for NLP tasks, dubbed as reStructured Pre-training (RST).
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019 · 1910
Earlier work this paper cites.
Computing machinery and intelligence
Alan M Turing and J Haugeland. 1950 · 1950
Earlier work this paper cites.
A proposal for the dartmouth summer research project on artificial intelligence, august 31, 1955
John McCarthy, Marvin L Minsky, Nathaniel Rochester, and Claude E Shannon. 2006 · 1955
Earlier work this paper cites.
Cross-lingual name tagging and linking for 282 languages
Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017 · 1958
Earlier work this paper cites.
Computers and thought
Edward A Feigenbaum, Julian Feldman, et al. 1963 · 1963
Earlier work this paper cites.
Disk-storage technology
Robert M White. 1980 · 1980
Earlier work this paper cites.
Japan’s fifth generation computer systems
Philip C Treleaven and Isabel Gouveia Lima. 1982 · 1982
Earlier work this paper cites.
The ATIS spoken language systems pilot corpus
Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990 · 1990
Earlier work this paper cites.
The attention system of the human brain
Michael I Posner and Steven E Petersen. 1990 · 1990
Earlier work this paper cites.
Understanding the new SQL: a complete guide
Jim Melton and Alan R Simon. 1993 · 1993
Earlier work this paper cites.
Strategic computing: DARPA and the quest for machine intelligence, 1983-1993
Alex Roland, Philip Shiman, William Aspray, et al. 2002 · 1993
Earlier work this paper cites.
Adaptive statistical language modeling; a maximum entropy approach
Ronald Rosenfeld. 1994 · 1994
Earlier work this paper cites.
Wordnet: a lexical database for english
George A Miller. 1995 · 1995
Earlier work this paper cites.
SENSEVAL-2: Overview
Philip Edmonds and Scott Cotton. 2001 · 2001
Earlier work this paper cites.
Gene selection for cancer classification using support vector machines
Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. 2002 · 2002
Earlier work this paper cites.
Learning question classifiers
Xin Li and Dan Roth. 2002 · 2002
Earlier work this paper cites.
Pre-trained models for natural language processing: A survey
Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020 · 2003
Earlier work this paper cites.
Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition
Erik F. Tjong Kim Sang and Fien De Meulder. 2003 · 2003
Earlier work this paper cites.
A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts
Bo Pang and Lillian Lee. 2004 · 2004
Earlier work this paper cites.
The English all-words task
Benjamin Snyder and Martha Palmer. 2004 · 2004
Earlier work this paper cites.
Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
Bo Pang and Lillian Lee. 2005 · 2005
Earlier work this paper cites.
Reducing the dimensionality of data with neural networks
Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006 · 2006
Earlier work this paper cites.
SemEval-2007 task-17: English lexical sample, SRL and all words
Sameer Pradhan, Edward Loper, Dmitriy Dligach, and Martha Palmer. 2007 · 2007
Earlier work this paper cites.
Curriculum learning
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009 · 2009
Earlier work this paper cites.
Probabilistic graphical models: principles and techniques
Daphne Koller and Nir Friedman. 2009 · 2009
Earlier work this paper cites.
Distant supervision for relation extraction without labeled data
Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009 · 2009
Earlier work this paper cites.
A view of cloud computing
Michael Armbrust, Armando Fox, Rean Griffith, Anthony D Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, et al. 2010 · 2010
Earlier work this paper cites.
SemEval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals
Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2010 · 2010
Earlier work this paper cites.
Language detection library for java
Shuyo Nakatani. 2010 · 2010
Earlier work this paper cites.
Learning word vectors for sentiment analysis
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011 · 2011
Earlier work this paper cites.
Sequence transduction with recurrent neural networks
Alex Graves. 2012 · 2012
Earlier work this paper cites.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012 · 2012
Earlier work this paper cites.
Wiki-ly supervised part-of-speech tagging
Shen Li, Joao Graça, and Ben Taskar. 2012 · 2012
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013 · 2013
Earlier work this paper cites.
SemEval-2013 task 12: Multilingual word sense disambiguation
Roberto Navigli, David Jurgens, and Daniele Vannella. 2013 · 2013
Earlier work this paper cites.
Recursive deep models for semantic compositionality over a sentiment treebank
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013 · 2013
Earlier work this paper cites.
Brainnet viewer: a network visualization tool for human brain connectomics
Mingrui Xia, Jinhui Wang, and Yong He. 2013 · 2013
Cited alongside, same era.
Good debt or bad debt: Detecting semantic orientations in economic texts
P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala. 2014 · 2014
Cited alongside, same era.
A SICK cure for the evaluation of compositional distributional semantic models
Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014 · 2014
Cited alongside, same era.
Class-based n-gram language difference models for data selection
Amittai Axelrod, Yogarshi Vyas, Marianna Martindale, and Marine Carpuat. 2015 · 2015
Cited alongside, same era.
A large annotated corpus for learning natural language inference
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015 · 2015
Cited alongside, same era.
Teaching machines to read and comprehend
BIGPATENT: A large-scale dataset for abstractive and coherent summarization
Eva Sharma, Chen Li, and Lu Wang. 2019 · 2019
Later among the works it cites.
DREAM: A challenge dataset and models for dialogue-based reading comprehension
Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. 2019 · 2019
Later among the works it cites.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019 · 2019
Later among the works it cites.
Benchmarking natural language understanding services for building conversational agents
Pawel Swietojanski Xingkun Liu, Arash Eshghi and Verena Rieser. 2019 · 2019
Later among the works it cites.
ERNIE: Enhanced language representation with informative entities
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019 · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015 · 2015
Cited alongside, same era.
Deep learning
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015 · 2015
Cited alongside, same era.
SemEval-2015 task 13: Multilingual all-words sense disambiguation and entity linking
Andrea Moro and Roberto Navigli. 2015 · 2015
Cited alongside, same era.
Character-level convolutional networks for text classification
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015a · 2015
Cited alongside, same era.
Semeval-2016 task 6: Detecting stance in tweets
Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016 · 2016
Cited alongside, same era.
Results of the WNUT2017 shared task on novel and emerging entity recognition
Leon Derczynski, Eric Nichols, Marieke van Erp, and Nut Limsopatham. 2017 · 2017
Cited alongside, same era.
Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017 · 2017
Cited alongside, same era.
HINT3: Raising the bar for intent detection in the wild
Gaurav Arora, Chirag Jain, Manas Chaturvedi, and Krupal Modi. 2020 · 2020
Later among the works it cites.
SLURP: A Spoken Language Understanding Resource Package
Emanuele Bastianelli, Andrea Vanzo, Pawel Swietojanski, and Verena Rieser. 2020 · 2020
Later among the works it cites.
TLDR: Extreme summarization of scientific documents
Isabel Cachola, Kyle Lo, Arman Cohan, and Daniel Weld. 2020 · 2020
Later among the works it cites.
Efficient intent detection with dual sentence encoders
Iñigo Casanueva, Tadas Temcinas, Daniela Gerz, Matthew Henderson, and Ivan Vulic. 2020 · 2020
Later among the works it cites.
Dynamic data selection and weighting for iterative back-translation
Zi-Yi Dou, Antonios Anastasopoulos, and Graham Neubig. 2020 · 2020
Later among the works it cites.
Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020 · 2020
Later among the works it cites.
SciREX: A challenge dataset for document-level information extraction
Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, and Iz Beltagy. 2020 · 2020
Later among the works it cites.
How can we know what language models know?
Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020 · 2020
Later among the works it cites.
Logiqa: A challenge dataset for machine reading comprehension with logical reasoning
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020b · 2020
Later among the works it cites.
Multi-XScience: A large-scale dataset for extreme multi-document summarization of scientific articles
Yao Lu, Yue Dong, and Laurent Charlin. 2020 · 2020
Later among the works it cites.
Adversarial nli: A new benchmark for natural language understanding
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020 · 2020
Later among the works it cites.
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020 · 2020
Later among the works it cites.
ERNIE 2.0: A continual pre-training framework for language understanding
Yu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020 · 2020
Later among the works it cites.
Temporal reasoning in natural language inference
Siddharth Vashishtha, Adam Poliak, Yash Kumar Lal, Benjamin Van Durme, and Aaron Steven White. 2020 · 2020
Later among the works it cites.
Reclor: A reading comprehension dataset requiring logical reasoning
Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. 2020 · 2020
Later among the works it cites.
Reasoning about goals, steps, and temporal ordering with WikiHow
Li Zhang, Qing Lyu, and Chris Callison-Burch. 2020 · 2020
Later among the works it cites.
WikiSum: Coherent summarization dataset for efficient human-evaluation
Nachshon Cohen, Oren Kalinsky, Yftah Ziser, and Alessandro Moschitti. 2021 · 2021
Later among the works it cites.
Efficient attentions for long document summarization
Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. 2021 · 2021
Later among the works it cites.
Towards data-centric machine learning: a short review
Lester James Miranda. 2021 · 2021
Later among the works it cites.
Multitask prompted training enables zero-shot task generalization
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Stella Biderman, Leo Gao, Tali Bers, Thomas Wolf, and Alexander M. Rush. 2021 · 2021
Later among the works it cites.
Fix your models by fixing your datasets
Atindriyo Sanyal, Vikram Chatterji, Nidhi Vyas, Ben Epstein, Nikita Demir, and Anthony Corletti. 2021 · 2021
Later among the works it cites.
Few-shot text generation with natural language instructions
Timo Schick and Hinrich Schütze. 2021b · 2021
Later among the works it cites.
ERNIE 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation
Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu, Weibao Gong, Jianzhong Liang, Zhizhou Shang, Peng Sun, Wei Liu, Xuan Ouyang, Dianhai Yu, Hao Tian, Hua Wu, and Haifeng Wang. 2021 · 2021
Later among the works it cites.
Multimodal few-shot learning with frozen language models
Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. 2021 · 2021
Later among the works it cites.
Bartscore: Evaluating generated text as text generation
Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021 · 2021
Later among the works it cites.
Calibrate before use: Improving few-shot performance of language models
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021 · 2021
Later among the works it cites.
Temporal reasoning on implicit events from distant supervision
Ben Zhou, Kyle Richardson, Qiang Ning, Tushar Khot, Ashish Sabharwal, and Dan Roth. 2021 · 2021
Later among the works it cites.
Polyglot prompt: Multilingual multitask promptraining
Jinlan Fu, See-Kiong Ng, and Pengfei Liu. 2022 · 2022
Closest in time.
PPT: Pre-trained prompt tuning for few-shot learning
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2022 · 2022
Closest in time.
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al. 2022 · 2022
Closest in time.
DataLab: A platform for data analysis and intervention
Yang Xiao, Jinlan Fu, Weizhe Yuan, Vijay Viswanathan, Zhoumianze Liu, Yixin Liu, Graham Neubig, and Pengfei Liu. 2022 · 2022
Closest in time.