Fetching the paper…
Reading the bibliography…
Vast efforts have been devoted to creating high-performance few-shot learners, i.e., large-scale pretrained language models (PLMs) that perform well with little downstream task training data.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 1901
Earlier work this paper cites.
Building a large annotated corpus of English: The Penn Treebank
Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993 · 1993
Earlier work this paper cites.
A semantic concordance
George A. Miller, Claudia Leacock, Randee Tengi, and Ross T. Bunker. 1993 · 1993
Earlier work this paper cites.
Improving generalization with active learning
David Cohn, Les Atlas, and Richard Ladner. 1994 · 1994
Earlier work this paper cites.
A sequential algorithm for training text classifiers
David D Lewis and William A Gale. 1994 · 1994
Earlier work this paper cites.
Unsupervised word sense disambiguation rivaling supervised methods
David Yarowsky. 1995 · 1995
Earlier work this paper cites.
Active learning with statistical models
David A Cohn, Zoubin Ghahramani, and Michael I Jordan. 1996 · 1996
Earlier work this paper cites.
Scaling laws for neural language models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020 · 2001
Earlier work this paper cites.
Toward optimal active learning through monte carlo estimation of error reduction
Nicholas Roy and Andrew McCallum. 2001 · 2001
Earlier work this paper cites.
Understanding the Yarowsky algorithm
Steven Abney. 2004 · 2004
Earlier work this paper cites.
Investigating the effects of selective sampling on the annotation task
Ben Hachey, Beatrice Alex, and Markus Becker. 2005 · 2005
Earlier work this paper cites.
The pascal recognising textual entailment challenge
Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006 · 2006
Earlier work this paper cites.
The rise of crowdsourcing
Jeff Howe et al. 2006 · 2006
Earlier work this paper cites.
Crowdsourcing for relevance evaluation
Omar Alonso, Daniel E. Rose, and Benjamin Stewart. 2008 · 2008
Earlier work this paper cites.
Crowdsourcing: How the power of the crowd is driving the future of business
Jeff Howe. 2008 · 2008
Earlier work this paper cites.
A survey of active learning for text classification using deep neural networks
Christopher Schröder and Andreas Niekler. 2020 · 2008
Earlier work this paper cites.
An analysis of active learning strategies for sequence labeling tasks
Burr Settles and Mark Craven. 2008 · 2008
Earlier work this paper cites.
Cheap and fast – but is it good? evaluating non-expert annotations for natural language tasks
Rion Snow, Brendan O’Connor, Daniel Jurafsky, and Andrew Ng. 2008 · 2008
Earlier work this paper cites.
Fast, cheap, and creative: Evaluating translation quality using Amazon’s Mechanical Turk
Chris Callison-Burch. 2009 · 2009
Earlier work this paper cites.
Active learning: An introduction
Richard M Felder and Rebecca Brent. 2009 · 2009
Earlier work this paper cites.
Active learning literature survey
Burr Settles. 2009 · 2009
Earlier work this paper cites.
Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk . Association for Computational Linguistics, Los Angeles
Chris Callison-Burch and Mark Dredze, editors. 2010 · 2010
Earlier work this paper cites.
Active learning with Amazon Mechanical Turk
Florian Laws, Christian Scheible, and Hinrich Schütze. 2011 · 2011
Earlier work this paper cites.
Choice of plausible alternatives: An evaluation of commonsense causal reasoning
Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. 2011 · 2011
Earlier work this paper cites.
A survey of crowdsourcing systems
Man-Ching Yuen, Irwin King, and Kwong-Sak Leung. 2011 · 2011
Earlier work this paper cites.
Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks
Dong-Hyun Lee et al. 2013 · 2013
Earlier work this paper cites.
Recursive deep models for semantic compositionality over a sentiment treebank
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013 · 2013
Earlier work this paper cites.
Question-answer driven semantic role labeling: Using natural language to annotate natural language
Luheng He, Mike Lewis, and Luke Zettlemoyer. 2015 · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015 · 2015
Earlier work this paper cites.
Noise or additional information? leveraging crowdsource annotation item agreement for natural language tasks
Emily Jamison and Iryna Gurevych. 2015 · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba. 2015 · 2015
Earlier work this paper cites.
Character-level convolutional networks for text classification
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015 · 2015
Earlier work this paper cites.
On calibration of modern neural networks
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017 · 2017
Earlier work this paper cites.
Deep active learning for named entity recognition
Yanyao Shen, Hyokun Yun, Zachary Lipton, Yakov Kronrod, and Animashree Anandkumar. 2017 · 2017
Earlier work this paper cites.
Instance weighting for neural machine translation domain adaptation
Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. 2017 · 2017
Cited alongside, same era.
Active discriminative text representation learning
Ye Zhang, Matthew Lease, and Byron C. Wallace. 2017 · 2017
Cited alongside, same era.
Comparing Bayesian models of annotation
Silviu Paun, Bob Carpenter, Jon Chamberlain, Dirk Hovy, Udo Kruschwitz, and Massimo Poesio. 2018 · 2018
Cited alongside, same era.
Deep contextualized word representations
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 · 2018
Cited alongside, same era.
Deep Bayesian active learning for natural language processing: Results of a large-scale empirical study
Aditya Siddhant and Zachary C. Lipton. 2018 · 2018
Cited alongside, same era.
Finding convincing arguments using scalable Bayesian preference learning
Dataset cartography: Mapping and diagnosing datasets with training dynamics
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi. 2020 · 2020
Later among the works it cites.
Fine-grained argument unit recognition and classification
Dietrich Trautmann, Johannes Daxenberger, Christian Stab, Hinrich Schütze, and Iryna Gurevych. 2020 · 2020
Later among the works it cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020 · 2020
Later among the works it cites.
CLUE: A Chinese language understanding evaluation benchmark
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, and Zhenzhong Lan. 2020 · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Edwin Simpson and Iryna Gurevych. 2018 · 2018
Cited alongside, same era.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018 · 2018
Cited alongside, same era.
Analysis methods in neural language processing: A survey
Yonatan Belinkov and James Glass. 2019 · 2019
Cited alongside, same era.
CaRB: A crowdsourced benchmark for open IE
Sangnie Bhardwaj, Samarth Aggarwal, and Mausam Mausam. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Designing and interpreting probes with control tasks
John Hewitt and Percy Liang. 2019 · 2019
Cited alongside, same era.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D. Manning. 2019 · 2019
Cited alongside, same era.
Universal natural language processing with limited annotations: Try few-shot textual entailment as a start
Wenpeng Yin, Nazneen Fatema Rajani, Dragomir Radev, Richard Socher, and Caiming Xiong. 2020 · 2020
Later among the works it cites.
Quantifying the contextualization of word representations with semantic class probing
Mengjie Zhao, Philipp Dufter, Yadollah Yaghoobzadeh, and Hinrich Schütze. 2020a · 2020
Later among the works it cites.
Masking as an efficient alternative to finetuning for pretrained language models
Mengjie Zhao, Tao Lin, Fei Mi, Martin Jaggi, and Hinrich Schütze. 2020b · 2020
Later among the works it cites.
MasakhaNER: Named Entity Recognition for African Languages
David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen H. Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Aremu Anuoluwapo, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Rabiu Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, Verrah Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, Chiamaka Chukwuneke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, and Salomey Osei. 2021 · 2021
Closest in time.
Pre-train or annotate? domain adaptation with a constrained budget
Fan Bai, Alan Ritter, and Wei Xu. 2021 · 2021
Closest in time.
On the opportunities and risks of foundation models
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021 · 2021
Closest in time.
Empirical methodology for crowdsourcing ground truth
Anca Dumitrache, Oana Inel, Benjamin Timmermans, Carlos Ortiz, Robert-Jan Sips, Lora Aroyo, and Chris Welty. 2021 · 2021
Closest in time.
Making pre-trained language models better few-shot learners
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021 · 2021
Closest in time.
Multilingual LAMA: Investigating knowledge in multilingual pretrained language models
Nora Kassner, Philipp Dufter, and Hinrich Schütze. 2021 · 2021
Closest in time.
ParsiNLU: A suite of language understanding challenges for Persian
Daniel Khashabi, Arman Cohan, Siamak Shakeri, Pedram Hosseini, Pouya Pezeshkpour, Malihe Alikhani, Moin Aminnaseri, Marzieh Bitaab, Faeze Brahman, Sarik Ghazarian, Mozhdeh Gheini, Arman Kabiri, Rabeeh Karimi Mahabagdi, Omid Memarrast, Ahmadreza Mosallanezhad, Erfan Noury, Shahab Raji, Mohammad Sadegh Rasooli, Sepideh Sadeghi, Erfan Sadeqi Azer, Niloofar Safi Samghabadi, Mahsa Shafaei, Saber Sheybani, Ali Tazarv, and Yadollah Yaghoobzadeh. 2021 · 2021
Closest in time.
Annotation curricula to implicitly train non-expert annotators
Ji-Ung Lee, Jan-Christoph Klie, and Iryna Gurevych. 2021 · 2021
Closest in time.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021 · 2021
Closest in time.
Prefix-tuning: Optimizing continuous prompts for generation
Xiang Lisa Li and Percy Liang. 2021 · 2021
Closest in time.
Few-shot learning with multilingual language models
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, et al. 2021 · 2021
Closest in time.
Active learning by acquiring contrastive examples
Katerina Margatina, Giorgos Vernikos, Loïc Barrault, and Nikolaos Aletras. 2021 · 2021
Closest in time.
Self-training improves pre-training for few-shot learning in task-oriented dialog systems
Fei Mi, Wanhao Zhou, Lingjing Kong, Fengyu Cai, Minlie Huang, and Boi Faltings. 2021b · 2021
Closest in time.
Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
Robert Munro Monarch. 2021 · 2021
Closest in time.
What ingredients make for an effective crowdsourcing protocol for difficult NLU data collection tasks?
Nikita Nangia, Saku Sugawara, Harsh Trivedi, Alex Warstadt, Clara Vania, and Samuel R. Bowman. 2021 · 2021
Closest in time.
KLUE: Korean language understanding evaluation
Sungjoon Park, Jihyung Moon, Sung-Dong Kim, Won Ik Cho, Jiyoon Han, Jangwon Park, Chisung Song, Junseong Kim, Yongsook Song, Tae Hwan Oh, Joohong Lee, Juhyun Oh, Sungwon Lyu, Young kuk Jeong, Inkwon Lee, Sang gyu Seo, Dongjun Lee, Hyunwoo Kim, Myeonghwa Lee, Seongbo Jang, Seungwon Do, Sunkyoung Kim, Kyungtae Lim, Jongwon Lee, Kyumin Park, Jamin Shin, Seonghyun Kim, Lucy Park, Alice H. Oh, Jung-Woo Ha, and Kyunghyun Cho. 2021 · 2021
Closest in time.
Carbon emissions and large neural network training
David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. 2021 · 2021
Closest in time.
It’s not just size that matters: Small language models are also few-shot learners
Timo Schick and Hinrich Schütze. 2021b · 2021
Closest in time.
Improving and simplifying pattern exploiting training
Derek Tam, Rakesh R. Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. 2021 · 2021
Closest in time.
Multimodal few-shot learning with frozen language models
Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. M. Ali Eslami, Oriol Vinyals, and Felix Hill. 2021 · 2021
Closest in time.
Want to reduce labeling cost? GPT-3 can help
Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021 · 2021
Closest in time.
Language models are few-shot multilingual learners
Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021 · 2021
Closest in time.
GPT3Mix: Leveraging large-scale language models for text augmentation
Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, and Woomyoung Park. 2021 · 2021
Closest in time.
Cartography active learning
Mike Zhang and Barbara Plank. 2021 · 2021
Closest in time.
Discrete and soft prompting for multilingual models
Mengjie Zhao and Hinrich Schütze. 2021 · 2021
Closest in time.
A closer look at few-shot crosslingual transfer: The choice of shots matters
Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, and Hinrich Schütze. 2021 · 2021
Closest in time.