Fetching the paper…
Reading the bibliography…
Large language models (LLMs) can label data faster and cheaper than humans for various NLP tasks.
Challenges in data crowdsourcing
Hector Garcia-Molina, Manas Joglekar, Adam Marcus, Aditya Parameswaran, and Vasilis Verroios. 2016 · 2016
Earlier work this paper cites.
Jupyter notebooks – a publishing format for reproducible computational workflows
Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Damián Avila, Safia Abdalla, and Carol Willing. 2016 · 2016
Earlier work this paper cites.
The INCEpTION platform: Machine-assisted and knowledge-oriented interactive annotation
Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart de Castilho, and Iryna Gurevych. 2018 · 2018
Earlier work this paper cites.
Prodigy: A new annotation tool for radically efficient machine teaching
Ines Montani and Matthew Honnibal. 2018 · 2018
Earlier work this paper cites.
Crowdsourcing research: data collection with amazon’s mechanical turk
Kim Bartel Sheehan. 2018 · 2018
Earlier work this paper cites.
Persistent anti-muslim bias in large language models
Abubakar Abid, Maheen Farooqi, and James Zou. 2021 · 2021
Earlier work this paper cites.
Reply to mturk, prolific or panels? choosing the right audience for online research
Leib Litman, Aaron Moss, Cheskie Rosenzweig, and Jonathan Robinson. 2021 · 2021
Earlier work this paper cites.
Societal biases in language generation: Progress and challenges
Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2021 · 2021
Earlier work this paper cites.
Want to reduce labeling cost? GPT-3 can help
Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021 · 2021
Earlier work this paper cites.
Calibrate before use: Improving few-shot performance of language models
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021 · 2021
Earlier work this paper cites.
Teaching models to express their uncertainty in words
Stephanie Lin, Jacob Hilton, and Owain Evans. 2022 · 2022
Cited alongside, same era.
Label sleuth: From unlabeled text to a classifier in a few hours
Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, and Dakuo Wang. 2022 · 2022
Cited alongside, same era.
Label Studio: Data labeling software
Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. 2020-2022 · 2022
Cited alongside, same era.
MEGAnno: Exploratory labeling for NLP in computational notebooks
Dan Zhang, Hannah Kim, Rafael Li Chen, Eser Kandogan, and Estevam Hruschka. 2022 · 2022
Cited alongside, same era.
How is chatgpt’s behavior changing over time?
Lingjiao Chen, Matei Zaharia, and James Zou. 2023 · 2023
Cited alongside, same era.
Taja Kuzman, Igor Mozetič, and Nikola Ljubešić. 2023 · 2023
Later among the works it cites.
Self-refine: Iterative refinement with self-feedback
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023 · 2023
Later among the works it cites.
Large language models sensitivity to the order of options in multiple-choice questions
Pouya Pezeshkpour and Estevam Hruschka. 2023 · 2023
Later among the works it cites.
Petter Törnberg. 2023 · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2023 · 2023
Cited alongside, same era.
Is GPT-3 a good data annotator?
Bosheng Ding, Chengwei Qin, Linlin Liu, Yew Ken Chia, Boyang Li, Shafiq Joty, and Lidong Bing. 2023 · 2023
Cited alongside, same era.
Data quality in online human-subjects research: Comparisons between mturk, prolific, cloudresearch, qualtrics, and sona
Benjamin D Douglas, Patrick J Ewell, and Markus Brauer. 2023 · 2023
Cited alongside, same era.
Chatgpt outperforms crowd workers for text-annotation tasks
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023 · 2023
Cited alongside, same era.
AnnoLLM: Making large language models to be better crowdsourced annotators
Xingwei He, Zhenghao Lin, Yeyun Gong, A-Long Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. 2023 · 2023
Cited alongside, same era.
Autolabel
Cited in the paper.
Humanloop.com
Cited in the paper.
Self-consistency improves chain of thought reasoning in language models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 · 2023
Later among the works it cites.
Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms
Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2023 · 2023
Later among the works it cites.
Can chatgpt reproduce human-generated labels? a study of social computing tasks
Yiming Zhu, Peixian Zhang, Ehsan-Ul Haq, Pan Hui, and Gareth Tyson. 2023 · 2023
Later among the works it cites.
Can Large Language Models Transform Computational Social Science?
Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2023 · 2023
Later among the works it cites.
Human-LLM collaborative annotation through effective verification of LLM labels
Xinru Wang, Hannah Kim, Sajjadur Rahman, Kushan Mitra, and Zhengjie Miao. 2024 · 2024
Closest in time.