Understand
In this paper, we investigate the use of data obtained from prompting a large generative language model, ChatGPT, to generate synthetic training data with the aim of augmenting data in low resource scenarios.
- We show that with appropriate task-specific ChatGPT prompts, we outperform the most popular existing approaches for such data augmentation.
- Furthermore, we investigate methodologies for evaluating the similarity of the augmented data generated from ChatGPT with the aim of validating and assessing the quality of the data generated.
Built on
“Learning question classifiers”
Xin Li and Dan Roth · 2002
Earlier work this paper cites.
“Recursive deep models for semantic compositionality over a sentiment treebank”
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng and Christopher Potts · 2013
Earlier work this paper cites.
“Improving neural machine translation models with monolingual data”
Rico Sennrich, Barry Haddow and Alexandra Birch · 2015
Earlier work this paper cites.
Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone and Thibaut Lavril · 2018
Earlier work this paper cites.
“Bert: Pre-training of deep bidirectional transformers for language understanding”
Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova · 2018
Earlier work this paper cites.
Similar
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer · 2019
Cited alongside, same era.
“Language models are unsupervised multitask learners”
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever · 2019
Cited alongside, same era.
“Eda: Easy data augmentation techniques for boosting performance on text classification tasks”
Jason Wei and Kai Zou · 2019
Cited alongside, same era.
“Conditional bert contextual augmentation”
Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han and Songlin Hu · 2019
Cited alongside, same era.
“Data augmentation using pre-trained transformer models”
Varun Kumar, Ashutosh Choudhary and Eunah Cho · 2020
Cited alongside, same era.
Then
“Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers”
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang and Ming Zhou · 2020
Later among the works it cites.
“Chataug: Leveraging chatgpt for text data augmentation”
Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Zihao Wu, Lin Zhao, Wei Liu, Ninghao Liu, Sheng Li and Dajiang Zhu · 2023
Closest in time.
“GPT-3.5” [Accessed: 07.04.2023]
OpenAI · 2023
Closest in time.
“Introducing ChatGPT.” [Accessed: 07.04.2023]
OpenAI · 2023
Closest in time.
“SentenceTransformers Documentation” [Accessed: 07.04.2023]
SBERT.net · 2023
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…