Built on

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” in Neural Information Processing Systems , 2020, pp. 1877–1901
1901
Earlier work this paper cites.
X.-H. Phan, L.-M. Nguyen, and S. Horiguchi, “Learning to classify short and sparse text & web with hidden topics from large-scale data collections,” in International Conference on World Wide Web , 2008, pp. 91–100
2008
Earlier work this paper cites.
J. Hu, G. Wang, F. Lochovsky, J.-t. Sun, and Z. Chen, “Understanding user’s query intent with wikipedia,” in International Conference on World Wide Web , 2009, pp. 471–480
2009
Earlier work this paper cites.
B. Sriram, D. Fuhry, E. Demir, H. Ferhatosmanoglu, and M. Demirbas, “Short text classification in twitter to improve information filtering,” in International ACM SIGIR conference on Research and development in information retrieval , 2010, pp. 841–842
2010
Earlier work this paper cites.
Z. Liu, W. Yu, W. Chen, S. Wang, and F. Wu, “Short text feature selection for micro-blog mining,” in International Conference on Computational Intelligence and Software Engineering , 2010, pp. 1–4
2010
Earlier work this paper cites.
A. Sun, “Short text classification using very few words,” in International ACM SIGIR conference on Research and development in information retrieval , 2012, pp. 1145–1146
2012
Earlier work this paper cites.
Z. Lu and H. Li, “A deep architecture for matching short texts,” Advances in neural information processing systems , vol. 26, pp. 1367––1375, 2013
2013
Earlier work this paper cites.
M. Faqeeh, N. Abdulla, M. Al-Ayyoub, Y. Jararweh, and M. Quwaider, “Cross-lingual short-text document classification for facebook comments,” in International Conference on Future Internet of Things and Cloud , 2014, pp. 573–578
2014
Earlier work this paper cites.
Y. Kim, “Convolutional neural networks for sentence classification. corr abs/1408.5882 (2014),” arXiv preprint arXiv:1408.5882 , 2014
Original
2014
Earlier work this paper cites.
S. Lai, L. Xu, K. Liu, and J. Zhao, “Recurrent convolutional neural networks for text classification,” in AAAI conference on artificial intelligence , 2015
2015
Earlier work this paper cites.
X. Zhang, J. Zhao, and Y. LeCun, “Character-level convolutional networks for text classification,” Advances in neural information processing systems , vol. 28, pp. 649–657, 2015
2015
Earlier work this paper cites.
D. Bollegala, V. Atanasov, T. Maehara, and K.-i. Kawarabayashi, “Classinet–predicting missing features for short-text classification,” ACM Transactions on Knowledge Discovery from Data (TKDD) , vol. 12, no. 5, pp. 1–29, 2018
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.
J. Chen, Y. Hu, J. Liu, Y. Xiao, and H. Jiang, “Deep short text classification with knowledge powered attention,” in AAAI Conference on Artificial Intelligence , vol. 33, no. 01, 2019, pp. 6252–6259
2019
Earlier work this paper cites.
J. Xu and Y. Cai, “Incorporating context-relevant knowledge into convolutional neural networks for short text classification,” in AAAI Conference on Artificial Intelligence , vol. 33, no. 01, 2019, pp. 10 067–10 068
2019
Earlier work this paper cites.
W. Yang, Y. Xie, L. Tan, K. Xiong, M. Li, and J. Lin, “Data augmentation for bert fine-tuning in open-domain question answering,” arXiv preprint arXiv:1904.06652 , 2019
Original
2019
Earlier work this paper cites.
Y. Goldberg, “Assessing bert’s syntactic abilities,” arXiv preprint arXiv:1901.05287 , 2019
Original
2019
Earlier work this paper cites.
X. Ma, Z. Wang, P. Ng, R. Nallapati, and B. Xiang, “Universal text representation from bert: An empirical study,” arXiv preprint arXiv:1910.07973 , 2019
Original
2019
Earlier work this paper cites.

Similar

G. Jawahar, B. Sagot, and D. Seddah, “What does bert learn about the structure of language?” in Annual Meeting of the Association for Computational Linguistics , 2019, pp. 3651–3657
2019
Cited alongside, same era.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to fine-tune bert for text classification?” in China national conference on Chinese computational linguistics , 2019, pp. 194–206
2019
Cited alongside, same era.
F. Petroni, T. Rocktäschel, P. Lewis, A. Bakhtin, Y. Wu, A. H. Miller, and S. Riedel, “Language models as knowledge bases?” arXiv preprint arXiv:1909.01066 , 2019
Original
2019
Cited alongside, same era.
J. Davison, J. Feldman, and A. M. Rush, “Commonsense knowledge mining from pretrained models,” in EMNLP-IJCNLP , 2019, pp. 1173–1178
2019
Cited alongside, same era.
A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Superglue: A stickier benchmark for general-purpose language understanding systems,” arXiv preprint arXiv:1905.00537 , 2019

Then

J. Zhu, Y. Xia, L. Wu, D. He, T. Qin, W. Zhou, H. Li, and T.-Y. Liu, “Incorporating bert into neural machine translation,” arXiv preprint arXiv:2002.06823 , 2020
Original
2020
Later among the works it cites.
R. Bao, J. Wang, Z. Zhang, and H. Zhao, “Enhancing pre-trained language model with lexical simplification,” arXiv preprint arXiv:2012.15070 , 2020
Original
2020
Later among the works it cites.
Z. Ye, G. Jiang, Y. Liu, Z. Li, and J. Yuan, “Document and word representations generated by graph convolutional network and bert for short text classification,” in European Conference on Artificial Intelligence , 2020, pp. 2275–2281
2020
Later among the works it cites.
T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh, “Autoprompt: Eliciting knowledge from language models with automatically generated prompts,” arXiv preprint arXiv:2010.15980 , 2020
Original
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2022

Prompt-Learning for Short Text Classification

Zhu, Yi, Zhou, Xinke, Qiang, Jipeng et al.

Understand

In the short text, the extremely short length, feature sparsity, and high ambiguity pose huge challenges to classification tasks.

Recently, as an effective method for tuning Pre-trained Language Models for specific downstream tasks, prompt-learning has attracted a vast amount of attention and research.
The main intuition behind the prompt-learning is to insert the template into the input and convert the text classification tasks into equivalent cloze-style tasks.
However, most prompt-learning methods expand label words manually or only consider the class name for knowledge incorporating in cloze-style prediction, which will inevitably incur omissions and bias in short text classification tasks.

Built on

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” in Neural Information Processing Systems , 2020, pp. 1877–1901
1901
Earlier work this paper cites.
X.-H. Phan, L.-M. Nguyen, and S. Horiguchi, “Learning to classify short and sparse text & web with hidden topics from large-scale data collections,” in International Conference on World Wide Web , 2008, pp. 91–100
2008
Earlier work this paper cites.
J. Hu, G. Wang, F. Lochovsky, J.-t. Sun, and Z. Chen, “Understanding user’s query intent with wikipedia,” in International Conference on World Wide Web , 2009, pp. 471–480
2009
Earlier work this paper cites.
B. Sriram, D. Fuhry, E. Demir, H. Ferhatosmanoglu, and M. Demirbas, “Short text classification in twitter to improve information filtering,” in International ACM SIGIR conference on Research and development in information retrieval , 2010, pp. 841–842
2010
Earlier work this paper cites.
Z. Liu, W. Yu, W. Chen, S. Wang, and F. Wu, “Short text feature selection for micro-blog mining,” in International Conference on Computational Intelligence and Software Engineering , 2010, pp. 1–4
2010
Earlier work this paper cites.
A. Sun, “Short text classification using very few words,” in International ACM SIGIR conference on Research and development in information retrieval , 2012, pp. 1145–1146
2012
Earlier work this paper cites.
Z. Lu and H. Li, “A deep architecture for matching short texts,” Advances in neural information processing systems , vol. 26, pp. 1367––1375, 2013
2013
Earlier work this paper cites.
M. Faqeeh, N. Abdulla, M. Al-Ayyoub, Y. Jararweh, and M. Quwaider, “Cross-lingual short-text document classification for facebook comments,” in International Conference on Future Internet of Things and Cloud , 2014, pp. 573–578
2014
Earlier work this paper cites.
Y. Kim, “Convolutional neural networks for sentence classification. corr abs/1408.5882 (2014),” arXiv preprint arXiv:1408.5882 , 2014
Original
2014
Earlier work this paper cites.
S. Lai, L. Xu, K. Liu, and J. Zhao, “Recurrent convolutional neural networks for text classification,” in AAAI conference on artificial intelligence , 2015
2015
Earlier work this paper cites.
X. Zhang, J. Zhao, and Y. LeCun, “Character-level convolutional networks for text classification,” Advances in neural information processing systems , vol. 28, pp. 649–657, 2015
2015
Earlier work this paper cites.
D. Bollegala, V. Atanasov, T. Maehara, and K.-i. Kawarabayashi, “Classinet–predicting missing features for short-text classification,” ACM Transactions on Knowledge Discovery from Data (TKDD) , vol. 12, no. 5, pp. 1–29, 2018
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.
J. Chen, Y. Hu, J. Liu, Y. Xiao, and H. Jiang, “Deep short text classification with knowledge powered attention,” in AAAI Conference on Artificial Intelligence , vol. 33, no. 01, 2019, pp. 6252–6259
2019
Earlier work this paper cites.
J. Xu and Y. Cai, “Incorporating context-relevant knowledge into convolutional neural networks for short text classification,” in AAAI Conference on Artificial Intelligence , vol. 33, no. 01, 2019, pp. 10 067–10 068
2019
Earlier work this paper cites.
W. Yang, Y. Xie, L. Tan, K. Xiong, M. Li, and J. Lin, “Data augmentation for bert fine-tuning in open-domain question answering,” arXiv preprint arXiv:1904.06652 , 2019
Original
2019
Earlier work this paper cites.
Y. Goldberg, “Assessing bert’s syntactic abilities,” arXiv preprint arXiv:1901.05287 , 2019
Original
2019
Earlier work this paper cites.
X. Ma, Z. Wang, P. Ng, R. Nallapati, and B. Xiang, “Universal text representation from bert: An empirical study,” arXiv preprint arXiv:1910.07973 , 2019
Original
2019
Earlier work this paper cites.

Similar

G. Jawahar, B. Sagot, and D. Seddah, “What does bert learn about the structure of language?” in Annual Meeting of the Association for Computational Linguistics , 2019, pp. 3651–3657
2019
Cited alongside, same era.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to fine-tune bert for text classification?” in China national conference on Chinese computational linguistics , 2019, pp. 194–206
2019
Cited alongside, same era.
F. Petroni, T. Rocktäschel, P. Lewis, A. Bakhtin, Y. Wu, A. H. Miller, and S. Riedel, “Language models as knowledge bases?” arXiv preprint arXiv:1909.01066 , 2019
Original
2019
Cited alongside, same era.
J. Davison, J. Feldman, and A. M. Rush, “Commonsense knowledge mining from pretrained models,” in EMNLP-IJCNLP , 2019, pp. 1173–1178
2019
Cited alongside, same era.
A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Superglue: A stickier benchmark for general-purpose language understanding systems,” arXiv preprint arXiv:1905.00537 , 2019

Then

J. Zhu, Y. Xia, L. Wu, D. He, T. Qin, W. Zhou, H. Li, and T.-Y. Liu, “Incorporating bert into neural machine translation,” arXiv preprint arXiv:2002.06823 , 2020
Original
2020
Later among the works it cites.
R. Bao, J. Wang, Z. Zhang, and H. Zhao, “Enhancing pre-trained language model with lexical simplification,” arXiv preprint arXiv:2012.15070 , 2020
Original
2020
Later among the works it cites.
Z. Ye, G. Jiang, Y. Liu, Z. Li, and J. Yuan, “Document and word representations generated by graph convolutional network and bert for short text classification,” in European Conference on Artificial Intelligence , 2020, pp. 2275–2281
2020
Later among the works it cites.
T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh, “Autoprompt: Eliciting knowledge from language models with automatically generated prompts,” arXiv preprint arXiv:2010.15980 , 2020
Original
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

2019

Cited alongside, same era.

H. Linmei, T. Yang, C. Shi, H. Ji, and X. Li, “Heterogeneous graph attention networks for semi-supervised short text classification,” in EMNLP-IJCNLP , 2019, pp. 4821–4830

2019

Cited alongside, same era.

Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692 , 2019

Original

2019

Cited alongside, same era.

R. Weng, H. Yu, S. Huang, S. Cheng, and W. Luo, “Acquiring knowledge from pre-trained model to neural machine translation,” in AAAI Conference on Artificial Intelligence , vol. 34, no. 05, 2020, pp. 9266–9273

2020

Cited alongside, same era.

J. Qiang, Y. Li, Y. Zhu, Y. Yuan, and X. Wu, “Lexical simplification with pretrained encoders,” in AAAI Conference on Artificial Intelligence , vol. 34, no. 05, 2020, pp. 8649–8656

2020

Cited alongside, same era.

T. Gao, A. Fisch, and D. Chen, “Making pre-trained language models better few-shot learners,” arXiv preprint arXiv:2012.15723 , 2020

Original

2020

Cited alongside, same era.

T. Schick, H. Schmid, and H. Schütze, “Automatically identifying words that can serve as labels for few-shot text classification,” arXiv preprint arXiv:2010.13641 , 2020

Original

2020

Cited alongside, same era.

T. Schick and H. Schütze, “Exploiting cloze questions for few shot text classification and natural language inference,” arXiv preprint arXiv:2001.07676 , 2020

Original

2020

Cited alongside, same era.

Y. Cui, W. Che, T. Liu, B. Qin, and Z. Yang, “Pre-training with whole word masking for chinese bert,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 3504–3514, 2021

2021