Fetching the paper…

Reading the bibliography…

2021

RoBERTa-wwm-ext Fine-Tuning for Chinese Text Classification

Xu, Zhuo

Original
PDF
Post on X

Understand

Bidirectional Encoder Representations from Transformers (BERT) have shown to be a promising way to dramatically improve the performance across various Natural Language Processing tasks [Devlin et al., 2019].

Meanwhile, progress made over the past few years by various Neural Net-work has also proved the effectiveness of Neural Network in the field of Natural Language Processing.
In this project, RoBERTa-wwm-ext [Cui et al., 2019] pre-train language model was adopted and fine-tuned for Chinese text classification.
The models were able to classify Chinese texts into two categories, containing descriptions of legal behavior and descriptions of illegal behavior.

RoBERTa-wwm-ext Fine-Tuning for Chinese Text Classification · Around

Built on

Convolutional neural networks for sentence classification
Original
Kim, Y. (2014) · 2014
Earlier work this paper cites.
Recurrent neural network for text classification with multi-task learning
Original
Liu, P., X. Qiu, and X. Huang (2016) · 2016
Earlier work this paper cites.

Similar

Deep pyramid convolutional neural networks for text categorization
Johnson, R. and T. Zhang (2017, July) · 2017
Cited alongside, same era.
Pre-training with whole word masking for chinese bert
Cui, Y., W. Che, T. Liu, B. Qin, Z. Yang, S. Wang, and G. Hu (2019) · 2019
Cited alongside, same era.
Recurrent convolutional neural networks for text classification
Lai, S., L. Xu, K. Liu, and J. Zhao
Cited in the paper.

Then

BERT: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova (2019, June) · 2019
Later among the works it cites.
Roberta: A robustly optimized bert pretraining approach
Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019) · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…