2017

Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF

Shao, Yan, Hardmeier, Christian, Tiedemann, Jörg et al.

Understand

We present a character-based model for joint segmentation and POS tagging for Chinese.

The bidirectional RNN-CRF architecture for general sequence tagging is adapted and applied with novel vector representations of Chinese characters that capture rich contextual information and lower-than-character level features.
The proposed model is extensively evaluated and compared with a state-of-the-art tagger respectively on CTB5, CTB9 and UD Chinese.
The experimental results indicate that our model is accurate and robust across datasets in different sizes, genres and annotation schemes.

Built on

Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014 · 1958
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
Conditional random fields: Probabilistic models for segmenting and labeling sequence data
John Lafferty, Andrew McCallum, Fernando Pereira, et al. 2001 · 2001
Earlier work this paper cites.
Chinese part-of-speech tagging: One-at-a-time or all-at-once? word-based or character-based?
Hwee Tou Ng and Jin Kiat Low. 2004 · 2004
Earlier work this paper cites.
The Penn Chinese treebank: Phrase structure annotation of a large corpus
Naiwen Xue, Fei Xia, Fu-Dong Chiou, and Marta Palmer. 2005 · 2005
Earlier work this paper cites.
A cascaded linear model for joint Chinese word segmentation and part-of-speech tagging
Wenbin Jiang, Liang Huang, Qun Liu, and Yajuan Lü. 2008 · 2008
Earlier work this paper cites.
Joint word segmentation and POS tagging using a single perceptron
Yue Zhang and Stephen Clark. 2008 · 2008
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio. 2010 · 2010
Earlier work this paper cites.
A fast decoder for joint word segmentation and POS-tagging using a single discriminative model
Yue Zhang and Stephen Clark. 2010 · 2010
Earlier work this paper cites.
Natural language processing (almost) from scratch
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011 · 2011
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
John Duchi, Elad Hazan, and Yoram Singer. 2011 · 2011
Earlier work this paper cites.
A stacked sub-word model for joint Chinese word segmentation and part-of-speech tagging
Weiwei Sun. 2011 · 2011
Earlier work this paper cites.

Similar

Improving Chinese word segmentation and POS tagging with semi-supervised methods using large auto-analyzed data
Yiou Wang, Jun’ichi Kazama, Yoshimasa Tsuruoka, Wenliang Chen, Yujie Zhang, and Kentaro Torisawa. 2011 · 2011
Cited alongside, same era.
The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional
Morten W Fagerland, Stian Lydersen, and Petter Laake. 2013 · 2013
Cited alongside, same era.
Efficient estimation of word representations in vector space
Original
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 · 2013
Cited alongside, same era.
Deep learning for Chinese word segmentation and POS tagging
Xiaoqing Zheng, Hanyang Chen, and Tianyu Xu. 2013 · 2013
Cited alongside, same era.
On the properties of neural machine translation: Encoder-decoder approaches
Original
Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014 · 2014
Cited alongside, same era.
Empirical evaluation of gated recurrent neural networks on sequence modeling
Original
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014 · 2014
Cited alongside, same era.
Learning character-level representations for part-of-speech tagging
Cícero Nogueira dos Santos and Bianca Zadrozny. 2014 · 2014
Cited alongside, same era.
GloVe: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014 · 2014
Cited alongside, same era.
Chinese morphological analysis with character-level POS tagging
Mo Shen, Hongxiao Liu, Daisuke Kawahara, and Sadao Kurohashi. 2014 · 2014
Cited alongside, same era.
Radical-enhanced Chinese character embedding
Yaming Sun, Lei Lin, Nan Yang, Zhenzhou Ji, and Xiaolong Wang. 2014 · 2014
Cited alongside, same era.
Bidirectional LSTM-CRF models for sequence tagging
Original
Zhiheng Huang, Wei Xu, and Kai Yu. 2015 · 2015
Cited alongside, same era.
An error-driven word-character hybrid model for joint Chinese word segmentation and POS tagging
Canasai Kruengkrai, Kiyotaka Uchimoto, Jun’ichi Kazama, Yiou Wang, Kentaro Torisawa, and Hitoshi Isahara. 2009a
Cited in the paper.

Then

Segmental recurrent neural networks
Original
Lingpeng Kong, Chris Dyer, and Noah A Smith. 2015 · 2015
Later among the works it cites.
Finding function in form: Compositional character models for open vocabulary word representation
Wang Ling, Chris Dyer, W. Alan Black, Isabel Trancoso, Ramon Fermandez, Silvio Amir, Luis Marujo, and Tiago Luis. 2015 · 2015
Later among the works it cites.
TensorFlow: A system for large-scale machine learning
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016 · 2016
Later among the works it cites.
Enriching word vectors with subword information
Original
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016 · 2016
Later among the works it cites.
Character-based LSTM-CRF with radical-level features for Chinese named entity recognition
Chuanhai Dong, Jiajun Zhang, Chengqing Zong, Masanori Hattori, and Hui Di. 2016 · 2016
Later among the works it cites.
A new recurrent neural CRF for learning non-linear edge features
Original
Shuming Ma and Xu Sun. 2016 · 2016
Later among the works it cites.
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF
Xuezhe Ma and Eduard Hovy. 2016 · 2016
Later among the works it cites.
Universal dependencies v1: A multilingual treebank collection
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016 · 2016
Later among the works it cites.
Multi-task multi-domain representation learning for sequence tagging
Original
Nanyun Peng and Mark Dredze. 2016 · 2016
Later among the works it cites.
CHARAGRAM: Embedding words and sentences via character n-grams
Original
John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2016 · 2016
Later among the works it cites.
Transition-based neural word segmentation
Meishan Zhang, Yue Zhang, and Guohong Fu. 2016 · 2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…