2018

AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification

You, Ronghui, Zhang, Zihan, Wang, Ziye et al.

Understand

Extreme multi-label text classification (XMTC) is an important problem in the era of big data, for tagging a given text with the most relevant multiple labels from an extremely large-scale label set.

XMTC can be found in many applications, such as item categorization, web page tagging, and news annotation.
Traditionally most methods used bag-of-words (BOW) as inputs, ignoring word context as well as deep semantic information.
Recent attempts to overcome the problems of BOW by deep learning still suffer from 1) failing to capture the important subtext for each label and 2) lack of scalability against the huge number of labels.

Built on

Learning long-term dependencies with gradient descent is difficult
Y. Bengio, P. Simard, and P. Frasconi · 1994
Earlier work this paper cites.
Long short-term memory
S. Hochreiter and J. Schmidhuber · 1997
Earlier work this paper cites.
Efficient pairwise multilabel classification for large-scale problems in the legal domain
E. L. Mencia and J. Fürnkranz · 2008
Earlier work this paper cites.
Enhancing navigation on wikipedia with social tags
Original
A. Zubiaga · 2012
Earlier work this paper cites.
On flat versus hierarchical classification in large-scale taxonomies
R. Babbar, I. Partalas, E. Gaussier, and M. R. Amini · 2013
Earlier work this paper cites.
Hidden factors and hidden topics: understanding rating dimensions with review text
J. McAuley and J. Leskovec · 2013
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Original
D. Bahdanau, K. Cho, and Y. Bengio · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
D. P. Kingma and J. Ba · 2014
Earlier work this paper cites.
Glove: Global vectors for word representation
J. Pennington, R. Socher, and C. Manning · 2014
Earlier work this paper cites.
FastXML: A fast, accurate and stable tree-classifier for extreme multi-label learning
Y. Prabhu and M. Varma · 2014
Earlier work this paper cites.
Dropout: a simple way to prevent neural networks from overfitting
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov · 2014
Earlier work this paper cites.

Similar

Sparse local embeddings for extreme multi-label classification
K. Bhatia, H. Jain, P. Kar, M. Varma, and P. Jain · 2015
Cited alongside, same era.
Effective approaches to attention-based neural machine translation
T. Luong, H. Pham, and C. D. Manning · 2015
Cited alongside, same era.
Learning taxonomy adaptation in large-scale classification
R. Babbar, I. Partalas, E. Gaussier, M.-R. Amini, and C. Amblard · 2016
Cited alongside, same era.
Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications
H. Jain, Y. Prabhu, and M. Varma · 2016
Cited alongside, same era.
Extreme f-measure maximization using sparse probability estimates
K. Jasinska, K. Dembczynski, R. Busa-Fekete, K. Pfannschmidt, T. Klerx, and E. Hullermeier · 2016
Cited alongside, same era.
PD-Sparse: a primal and dual sparse approach to extreme multiclass and multilabel classification
I. E.-H. Yen, X. Huang, P. Ravikumar, K. Zhong, and I. Dhillon · 2016
Cited alongside, same era.
DiSMEC: distributed sparse machines for extreme multi-label classification
R. Babbar and B. Schölkopf · 2017
Cited alongside, same era.
Bag of tricks for efficient text classification
A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov · 2017
Cited alongside, same era.
A structured self-attentive sentence embedding
Original
Z. Lin, M. Feng, C. N. d. Santos, M. Yu, B. Xiang, B. Zhou, and Y. Bengio · 2017
Cited alongside, same era.
Deep learning for extreme multi-label text classification
J. Liu, W.-C. Chang, Y. Wu, and Y. Yang · 2017
Cited alongside, same era.
Maximizing subset accuracy with recurrent neural networks in multi-label classification
J. Nam, E. L. Mencía, H. J. Kim, and J. Fürnkranz · 2017
Cited alongside, same era.

Then

AnnexML: approximate nearest neighbor search for extreme multi-label classification
Y. Tagami · 2017
Later among the works it cites.
PPDsparse: a parallel primal-dual sparse method for extreme classification
I. E. Yen, X. Huang, W. Dai, P. Ravikumar, I. Dhillon, and E. Xing · 2017
Later among the works it cites.
Averaging weights leads to wider optima and better generalization
Original
P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson · 2018
Closest in time.
Semantic-unit-based dilated convolution for multi-label text classification
J. Lin, Q. Su, P. Yang, S. Ma, and X. Sun · 2018
Closest in time.
Extreme multi-label learning with label features for warm-start tagging, ranking & recommendation
Y. Prabhu, A. Kag, S. Gopinath, K. Dahiya, S. Harsola, R. Agrawal, and M. Varma · 2018
Closest in time.
Parabel: Partitioned label trees for extreme classification with application to dynamic search advertising
Y. Prabhu, A. Kag, S. Harsola, R. Agrawal, and M. Varma · 2018
Closest in time.
A no-regret generalization of hierarchical softmax to extreme multi-label classification
M. Wydmuch, K. Jasinska, M. Kuznetsov, R. Busa-Fekete, and K. Dembczynski · 2018
Closest in time.
SGM: sequence generation model for multi-label classification
P. Yang, X. Sun, W. Li, S. Ma, W. Wu, and H. Wang · 2018
Closest in time.
Data scarcity, robustness and extreme multi-label classification
R. Babbar and B. Schölkopf · 2019
Closest in time.
Bonsai-diverse and shallow trees for extreme multi-label classification
Original
S. Khandagale, H. Xiao, and R. Babbar · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…