Fetching the paper…

Reading the bibliography…

BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Tasks · Around

2020

BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Tasks

Huang, Tongwen, She, Qingyun, Zhang, Junlin

Original
PDF
Post on X

Understand

As a pre-trained Transformer model, BERT (Bidirectional Encoder Representations from Transformers) has achieved ground-breaking performance on multiple NLP tasks.

On the other hand, Boosting is a popular ensemble learning technique which combines many base classifiers and has been demonstrated to yield better generalization performance in many machine learning tasks.
Some works have indicated that ensemble of BERT can further improve the application performance.
However, current ensemble approaches focus on bagging or stacking and there has not been much effort on exploring the boosting.

Built on

Passage re-ranking with BERT
Original
Rodrigo Nogueira and Kyunghyun Cho. 2019 · 1901
Earlier work this paper cites.
Dual co-matching network for multi-choice reading comprehension
Original
Shuailiang Zhang, Hai Zhao, Yuwei Wu, Zhuosheng Zhang, Xi Zhou, and Xiang Zhou. 2019 · 1901
Earlier work this paper cites.
But-fit at semeval-2019 task 7: Determining the rumour stance with pre-trained deep bidirectional transformers
Original
Martin Fajcik, Lukáš Burget, and Pavel Smrz. 2019 · 1902
Earlier work this paper cites.
Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence
Original
Chi Sun, Luyao Huang, and Xipeng Qiu. 2019 · 1903
Earlier work this paper cites.
Contextualized word embeddings enhanced event temporal relation extraction for story understanding
Original
Rujun Han, Mengyue Liang, Bashar Alhafni, and Nanyun Peng. 2019 · 1904
Earlier work this paper cites.
ANA at semeval-2019 task 3: Contextual emotion detection in conversations through hierarchical lstms and BERT
Original
Chenyang Huang, Amine Trabelsi, and Osmar R. Zaïane. 2019 · 1904
Earlier work this paper cites.
CEDR: contextualized embeddings for document ranking
Original
Sean MacAvaney, Andrew Yates, Arman Cohan, and Nazli Goharian. 2019 · 1904
Earlier work this paper cites.
Bam! born-again multi-task networks for natural language understanding
Original
Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal, Christopher D Manning, and Quoc V Le. 2019 · 1907
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Original
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b · 1907
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Original
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019 · 1910
Earlier work this paper cites.
Bias, variance, and arcing classifiers
Leo Breiman. 1996 · 1996
Earlier work this paper cites.

Similar

A decision-theoretic generalization of on-line learning and an application to boosting
Yoav Freund and Robert E Schapire. 1997 · 1997
Cited alongside, same era.
Automatically constructing a corpus of sentential paraphrases
William B Dolan and Chris Brockett. 2005 · 2005
Cited alongside, same era.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009 · 2009
Cited alongside, same era.
Multi-class adaboost
Original
Ji Zhu, Hui Zou, Saharon Rosset, and Trevor Hastie. 2009 · 2009
Cited alongside, same era.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio. 2010 · 2010
Cited alongside, same era.
Sentiment classification analysis based on extraction of sentiment key sentence

Then

Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Later among the works it cites.
A broad-coverage challenge corpus for sentence understanding through inference
Original
Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017 · 2017
Later among the works it cites.
The first evaluation of chinese human-computer dialogue technology
Original
Wei-Nan Zhang, Zhigang Chen, Wanxiang Che, Guoping Hu, and Ting Liu. 2017 · 2017
Later among the works it cites.
Xnli: Evaluating cross-lingual sentence representations
Original
Alexis Conneau, Guillaume Lample, Ruty Rinott, Adina Williams, Samuel R Bowman, Holger Schwenk, and Veselin Stoyanov. 2018 · 2018
Later among the works it cites.
Bert: Pre-training of deep bidirectional transformers for language understanding

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Z Lin, S Tan, and X Cheng. 2012 · 2012

Cited alongside, same era.

Distributed representations of words and phrases and their compositionality

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013 · 2013

Cited alongside, same era.

Recursive deep models for semantic compositionality over a sentiment treebank

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013 · 2013

Cited alongside, same era.

Glove: Global vectors for word representation

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014 · 2014

Cited alongside, same era.

What makes imagenet good for transfer learning?

Minyoung Huh, Pulkit Agrawal, and Alexei A Efros. 2016 · 2016

Cited alongside, same era.

Google’s neural machine translation system: Bridging the gap between human and machine translation

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016 · 2016

Cited alongside, same era.

Trust or suspect? an empirical ensemble framework for fake news classification

Shuaipeng Liu, Shuo Liu, and Lei Ren. 2019a

Cited in the paper.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018

Later among the works it cites.

Universal language model fine-tuning for text classification

Jeremy Howard and Sebastian Ruder. 2018 · 2018

Later among the works it cites.

Deep contextualized word representations

Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 · 2018

Later among the works it cites.

Improving language understanding by generative pre-training

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018 · 2018

Later among the works it cites.

Neural network acceptability judgments

Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2018 · 2018

Later among the works it cites.

What does bert learn about the structure of language?

Ganesh Jawahar, Benoît Sagot, Djamé Seddah, Samuel Unicomb, Gerardo Iñiguez, Márton Karsai, Yannick Léo, Márton Karsai, Carlos Sarraute, Éric Fleury, et al. 2019 · 2019

Later among the works it cites.