Fetching the paper…
Reading the bibliography…
Recent advancements in NLP have witnessed the groundbreaking impact of pretrained models, yielding impressive outcomes across various tasks.
Long short-term memory
S. Hochreiter and J. Schmidhuber · 1997
Earlier work this paper cites.
Adam: A method for stochastic optimization
D. P. Kingma and J. Ba · 2015
Earlier work this paper cites.
Xgboost: A scalable tree boosting system
T. Chen and C. Guestrin · 2016
Earlier work this paper cites.
Tabert: Pretraining for joint understanding of textual and tabular data
P. Yin, G. Neubig, W.-t. Yih, and S. Riedel · 2016
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova · 2018
Earlier work this paper cites.
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer · 2019
Earlier work this paper cites.
Neural oblivious decision ensembles for deep learning on tabular data
S. Popov, S. Morozov, and A. Babenko · 2019
Earlier work this paper cites.
Autoint: Automatic feature interaction learning via self-attentive neural networks
W. Song, C. Shi, Z. Xiao, Z. Duan, Y. Xu, M. Zhang, and J. Tang · 2019
Earlier work this paper cites.
Recall and learn: Fine-tuning deep pretrained language models with less forgetting
S. Chen, Y. Hou, Y. Cui, W. Che, T. Liu, and X. Yu · 2020
Earlier work this paper cites.
Tapas: Weakly supervised table parsing via pre-training
J. Herzig, P. K. Nowak, T. Müller, F. Piccinno, and J. M. Eisenschlos · 2020
Cited alongside, same era.
Tabtransformer: Tabular data modeling using contextual embeddings
X. Huang, A. Khetan, M. Cvitkovic, and Z. Karnin · 2020
Cited alongside, same era.
Deep encoder, shallow decoder: Reevaluating non-autoregressive machine translation
J. Kasai, N. Pappas, H. Peng, J. Cross, and N. A. Smith · 2020
Cited alongside, same era.
Supervised contrastive learning
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan · 2020
Cited alongside, same era.
What makes for good views for contrastive learning?
Y. Tian, C. Sun, B. Poole, D. Krishnan, C. Schmid, and P. Isola · 2020
Cited alongside, same era.
Instantaneous grammatical error correction with shallow aggressive decoding
X. Sun, T. Ge, F. Wei, and H. Wang · 2021
Later among the works it cites.
Tuta: tree-based transformers for generally structured table pre-training
Z. Wang, H. Dong, R. Jia, J. Li, Z. Fu, S. Han, and D. Zhang · 2021
Later among the works it cites.
Efficient training of language models to fill in the middle
M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen · 2022
Later among the works it cites.
On embeddings for numerical features in tabular deep learning
Y. Gorishniy, I. Rubachev, and A. Babenko · 2022
Later among the works it cites.
Tabpfn: A transformer that solves small tabular classification problems in a second
N. Hollmann, S. Müller, K. Eggensperger, and F. Hutter · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tabnet: Attentive interpretable tabular learning
S. Ö. Arik and T. Pfister · 2021
Cited alongside, same era.
Mate: Multi-view attention for table transformer efficiency
J. M. Eisenschlos, M. Gor, T. Müller, and W. W. Cohen · 2021
Cited alongside, same era.
Revisiting deep learning models for tabular data
Y. Gorishniy, I. Rubachev, V. Khrulkov, and A. Babenko · 2021
Cited alongside, same era.
An efficient transformer decoder with compressed sub-layers
Y. Li, Y. Lin, T. Xiao, and J. Zhu · 2021
Cited alongside, same era.
Saint: Improved neural networks for tabular data via row attention and contrastive pre-training
G. Somepalli, M. Goldblum, A. Schwarzschild, C. B. Bruss, and T. Goldstein · 2021
Cited alongside, same era.
TaBERT: Pretraining for joint understanding of textual and tabular data
P. Yin, G. Neubig, W. tau Yih, and S. Riedel
Cited in the paper.
Later among the works it cites.
Multilingual neural machine translation with deep encoder and multiple shallow decoders
X. Kong, A. Renduchintala, J. Cross, Y. Tang, J. Gu, and X. Li · 2022
Later among the works it cites.
Ptab: Using the pre-trained language model for modeling tabular data
G. Liu, J. Yang, and L. Wu · 2022
Later among the works it cites.
Transtab: Learning transferable tabular transformers across tables
Z. Wang and J. Sun · 2022
Later among the works it cites.
Gandalf: Gated adaptive network for deep automated learning of features, 2023
M. Joseph and H. Raj · 2023
Closest in time.
Llama 2: Open foundation and fine-tuned chat models
H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al · 2023
Closest in time.