Fetching the paper…
Reading the bibliography…
The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J · 2015
Earlier work this paper cites.
Ba, J. L., Kiros, J. R., and Hinton, G. E · 2016
Earlier work this paper cites.
Xgboost: A scalable tree boosting system
Chen, T. and Guestrin, C · 2016
Earlier work this paper cites.
Lightgbm: A highly efficient gradient boosting decision tree
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y · 2017
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Earlier work this paper cites.
Catboost: gradient boosting with categorical features support
Dorogush, A. V., Ershov, V., and Gulin, A · 2018
Earlier work this paper cites.
On the convergence of fedavg on non-iid data
Li, X., Huang, K., Yang, W., Wang, S., and Zhang, Z · 2019
Earlier work this paper cites.
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G · 2020
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al · 2020
Earlier work this paper cites.
Autogluon-tabular: Robust and accurate automl for structured data
Erickson, N., Mueller, J., Shirkov, A., Zhang, H., Larroy, P., Li, M., and Smola, A · 2020
Earlier work this paper cites.
Fastai: a layered api for deep learning
Howard, J. and Gugger, S · 2020
Earlier work this paper cites.
Glu variants improve transformer
Shazeer, N · 2020
Cited alongside, same era.
Practical and sample efficient zero-shot hpo
Winkelmolen, F., Ivkin, N., Bozkurt, H. F., and Karnin, Z · 2020
Cited alongside, same era.
TaBERT: Pretraining for joint understanding of textual and tabular data
Yin, P., Neubig, G., Yih, W.-t., and Riedel, S · 2020
Cited alongside, same era.
Vime: Extending the success of self-and semi-supervised learning to tabular domain
Yoon, J., Zhang, Y., Jordon, J., and van der Schaar, M · 2020
Cited alongside, same era.
Muppet: Massive multi-task representations with pre-finetuning
Aghajanyan, A., Gupta, A., Shrivastava, A., Chen, X., Zettlemoyer, L., and Gupta, S · 2021
Fastformer: Additive attention can be all you need
Wu, C., Wu, F., Qi, T., Huang, Y., and Xie, X · 2021
Later among the works it cites.
Fedavg with fine tuning: Local updates lead to representation learning
Collins, L., Hassani, H., Mokhtari, A., and Shakkottai, S · 2022
Later among the works it cites.
Gijsbers, P., Bueno, M. L., Coors, S., LeDell, E., Poirier, S., Thomas, J., Bischl, B., and Vanschoren, J · 2022
Later among the works it cites.
Why do tree-based models still outperform deep learning on typical tabular data?
Grinsztajn, L., Oyallon, E., and Varoquaux, G · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Scarf: Self-supervised contrastive learning using random feature corruption
Bahri, D., Jiang, H., Tay, Y., and Metzler, D · 2021
Cited alongside, same era.
On the opportunities and risks of foundation models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al · 2021
Cited alongside, same era.
Revisiting deep learning models for tabular data
Gorishniy, Y., Rubachev, I., Khrulkov, V., and Babenko, A · 2021
Cited alongside, same era.
Kaushik, P., Gain, A., Kortylewski, A., and Yuille, A · 2021
Cited alongside, same era.
Swin transformer: Hierarchical vision transformer using shifted windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B · 2021
Cited alongside, same era.
Effect of scale on catastrophic forgetting in neural networks
Ramasesh, V. V., Lewkowycz, A., and Dyer, E · 2021
Cited alongside, same era.
Saint: Improved neural networks for tabular data via row attention and contrastive pre-training
Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C. B., and Goldstein, T · 2021
Cited alongside, same era.
Tabpfn: A transformer that solves small tabular classification problems in a second
Hollmann, N., Müller, S., Eggensperger, K., and Hutter, F · 2022
Later among the works it cites.
In defense of the unitary scalarization for deep multi-task learning
Kurin, V., De Palma, A., Kostrikov, I., Whiteson, S., and Kumar, M. P · 2022
Later among the works it cites.
Transfer learning with deep tabular models
Levin, R., Cherepanova, V., Schwarzschild, A., Bansal, A., Bruss, C. B., Goldstein, T., Wilson, A. G., and Goldblum, M · 2022
Later among the works it cites.
Met: Masked encoding for tabular data
Majmundar, K., Goyal, S., Netrapalli, P., and Jain, P · 2022
Later among the works it cites.
Revisiting pretraining objectives for tabular deep learning
Rubachev, I., Alekberov, A., Gorishniy, Y., and Babenko, A · 2022
Later among the works it cites.
Transtab: Learning transferable tabular transformers across tables
Wang, Z. and Sun, J · 2022
Later among the works it cites.
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., et al · 2022
Later among the works it cites.