Fetching the paper…
Reading the bibliography…
Recent benchmarks found In-Context Learning (ICL) outperforms both deep learning and tree-based algorithms on small tabular datasets.
The regression analysis of binary sequences
David R Cox · 1958
Earlier work this paper cites.
Nearest neighbor pattern classification
Thomas Cover and Peter Hart · 1967
Earlier work this paper cites.
Induction of decision trees
J. Ross Quinlan · 1986
Earlier work this paper cites.
Support-vector networks
Corinna Cortes and Vladimir Vapnik · 1995
Earlier work this paper cites.
Constrained k-means clustering
Paul S Bradley, Kristin P Bennett, and Ayhan Demiriz · 2000
Earlier work this paper cites.
Classification and regression by randomforest
Andy Liaw, Matthew Wiener, et al · 2002
Earlier work this paper cites.
Minimum redundancy feature selection from microarray gene expression data
Chris Ding and Hanchuan Peng · 2005
Earlier work this paper cites.
Causality
Judea Pearl · 2009
Earlier work this paper cites.
Xgboost: A scalable tree boosting system
Tianqi Chen and Carlos Guestrin · 2016
Earlier work this paper cites.
Deep learning , volume 1
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio · 2016
Earlier work this paper cites.
Lightgbm: A highly efficient gradient boosting decision tree
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu · 2017
Earlier work this paper cites.
Elements of causal inference: foundations and learning algorithms
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf · 2017
Earlier work this paper cites.
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Meta-learning probabilistic inference for prediction
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E Turner · 2018
Earlier work this paper cites.
Catboost: unbiased boosting with categorical features
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin · 2018
Earlier work this paper cites.
Simple, scalable adaptation for neural machine translation
Ankur Bapna, Naveen Arivazhagan, and Orhan Firat · 2019
Earlier work this paper cites.
Parameter-efficient transfer learning for nlp
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly · 2019
Earlier work this paper cites.
Generalization through memorization: Nearest neighbor language models
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis · 2019
Earlier work this paper cites.
Neural oblivious decision ensembles for deep learning on tabular data
Sergei Popov, Stanislav Morozov, and Artem Babenko · 2019
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Rethinking attention with performers
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al · 2020
Earlier work this paper cites.
Retrieval augmented language model pre-training
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang · 2020
Earlier work this paper cites.
The tree ensemble layer: Differentiability meets conditional computation
Hussein Hazimeh, Natalia Ponomareva, Petros Mol, Zhenyu Tan, and Rahul Mazumder · 2020
Earlier work this paper cites.
Tabtransformer: Tabular data modeling using contextual embeddings
Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin · 2020
Cited alongside, same era.
Transformers are rnns: Fast autoregressive transformers with linear attention
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret · 2020
Cited alongside, same era.
Net-dnf: Effective deep modeling of tabular data
Liran Katzir, Gal Elidan, and Ran El-Yaniv · 2020
Cited alongside, same era.
Linformer: Self-attention with linear complexity
Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma · 2020
Cited alongside, same era.
Feature selection using stochastic gates
Yutaro Yamada, Ofir Lindenbaum, Sahand Negahban, and Yuval Kluger · 2020
Cited alongside, same era.
Structured prompting: Scaling in-context learning to 1,000 examples
Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, and Furu Wei · 2022
Later among the works it cites.
Tabpfn: A transformer that solves small tabular classification problems in a second
Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter · 2022
Later among the works it cites.
Gate: Gated additive tree ensemble for tabular classification and regression
Manu Joseph and Harsh Raj · 2022
Later among the works it cites.
Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Vime: Extending the success of self-and semi-supervised learning to tabular domain
Jinsung Yoon, Yao Zhang, James Jordon, and Mihaela van der Schaar · 2020
Cited alongside, same era.
Big bird: Transformers for longer sequences
Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al · 2020
Cited alongside, same era.
Tabnet: Attentive interpretable tabular learning
Sercan Ö Arik and Tomas Pfister · 2021
Cited alongside, same era.
Revisiting deep learning models for tabular data
Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko · 2021
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Cited alongside, same era.
Well-tuned simple nets excel on tabular datasets
Arlind Kadra, Marius Lindauer, Frank Hutter, and Josif Grabocka · 2021
Cited alongside, same era.
Base layers: Simplifying training of large, sparse models
Mike Lewis, Shruti Bhosale, Tim Dettmers, Naman Goyal, and Luke Zettlemoyer · 2021
Cited alongside, same era.
Zhen Qin, Xiaodong Han, Weixuan Sun, Dongxu Li, Lingpeng Kong, Nick Barnes, and Yiran Zhong · 2022
Later among the works it cites.
Hopular: Modern hopfield networks for tabular data
Bernhard Schäfl, Lukas Gruber, Angela Bitto-Nemling, and Sepp Hochreiter · 2022
Later among the works it cites.
Tabular data: Deep learning is not all you need
Ravid Shwartz-Ziv and Amitai Armon · 2022
Later among the works it cites.
Lamda: Language models for dialog applications
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al · 2022
Later among the works it cites.
Introducing semantics into speech encoders
Derek Xu, Shuyan Dong, Changhan Wang, Suyoun Kim, Zhaojiang Lin, Akshat Shrivastava, Shang-Wen Li, Liang-Hsuan Tseng, Alexei Baevski, Guan-Ting Lin, et al · 2022
Later among the works it cites.
Xtab: Cross-table pretraining for tabular transformers
Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, and Mahsa Shoaran · 2022
Later among the works it cites.
Scaling transformer to 1m tokens and beyond with rmt
Aydar Bulatov, Yuri Kuratov, and Mikhail S Burtsev · 2023
Later among the works it cites.
Palm: Scaling language modeling with pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al · 2023
Later among the works it cites.
Scaling tabpfn: Sketching and feature selection for tabular prior-data fitted networks
Benjamin Feuer, Chinmay Hegde, and Niv Cohen · 2023
Later among the works it cites.
Pre-training to learn in context
Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang · 2023
Later among the works it cites.
When do neural nets outperform boosted trees on tabular data?
Duncan McElfresh, Sujay Khandagale, Jonathan Valverde, Ganesh Ramakrishnan, Micah Goldblum, Colin White, et al · 2023
Later among the works it cites.
Transformers learn in-context by gradient descent
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov · 2023
Later among the works it cites.
k k nn prompting: Beyond-context learning with calibration-free nearest neighbor inference
Benfeng Xu, Quan Wang, Zhendong Mao, Yajuan Lyu, Qiaoqiao She, and Yongdong Zhang · 2023
Later among the works it cites.
Unitabe: Pretraining a unified tabular encoder for heterogeneous tabular data
Yazheng Yang, Yuqi Wang, Guang Liu, Ledell Wu, and Qi Liu · 2023
Later among the works it cites.
https://github.com/naszilla/tabzilla/issues/96
abvesa · 2024
Closest in time.
Mixture-of-experts in prompt optimization, 2024
Anonymous · 2024
Closest in time.
The faiss library
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou · 2024
Closest in time.
https://github.com/naszilla/tabzilla
naszilla · 2024
Closest in time.
In-context learning with retrieved demonstrations for language models: A survey
Xin Xu, Yue Liu, Panupong Pasupat, Mehran Kazemi, et al · 2024
Closest in time.