Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution

Smith, J.W., Everhart, J.E., Dickson, W., Knowler, W.C., Johannes, R.S.: Using the adap learning algorithm to forecast the onset of diabetes mellitus. In: Proceedings of the Annual Symposium on Computer Application in Medical Care, p. 261 (1988). American Medical Informatics Association

1988

Earlier work this paper cites.

Marzal, A., Vidal, E.: Computation of normalized edit distance and applications. IEEE Trans. Pattern Anal. Mach. Intell. 15

1993

Earlier work this paper cites.

Hofmann, H.: Statlog (German Credit Data). UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5NC77 (1994)

1994

Earlier work this paper cites.

Becker, B., Kohavi, R.: Adult. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5XW20 (1996)

1996

Earlier work this paper cites.

Pace, R.K., Barry, R.: Sparse spatial autoregressions. Statistics & Probability Letters 33

1997

Earlier work this paper cites.

Bohanec, M.: Car Evaluation. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5JP48 (1997)

1997

Earlier work this paper cites.

Blackard, J.: Covertype. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C50K5N (1998)

1998

Earlier work this paper cites.

Papineni, K., Roukos, S., Ward, T., Zhu, W.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pp. 311–318. ACL, ??? (2002). https://doi.org/10.3115/1073083.1073135 . https://aclanthology.org/P02-1040/

2002

Earlier work this paper cites.

Webber, W., Moffat, A., Zobel, J.: A similarity measure for indefinite rankings. ACM Trans. Inf. Syst. 28

2010

Earlier work this paper cites.

Deng, L.: The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine 29

2012

Earlier work this paper cites.

Moro, S., Rita, P., , Cortez, P.: Bank Marketing. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5K306 (2012)

2012

Earlier work this paper cites.

Mansouri, K., Ringsted, T., Ballabio, D., Todeschini, R., Consonni, V.: QSAR biodegradation. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5H60M (2013)

2013

Earlier work this paper cites.

Buza, K.: BlogFeedback. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C58S3F (2014)

2014

Earlier work this paper cites.

Whiteson, D.: HIGGS. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5V312 (2014)

2014

Earlier work this paper cites.

Harper, F.M., Konstan, J.A.: The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5

2015

Earlier work this paper cites.

Pasupat, P., Liang, P.: Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305 (2015)

Original

2015

Earlier work this paper cites.

Lehmberg, O., Ritze, D., Meusel, R., Bizer, C.: A large public corpus of web tables containing time and context metadata. In: Proceedings of the 25th International Conference Companion on World Wide Web, pp. 75–76 (2016)

2016

Earlier work this paper cites.

Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: Deepfm: a factorization-machine based neural network for ctr prediction. arXiv preprint arXiv:1703.04247 (2017)

Original

2017

Earlier work this paper cites.

Zhong, V., Xiong, C., Socher, R.: Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR abs/1709.00103

Original

2017

Earlier work this paper cites.

Zhong, V., Xiong, C., Socher, R.: Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103 (2017)

Original

2017

Earlier work this paper cites.

Iyyer, M., Yih, W.-t., Chang, M.-W.: Search-based neural structured learning for sequential question answering. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1821–1831 (2017)

2017

Earlier work this paper cites.

Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

Original

2018

Earlier work this paper cites.

Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146 (2018)

Original

2018

Earlier work this paper cites.

Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al.: Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887 (2018)

Original

2018

Earlier work this paper cites.

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al

2019

Earlier work this paper cites.

Zhang, L., Zhang, S., Balog, K.: Table2vec: Neural word and entity embeddings for table population and retrieval. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1029–1032 (2019)

2019

Earlier work this paper cites.

Chen, W., Wang, H., Chen, J., Zhang, Y., Wang, H., Li, S., Zhou, X., Wang, W.Y.: Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv:1909.02164 (2019)

Original

2019

Earlier work this paper cites.

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

Original

2019

Earlier work this paper cites.

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., Zettlemoyer, L.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)

Original

2019

Earlier work this paper cites.

Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (2019). Association for Computational Linguistics

2019

Earlier work this paper cites.

Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., Gelly, S.: Parameter-efficient transfer learning for nlp. In: International Conference on Machine Learning, pp. 2790–2799 (2019). PMLR

2019

Earlier work this paper cites.

Ansuini, A., Laio, A., Macke, J.H., Zoccolan, D.: Intrinsic dimension of data representations in deep neural networks. Advances in Neural Information Processing Systems 32

2019

Earlier work this paper cites.

Clements, J.M., Xu, D., Yousefi, N., Efimov, D.: Sequential deep learning for credit risk monitoring with tabular financial data. arXiv preprint arXiv:2012.15330 (2020)

Original

2020

Earlier work this paper cites.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

2020

Earlier work this paper cites.

Yin, P., Neubig, G., Yih, W.-t., Riedel, S.: TaBERT: Pretraining for joint understanding of textual and tabular data. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8413–8426. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.745 . https://aclanthology.org/2020.acl-main.745

2020

Earlier work this paper cites.

Yoon, J., Zhang, Y., Jordon, J., Schaar, M.: Vime: Extending the success of self-and semi-supervised learning to tabular domain. Advances in Neural Information Processing Systems 33

2020

Earlier work this paper cites.

Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z.: Tabtransformer: Tabular data modeling using contextual embeddings. arXiv preprint arXiv:2012.06678 (2020)

Original

2020

Earlier work this paper cites.

Chen, W., Zha, H., Chen, Z., Xiong, W., Wang, H., Wang, W.: Hybridqa: A dataset of multi-hop question answering over tabular and textual data. arXiv preprint arXiv:2004.07347 (2020)

Original

2020

Earlier work this paper cites.

Parikh, A.P., Wang, X., Gehrmann, S., Faruqui, M., Dhingra, B., Yang, D., Das, D.: Totto: A controlled table-to-text generation dataset. arXiv preprint arXiv:2004.14373 (2020)

Original

2020

Earlier work this paper cites.

Herzig, J., Nowak, P.K., Mueller, T., Piccinno, F., Eisenschlos, J.: Tapas: Weakly supervised table parsing via pre-training. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4320–4333 (2020)

2020

Earlier work this paper cites.

Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L.A., Mark, R.: Mimic-iv. PhysioNet. Available online at: https://physionet. org/content/mimiciv/1.0/(accessed August 23, 2021), 49–55 (2020)

2020

Earlier work this paper cites.

Somani, S., Russak, A.J., Richter, F., Zhao, S., Vaid, A., Chaudhry, F., De Freitas, J.K., Naik, N., Miotto, R., Nadkarni, G.N., et al

2021

Earlier work this paper cites.

Gianfrancesco, M.A., Goldstein, N.D.: A narrative review on the validity of electronic health record-based research in epidemiology. BMC medical research methodology 21

2021

Earlier work this paper cites.

Qin, J., Zhang, W., Su, R., Liu, Z., Liu, W., Tang, R., He, X., Yu, Y.: Retrieval & interaction machine for tabular data prediction. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1379–1389 (2021)

2021

Earlier work this paper cites.

Liu, Q., Chen, B., Guo, J., Ziyadi, M., Lin, Z., Chen, W., Lou, J.-G.: Tapex: Table pre-training via learning a neural sql executor. In: International Conference on Learning Representations (2021)

2021

Cited alongside, same era.

Iida, H., Thai, D., Manjunatha, V., Iyyer, M.: Tabbie: Pretrained representations of tabular data. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3446–3456 (2021)

2021

Cited alongside, same era.

Arik, S.Ö., Pfister, T.: Tabnet: Attentive interpretable tabular learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 6679–6687 (2021)

2021

Cited alongside, same era.

Darabi, S., Fazeli, S., Pazoki, A., Sankararaman, S., Sarrafzadeh, M.: Contrastive mixup: Self-and semi-supervised learning for tabular domain. arXiv preprint arXiv:2108.12296 (2021)

Original

2021

Cited alongside, same era.

Zhang, T., Xu, H., Genabith, J., Xiong, D., Zan, H.: Napg: Non-autoregressive program generation for hybrid tabular-textual question answering. In: CCF International Conference on Natural Language Processing and Chinese Computing, pp. 591–603 (2023). Springer

2023

Later among the works it cites.

Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W.X., Wen, J.-R.: Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645 (2023)

Original

2023

Later among the works it cites.

Yang, Y., Wang, Y., Liu, G., Wu, L., Liu, Q.: Unitabe: Pretraining a unified tabular encoder for heterogeneous tabular data. arXiv preprint arXiv:2307.09249 (2023)

Original

2023

Later among the works it cites.

Sui, Y., Zhou, M., Zhou, M., Han, S., Zhang, D.: GPT4Table: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study (2023)

2023

Later among the works it cites.