2020

FarsTail: A Persian Natural Language Inference Dataset

Amirkhani, Hossein, AzariJafari, Mohammad, Pourjafari, Zohreh et al.

Understand

Natural language inference (NLI) is known as one of the central tasks in natural language processing (NLP) which encapsulates many fundamental aspects of language understanding.

With the considerable achievements of data-hungry deep learning methods in NLP tasks, a great amount of effort has been devoted to develop more diverse datasets for different languages.
In this paper, we present a new dataset for the NLI task in the Persian language, also known as Farsi, which is one of the dominant languages in the Middle East.
This dataset, named FarsTail, includes 10,367 samples which are provided in both the Persian language as well as the indexed format to be useful for non-Persian researchers.

Built on

arXiv:2001.01582
Original
R. Baradaran, R. Ghiasi, H. Amirkhani, A survey on machine reading comprehension systems (2020) · 2001
Earlier work this paper cites.
arXiv:2002.07526
Original
S. Yang, Y. Wang, X. Chu, A survey of deep learning techniques for neural machine translation (2020) · 2002
Earlier work this paper cites.
B. MacCartney, Natural language inference, Ph.D. thesis, Stanford University (2009)
2009
Earlier work this paper cites.
J. Bos, F. M. Zanzotto, M. Pennacchiotti, Textual entailment at evalita 2009, Proceedings of EVALITA 2009 2 (6.4) (2009) 1–7
2009
Earlier work this paper cites.
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, in: Advances in neural information processing systems, 2013, pp. 3111–3119
2013
Earlier work this paper cites.
M. Alabbas, A dataset for Arabic textual entailment, in: Proceedings of the Student Research Workshop associated with RANLP 2013, 2013, pp. 7–13
2013
Earlier work this paper cites.
M. Marelli, L. Bentivogli, M. Baroni, R. Bernardi, S. Menini, R. Zamparelli, SemEval-2014 Task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment, in: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland, 2014, pp. 1–8
2014
Earlier work this paper cites.
K. Eichler, A. Gabryszak, G. Neumann, An analysis of textual inference in German customer emails, in: Proceedings of the Third Joint Conference on Lexical and Computational Semantics (*SEM 2014), 2014, pp. 69–74
2014
Earlier work this paper cites.
S. R. Bowman, G. Angeli, C. Potts, C. D. Manning, A large annotated corpus for learning natural language inference, in: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 632–642
2015
Earlier work this paper cites.
A. P. Parikh, O. Täckström, D. Das, J. Uszkoreit, A decomposable attention model for natural language inference, in: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016, pp. 2249–2255
2016
Earlier work this paper cites.
A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, R. G. Mark, MIMIC-III, a freely accessible critical care database, Scientific data 3 (1) (2016) 1–9
2016
Earlier work this paper cites.
E. R. Fonseca, L. Borges dos Santos, M. Criscuolo, S. M. Aluísio, Overview of the evaluation of semantic similarity and textual inference, Linguamática 8 (2) (2016) 3–13
2016
Earlier work this paper cites.
P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching word vectors with subword information, Transactions of the Association for Computational Linguistics 5 (2017) 135–146
2017
Earlier work this paper cites.

Similar

Q. Chen, X. Zhu, Z. Ling, S. Wei, H. Jiang, D. Inkpen, Enhanced LSTM for natural language inference, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1657–1668
2017
Cited alongside, same era.
V. Yadav, S. Bethard, A survey on recent advances in named entity recognition from deep learning models, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 2145–2158
2018
Cited alongside, same era.
T. Young, D. Hazarika, S. Poria, E. Cambria, Recent trends in deep learning based natural language processing, IEEE Computational Intelligence Magazine 13 (3) (2018) 55–75
2018
Cited alongside, same era.
A. Conneau, G. Lample, R. Rinott, A. Williams, S. R. Bowman, H. Schwenk, V. Stoyanov, XNLI: Evaluating cross-lingual sentence representations, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 2475–2485
2018
Cited alongside, same era.
T. Khot, A. Sabharwal, P. Clark, SciTail: A textual entailment dataset from science question answering, in: Thirty-Second AAAI Conference on Artificial Intelligence, 2018, pp. 5189–5197
2018
Cited alongside, same era.
M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Deep contextualized word representations, in: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2018, pp. 2227–2237
2018
Cited alongside, same era.
J. Howard, S. Ruder, Universal language model fine-tuning for text classification, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 328–339
2018
Cited alongside, same era.
A. Williams, N. Nangia, S. R. Bowman, A broad-coverage challenge corpus for sentence understanding through inference, in: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2018, pp. 1112–1122
2018
Cited alongside, same era.
A. Romanov, C. Shivade, Lessons from natural language inference in the clinical domain, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 1586–1596
2018
Cited alongside, same era.
arXiv:1809.02922
Original
D. Demszky, K. Guu, P. Liang, Transforming question answering datasets into natural language inference datasets (2018) · 2018
Cited alongside, same era.
P. Rajpurkar, R. Jia, P. Liang, Know what you don’t know: Unanswerable questions for SQuAD, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2018, pp. 784–789
2018
Cited alongside, same era.
S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. R. Bowman, N. A. Smith, Annotation artifacts in natural language inference data, in: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), 2018, pp. 107–112
2018
Cited alongside, same era.

Then

A. Poliak, J. Naradowsky, A. Haldar, R. Rudinger, B. Van Durme, Hypothesis only baselines in natural language inference, in: Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, 2018, pp. 180–191
2018
Later among the works it cites.
A. Keramatfar, H. Amirkhani, Bibliometrics of sentiment analysis literature, Journal of Information Science 45 (1) (2019) 3–15
2019
Later among the works it cites.
J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019, pp. 4171–4186
2019
Later among the works it cites.
M. Artetxe, H. Schwenk, Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond, Transactions of the Association for Computational Linguistics 7 (2019) 597–610
2019
Later among the works it cites.
A. Talman, A. Yli-Jyrä, J. Tiedemann, Sentence embeddings in NLI with iterative refinement encoders, Natural Language Engineering 25 (4) (2019) 467–482
2019
Later among the works it cites.
T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al., HuggingFace’s Transformers: State-of-the-art natural language processing, arXiv preprint arXiv:1910.03771 (2019)
Original
2019
Later among the works it cites.
D. W. Otter, J. R. Medina, J. K. Kalita, A survey of the usages of deep learning for natural language processing, IEEE Transactions on Neural Networks and Learning Systems (2020) 1–21
2020
Closest in time.
D. Khashabi, A. Cohan, S. Shakeri, P. Hosseini, P. Pezeshkpour, M. Alikhani, M. Aminnaseri, M. Bitaab, F. Brahman, S. Ghazarian, et al., ParsiNLU: A suite of language understanding challenges for persian, arXiv preprint arXiv:2012.06154 (2020)
Original
2020
Closest in time.
H. Hu, K. Richardson, L. Xu, L. Li, S. Kübler, L. S. Moss, OCNLI: Original chinese natural language inference, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, 2020, pp. 3512–3526
2020
Closest in time.
M. Farahani, M. Gharachorloo, M. Farahani, M. Manthouri, ParsBERT: Transformer-based model for persian language understanding, arXiv preprint arXiv:2005.12515 (2020)
Original
2020
Closest in time.
R. T. McCoy, E. Pavlick, T. Linzen, Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference, in: 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019, Association for Computational Linguistics (ACL), 2020, pp. 3428–3448
2020
Closest in time.
S. R. Bowman, J. Palomaki, L. B. Soares, E. Pitler, New protocols and negative results for textual entailment data collection., in: EMNLP (1), 2020, pp. 8203–8214
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…