Fetching the paper…
Reading the bibliography…
Evaluation of QA systems is very challenging and expensive, with the most reliable approach being human annotations of correctness of answers for questions.
BERTScore: Evaluating text generation with BERT
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020-02-24 · 1904
Earlier work this paper cites.
BLEU: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001 · 2001
Earlier work this paper cites.
ROUGE: A package for automatic evaluation of summaries
Chin-Yew Lin. 2004 · 2004
Earlier work this paper cites.
What is the jeopardy model? a quasi-synchronous grammar for qa
Mengqiu Wang, Noah A Smith, and Teruko Mitamura. 2007 · 2007
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba. 2015 · 2015
Earlier work this paper cites.
From word embeddings to document distances
Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, and Kilian Q. Weinberger. 2015 · 2015
Earlier work this paper cites.
Ms marco: A human generated machine reading comprehension dataset
Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018 · 2018
Earlier work this paper cites.
A Structured Review of the Validity of BLEU
Ehud Reiter. 2018 · 2018
Earlier work this paper cites.
Adaptations of ROUGE and BLEU to better evaluate machine reading comprehension task
An Yang, Kai Liu, Jing Liu, Yajuan Lyu, and Sujian Li. 2018 · 2018
Earlier work this paper cites.
Wikiqa: A challenge dataset for open-domain question answering
Yi Yang, Wen-tau Yih, and Christopher Meek. 2015 · 2018
Earlier work this paper cites.
Evaluating question answering evaluation
Anthony Chen, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019 · 2019
Earlier work this paper cites.
Sentence mover’s similarity: Automatic evaluation for multi-sentence texts
Elizabeth Clark, Asli Celikyilmaz, and Noah A. Smith. 2019 · 2019
Earlier work this paper cites.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019 · 2019
Cited alongside, same era.
Factual error correction for abstractive summarization models
Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung. 2020 · 2020
Cited alongside, same era.
Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection
Siddhant Garg, Thuy Vu, and Alessandro Moschitti. 2020 · 2020
Cited alongside, same era.
Neurips 2020 efficientqa competition: Systems, analyses and lessons learned
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, and Wen-tau Yih. 2021 · 2020
What’s in a name? answer equivalence for open-domain question answering
Chenglei Si, Chen Zhao, and Jordan Boyd-Graber. 2021 · 2021
Later among the works it cites.
AVA: an automatic eValuation approach for question answering systems
Thuy Vu and Alessandro Moschitti. 2021 · 2021
Later among the works it cites.
Does it capture STEL? a modular, similarity-based linguistic style evaluation framework
Anna Wegmann and Dong Nguyen. 2021 · 2021
Later among the works it cites.
Gradient-based adversarial factual consistency evaluation for abstractive summarization
Zhiyuan Zeng, Jiaze Chen, Weiran Xu, and Lei Li. 2021 · 2021
Later among the works it cites.
Joint models for answer verification in question answering systems
Zeyu Zhang, Thuy Vu, and Alessandro Moschitti. 2021 · 2021
Later among the works it cites.
Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
COMET: A neural framework for MT evaluation
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020 · 2020
Cited alongside, same era.
BLEURT: Learning robust metrics for text generation
Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020 · 2020
Cited alongside, same era.
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. 2020 · 2020
Cited alongside, same era.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020 · 2020
Cited alongside, same era.
Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021 · 2021
Cited alongside, same era.
Answer generation for retrieval-based question answering systems
Chao-Chun Hsu, Eric Lind, Luca Soldaini, and Alessandro Moschitti. 2021 · 2021
Cited alongside, same era.
Datasets: A community library for natural language processing
Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Šaško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matussière, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Goehringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf. 2021 · 2021
Cited alongside, same era.
Benjamin Muller, Luca Soldaini, Rik Koncel-Kedziorski, Eric Lind, and Alessandro Moschitti. 2021 · 2021
Cited alongside, same era.
Jannis Bulian, Christian Buck, Wojciech Gajewski, Benjamin Boerschinger, and Tal Schuster. 2022 · 2022
Later among the works it cites.
Pre-training transformer models with sentence-level objectives for answer sentence selection
Luca Di Liello, Siddhant Garg, Luca Soldaini, and Alessandro Moschitti. 2022 · 2022
Later among the works it cites.
Knowledge transfer from answer ranking to answer generation
Matteo Gabburo, Rik Koncel-Kedziorski, Siddhant Garg, Luca Soldaini, and Alessandro Moschitti. 2022 · 2022
Later among the works it cites.
Double retrieval and ranking for accurate question answering
Zeyu Zhang, Thuy Vu, and Alessandro Moschitti. 2022 · 2022
Later among the works it cites.
Falcon-40B: an open large language model with state-of-the-art performance
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023 · 2023
Closest in time.
Context-aware transformer pre-training for answer sentence selection
Luca Di Liello, Siddhant Garg, and Alessandro Moschitti. 2023 · 2023
Closest in time.
Learning answer generation using supervision from automatic question answering evaluators
Matteo Gabburo, Siddhant Garg, Rik Koncel-Kedziorski, and Alessandro Moschitti. 2023 · 2023
Closest in time.