2020

Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset: Preliminary Thoughts and Lessons Learned

Zhang, Edwin, Gupta, Nikhil, Nogueira, Rodrigo et al.

Understand

We present the Neural Covidex, a search engine that exploits the latest neural ranking architectures to provide information access to the COVID-19 Open Research Dataset curated by the Allen Institute for AI.

This web application exists as part of a suite of tools that we have developed over the past few weeks to help domain experts tackle the ongoing global pandemic.
We hope that improved information access capabilities to the scientific literature can inform evidence-based decision making and insight generation.
This paper describes our initial efforts and offers a few thoughts about lessons we have learned along the way.

Built on

Exploring the limits of transfer learning with a unified text-to-text transformer
Original
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019 · 1910
Earlier work this paper cites.
Transformers: State-of-the-art natural language processing
Original
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019 · 1910
Earlier work this paper cites.
Pivoted document length normalization
Amit Singhal, Chris Buckley, and Mandar Mitra. 1996 · 1996
Earlier work this paper cites.
Do batch and user evaluations give the same results?
William R. Hersh, Andrew Turpin, Susan Price, Benjamin Chan, Dale Kramer, Lynetta Sacherek, and Daniel Olson. 2000 · 2000
Earlier work this paper cites.
Why batch and user evaluations do not give the same results
Andrew Turpin and William R. Hersh. 2001 · 2001
Earlier work this paper cites.
Document ranking with a pretrained sequence-to-sequence model
Original
Rodrigo Nogueira, Zhiying Jiang, and Jimmy Lin. 2020 · 2003
Earlier work this paper cites.
High accuracy retrieval with multiple nested ranker
Irina Matveeva, Chris Burges, Timo Burkard, Andy Laucius, and Leon Wong. 2006 · 2006
Earlier work this paper cites.
How do users find things with PubMed? Towards automatic utility evaluation with user simulations
Jimmy Lin and Mark D. Smucker. 2008 · 2008
Earlier work this paper cites.
Is searching full text more effective than searching abstracts?
Jimmy Lin. 2009 · 2009
Earlier work this paper cites.

Similar

Query understanding at Bing
Jan Pedersen. 2010 · 2010
Cited alongside, same era.
A cascade ranking model for efficient ranked retrieval
Lidan Wang, Jimmy Lin, and Donald Metzler. 2011 · 2011
Cited alongside, same era.
Effectiveness/efficiency tradeoffs for candidate generation in multi-stage retrieval architectures
Nima Asadi and Jimmy Lin. 2013 · 2013
Cited alongside, same era.
Overview of the TREC-2015 Microblog Track
Jimmy Lin, Miles Efron, Yulu Wang, Garrick Sherman, and Ellen Voorhees. 2015 · 2015
Cited alongside, same era.
An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition
George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R. Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, et al. 2015 · 2015
Cited alongside, same era.
TensorFlow: A system for large-scale machine learning
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016 · 2016
Cited alongside, same era.
MS MARCO: a human-generated machine reading comprehension dataset
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016 · 2016
Cited alongside, same era.
Cascade ranking for operational e-commerce search
Shichen Liu, Fei Xiao, Wenwu Ou, and Luo Si. 2017 · 2017
Cited alongside, same era.
Anserini: enabling the use of Lucene for information retrieval research
Peilin Yang, Hui Fang, and Jimmy Lin. 2017 · 2017
Cited alongside, same era.

Then

Anserini: reproducible ranking baselines using Lucene
Peilin Yang, Hui Fang, and Jimmy Lin. 2018 · 2018
Later among the works it cites.
Solr integration in the Anserini information retrieval toolkit
Ryan Clancy, Toke Eskildsen, Nick Ruest, and Jimmy Lin. 2019 · 2019
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.
PyTorch: an imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019 · 2019
Later among the works it cites.
Overview of the TREC 2019 precision medicine track
Kirk Roberts, Dina Demner-Fushman, Ellen M. Voorhees, William R. Hersh, Steven Bedrick, Alexander J. Lazar, Shubham Pant, and Funda Meric-Bernstam. 2019 · 2019
Later among the works it cites.
End-to-end open-domain question answering with BERTserini
Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019 · 2019
Later among the works it cites.
Cross-domain modeling of sentence-level evidence for document retrieval
Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, and Jimmy Lin. 2019 · 2019
Later among the works it cites.
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020 · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…