Built on

Voorhees, E.M.: Overview of the trec 2004 robust track. Proceedings of the Thirteenth Text REtrieval Conference, TREC 2004, Gaithersburg, Maryland, November 16-19, 2004 (2004)
2004
Earlier work this paper cites.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NIPS (2017)
2017
Earlier work this paper cites.
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., Rosenber, M., Song, X., Stoica, A., Tiwary, S., Wang, T.: Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2018)
Original
2018
Earlier work this paper cites.
Dai, Z., Callan, J.: Context-aware sentence/passage term importance estimation for first stage retrieval. arXiv preprint arXiv:1910.10687 (2019)
Original
2019
Earlier work this paper cites.
MacAvaney, S., Yates, A., Cohan, A., Goharian, N.: Cedr: Contextualized embeddings for document ranking. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 1101–1104 (2019)
2019
Earlier work this paper cites.
Nogueira, R., Cho, K.: Passage re-ranking with bert. arXiv preprint arXiv:1901.04085 (2019)
Original
2019
Earlier work this paper cites.
Nogueira, R., Lin, J.: From doc2query to doctttttquery. Online preprint 6
2019
Earlier work this paper cites.
Nogueira, R., Yang, W., Lin, J., Cho, K.: Document expansion by query prediction. arXiv preprint arXiv:1904.08375 (2019)
Original
2019
Earlier work this paper cites.
Roberts, K., Demner-Fushman, D., Voorhees, E., Hersh, W., Bedrick, S., Lazar, A.J., Pant, S.: Overview of the trec 2019 precision medicine track. The … text REtrieval conference : TREC. Text REtrieval Conference 26
2019
Earlier work this paper cites.
Wang, Y.E., Wei, G.Y., Brooks, D.: Benchmarking tpu, gpu, and cpu platforms for deep learning. arXiv preprint arXiv:1907.10701 (2019)
Original
2019
Earlier work this paper cites.
Bai, Y., Li, X., Wang, G., Zhang, C., Shang, L., Xu, J., Wang, Z., Wang, F., Liu, Q.: Sparterm: Learning term-based sparse representation for fast text retrieval. arXiv preprint arXiv:2010.00768 (2020)
Original
2020
Earlier work this paper cites.
Boytsov, L., Nyberg, E.: Flexible retrieval with nmslib and flexneuart. In: Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS). pp. 32–43 (2020)
2020
Earlier work this paper cites.
Li, C., Yates, A., MacAvaney, S., He, B., Sun, Y.: Parade: Passage representation aggregation for document reranking. arXiv preprint arXiv:2008.09093 (2020)
Original
2020
Earlier work this paper cites.
Nogueira, R., Jiang, Z., Pradeep, R., Lin, J.: Document ranking with a pretrained sequence-to-sequence model. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. pp. 708–718 (2020)
2020
Earlier work this paper cites.
Pradeep, R., Ma, X., Zhang, X., Cui, H., Xu, R., Nogueira, R., Lin, J.J., Cheriton, D.R.: H2oloo at trec 2020: When all you got is a hammer… deep learning, health misinformation, and precision medicine. In: TREC (2020)
2020
Earlier work this paper cites.
Qiao, Y., Chen, H., Cao, L., Chen, L., Li, P., Wang, J., Gao, P., Ni, Y., Xie, G.: Pash at trec 2020 deep learning track: Dense matching for nested ranking. In: TREC (2020)
2020
Earlier work this paper cites.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21
2020
Earlier work this paper cites.
Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., Zhou, M.: Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957 (2020)
Original
2020
Earlier work this paper cites.
Zhang, E., Gupta, N., Nogueira, R., Cho, K., Lin, J.: Rapidly deploying a neural search engine for the covid-19 open research dataset. In: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020 (2020)
2020
Earlier work this paper cites.
Craswell, N., Mitra, B., Yilmaz, E., Campos, D.: Overview of the trec 2020 deep learning track (2021). https://doi.org/10.48550/ARXIV.2102.07662, https://arxiv.org/abs/2102.07662
Original
2021
Earlier work this paper cites.

Similar

Craswell, N., Mitra, B., Yilmaz, E., Campos, D., Lin, J.: Overview of the trec 2021 deep learning track. In: Text REtrieval Conference (TREC). TREC (May 2022), https://www.microsoft.com/en-us/research/publication/overview-of-the-trec-2021-deep-learning-track/
2021
Cited alongside, same era.
Formal, T., Lassance, C., Piwowarski, B., Clinchant, S.: Splade v2: Sparse lexical and expansion model for information retrieval. arXiv preprint arXiv:2109.10086 (2021)
Original
2021
Cited alongside, same era.
Formal, T., Piwowarski, B., Clinchant, S.: Splade: Sparse lexical and expansion model for first stage ranking. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 2288–2292 (2021)
2021
Cited alongside, same era.
Gao, L., Dai, Z., Callan, J.: Rethink training of bert rerankers in multi-stage retrieval pipeline. arXiv preprint arXiv:2101.08751 (2021)
Original
2021

Then

Formal, T., Lassance, C., Piwowarski, B., Clinchant, S.: From distillation to hard negative sampling: Making sparse neural ir models more effective. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 2353–2359. SIGIR ’22, Association for Computing Machinery, New York, NY, USA (2022). https://doi.org/10.1145/3477495.3531857, https://doi.org/10.1145/3477495.3531857
2022
Closest in time.
Gao, L., Callan, J.: Unsupervised corpus aware language model pre-training for dense passage retrieval. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2843–2853 (2022)
2022
Closest in time.
Gao, L., Callan, J.: Unsupervised corpus aware language model pre-training for dense passage retrieval. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2843–2853. Association for Computational Linguistics, Dublin, Ireland (May 2022), https://aclanthology.org/2022.acl-long.203
2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2022

In Defense of Cross-Encoders for Zero-Shot Retrieval

Rosa, Guilherme, Bonifacio, Luiz, Jeronymo, Vitor et al.

Understand

Bi-encoders and cross-encoders are widely used in many state-of-the-art retrieval pipelines.

In this work we study the generalization ability of these two types of architectures on a wide range of parameter count on both in-domain and out-of-domain scenarios.
We find that the number of parameters and early query-document interactions of cross-encoders play a significant role in the generalization ability of retrieval models.
Our experiments show that increasing model size results in marginal gains on in-domain test sets, but much larger gains in new domains never seen during fine-tuning.

Built on

Voorhees, E.M.: Overview of the trec 2004 robust track. Proceedings of the Thirteenth Text REtrieval Conference, TREC 2004, Gaithersburg, Maryland, November 16-19, 2004 (2004)
2004
Earlier work this paper cites.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NIPS (2017)
2017
Earlier work this paper cites.
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., Rosenber, M., Song, X., Stoica, A., Tiwary, S., Wang, T.: Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2018)
Original
2018
Earlier work this paper cites.
Dai, Z., Callan, J.: Context-aware sentence/passage term importance estimation for first stage retrieval. arXiv preprint arXiv:1910.10687 (2019)
Original
2019
Earlier work this paper cites.
MacAvaney, S., Yates, A., Cohan, A., Goharian, N.: Cedr: Contextualized embeddings for document ranking. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 1101–1104 (2019)
2019
Earlier work this paper cites.
Nogueira, R., Cho, K.: Passage re-ranking with bert. arXiv preprint arXiv:1901.04085 (2019)
Original
2019
Earlier work this paper cites.
Nogueira, R., Lin, J.: From doc2query to doctttttquery. Online preprint 6
2019
Earlier work this paper cites.
Nogueira, R., Yang, W., Lin, J., Cho, K.: Document expansion by query prediction. arXiv preprint arXiv:1904.08375 (2019)
Original
2019
Earlier work this paper cites.
Roberts, K., Demner-Fushman, D., Voorhees, E., Hersh, W., Bedrick, S., Lazar, A.J., Pant, S.: Overview of the trec 2019 precision medicine track. The … text REtrieval conference : TREC. Text REtrieval Conference 26
2019
Earlier work this paper cites.
Wang, Y.E., Wei, G.Y., Brooks, D.: Benchmarking tpu, gpu, and cpu platforms for deep learning. arXiv preprint arXiv:1907.10701 (2019)
Original
2019
Earlier work this paper cites.
Bai, Y., Li, X., Wang, G., Zhang, C., Shang, L., Xu, J., Wang, Z., Wang, F., Liu, Q.: Sparterm: Learning term-based sparse representation for fast text retrieval. arXiv preprint arXiv:2010.00768 (2020)
Original
2020
Earlier work this paper cites.
Boytsov, L., Nyberg, E.: Flexible retrieval with nmslib and flexneuart. In: Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS). pp. 32–43 (2020)
2020
Earlier work this paper cites.
Li, C., Yates, A., MacAvaney, S., He, B., Sun, Y.: Parade: Passage representation aggregation for document reranking. arXiv preprint arXiv:2008.09093 (2020)
Original
2020
Earlier work this paper cites.
Nogueira, R., Jiang, Z., Pradeep, R., Lin, J.: Document ranking with a pretrained sequence-to-sequence model. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. pp. 708–718 (2020)
2020
Earlier work this paper cites.
Pradeep, R., Ma, X., Zhang, X., Cui, H., Xu, R., Nogueira, R., Lin, J.J., Cheriton, D.R.: H2oloo at trec 2020: When all you got is a hammer… deep learning, health misinformation, and precision medicine. In: TREC (2020)
2020
Earlier work this paper cites.
Qiao, Y., Chen, H., Cao, L., Chen, L., Li, P., Wang, J., Gao, P., Ni, Y., Xie, G.: Pash at trec 2020 deep learning track: Dense matching for nested ranking. In: TREC (2020)
2020
Earlier work this paper cites.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21
2020
Earlier work this paper cites.
Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., Zhou, M.: Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957 (2020)
Original
2020
Earlier work this paper cites.
Zhang, E., Gupta, N., Nogueira, R., Cho, K., Lin, J.: Rapidly deploying a neural search engine for the covid-19 open research dataset. In: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020 (2020)
2020
Earlier work this paper cites.
Craswell, N., Mitra, B., Yilmaz, E., Campos, D.: Overview of the trec 2020 deep learning track (2021). https://doi.org/10.48550/ARXIV.2102.07662, https://arxiv.org/abs/2102.07662
Original
2021
Earlier work this paper cites.

Similar

Craswell, N., Mitra, B., Yilmaz, E., Campos, D., Lin, J.: Overview of the trec 2021 deep learning track. In: Text REtrieval Conference (TREC). TREC (May 2022), https://www.microsoft.com/en-us/research/publication/overview-of-the-trec-2021-deep-learning-track/
2021
Cited alongside, same era.
Formal, T., Lassance, C., Piwowarski, B., Clinchant, S.: Splade v2: Sparse lexical and expansion model for information retrieval. arXiv preprint arXiv:2109.10086 (2021)
Original
2021
Cited alongside, same era.
Formal, T., Piwowarski, B., Clinchant, S.: Splade: Sparse lexical and expansion model for first stage ranking. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 2288–2292 (2021)
2021
Cited alongside, same era.
Gao, L., Dai, Z., Callan, J.: Rethink training of bert rerankers in multi-stage retrieval pipeline. arXiv preprint arXiv:2101.08751 (2021)
Original
2021

Then

Formal, T., Lassance, C., Piwowarski, B., Clinchant, S.: From distillation to hard negative sampling: Making sparse neural ir models more effective. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 2353–2359. SIGIR ’22, Association for Computing Machinery, New York, NY, USA (2022). https://doi.org/10.1145/3477495.3531857, https://doi.org/10.1145/3477495.3531857
2022
Closest in time.
Gao, L., Callan, J.: Unsupervised corpus aware language model pre-training for dense passage retrieval. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2843–2853 (2022)
2022
Closest in time.
Gao, L., Callan, J.: Unsupervised corpus aware language model pre-training for dense passage retrieval. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2843–2853. Association for Computational Linguistics, Dublin, Ireland (May 2022), https://aclanthology.org/2022.acl-long.203
2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Hofstätter, S., Lin, S.C., Yang, J.H., Lin, J., Hanbury, A.: Efficiently teaching an effective dense retriever with balanced topic aware sampling. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 113–122 (2021)

2021

Cited alongside, same era.

Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., Grave, E.: Unsupervised dense information retrieval with contrastive learning (2021). https://doi.org/10.48550/ARXIV.2112.09118, https://arxiv.org/abs/2112.09118