Benchmarking Prompt Sensitivity in Large Language Models

Hauff, C., Hiemstra, D., de Jong, F.: A survey of pre-retrieval query performance predictors. In: CIKM (2008)

2008

Earlier work this paper cites.

Collins-Thompson, K., Bennett, P.N.: Predicting Query Performance via Classification, p. 140–152. Springer Berlin Heidelberg (2010). https://doi.org/10.1007/978-3-642-12275-0_15, http://dx.doi.org/10.1007/978-3-642-12275-0_15

2010

Earlier work this paper cites.

Joshi, M., Choi, E., Weld, D., Zettlemoyer, L.: TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In: Barzilay, R., Kan, M.Y. (eds.) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1601–1611. Association for Computational Linguistics, Vancouver, Canada (Jul 2017). https://doi.org/10.18653/v1/P17-1147, https://aclanthology.org/P17-1147

2017

Earlier work this paper cites.

Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., Manning, C.D.: HotpotQA: A dataset for diverse, explainable multi-hop question answering. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pp. 2369–2380. Association for Computational Linguistics, Brussels, Belgium (Oct-Nov 2018). https://doi.org/10.18653/v1/D18-1259, https://aclanthology.org/D18-1259

2018

Earlier work this paper cites.

Arabzadeh, N., Zarrinkalam, F., Jovanovic, J., Bagheri, E.: Geometric estimation of specificity within embedding spaces. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. p. 2109–2112. CIKM ’19, ACM (Nov 2019). https://doi.org/10.1145/3357384.3358152, http://dx.doi.org/10.1145/3357384.3358152

2019

Earlier work this paper cites.

Arabzadeh, N., Zarrinkalam, F., Jovanovic, J., Al-Obeidat, F., Bagheri, E.: Neural embedding-based specificity metrics for pre-retrieval query performance prediction. Information Processing & Management 57

2020

Earlier work this paper cites.

Arabzadeh, N., Zarrinkalam, F., Jovanovic, J., Bagheri, E.: Neural Embedding-Based Metrics for Pre-retrieval Query Performance Prediction, p. 78–85. Springer International Publishing (2020). https://doi.org/10.1007/978-3-030-45442-5_10, http://dx.doi.org/10.1007/978-3-030-45442-5_10

2020

Earlier work this paper cites.

Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., Zhou, M.: Minilm: deep self-attention distillation for task-agnostic compression of pre-trained transformers. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS ’20, Curran Associates Inc., Red Hook, NY, USA (2020)

2020

Earlier work this paper cites.

Arabzadeh, N., Bigdeli, A., Zihayat, M., Bagheri, E.: Query performance prediction through retrieval coherency. In: Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28–April 1, 2021, Proceedings, Part II 43. pp. 193–200. Springer (2021)

2021

Earlier work this paper cites.

Arabzadeh, N., Seifikar, M., Clarke, C.L.: Unsupervised question clarity prediction through retrieved item coherency. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. pp. 3811–3816 (2022)

2022

Earlier work this paper cites.

Gasparetto, A., Marcuzzo, M., Zangari, A., Albarelli, A.: A survey on text classification algorithms: From text to predictions. Information 13

2022

Earlier work this paper cites.

Arabzadeh, N., Hamidi Rad, R., Khodabakhsh, M., Bagheri, E.: Noisy perturbations for estimating query difficulty in dense retrievers. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. pp. 3722–3727 (2023)

2023

Earlier work this paper cites.

Bhargava, A., Witkowski, C., Shah, M., Thomson, M.W.: What’s the magic word? a control theory of llm prompting. ArXiv abs/2310.04444

Original

2023

Earlier work this paper cites.

Faggioli, G., Ferro, N., Muntean, C.I., Perego, R., Tonellotto, N.: A geometric framework for query performance prediction in conversational search. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1355–1365. SIGIR ’23, ACM (Jul 2023). https://doi.org/10.1145/3539618.3591625, http://dx.doi.org/10.1145/3539618.3591625

2023

Earlier work this paper cites.

Hambarde, K.A., Proença, H.: Information retrieval: Recent advances and beyond. IEEE Access 11

2023

Cited alongside, same era.

Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L.R., Lachaux, M.A., Stock, P., Scao, T.L., Lavril, T., Wang, T., Lacroix, T., Sayed, W.E.: Mistral 7b (2023), https://arxiv.org/abs/2310.06825

Original

2023

Cited alongside, same era.

Kamalloo, E., Dziri, N., Clarke, C., Rafiei, D.: Evaluating open-domain question answering in the era of large language models. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics (2023). https://doi.org/10.18653/v1/2023.acl-long.307, http://dx.doi.org/10.18653/v1/2023.acl-long.307

2023

Cited alongside, same era.

Lo, L.S.: The art and science of prompt engineering: A new literacy in the information age. Internet Reference Services Quarterly 27

2023

Arabzadeh, N., Meng, C., Aliannejadi, M., Bagheri, E.: Query performance prediction: Techniques and applications in modern information retrieval. In: Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region. pp. 291–294 (2024)

2024

Later among the works it cites.

Bigdeli, A., Arabzadeh, N., Bagheri, E.: Learning to jointly transform and rank difficult queries. In: European Conference on Information Retrieval. pp. 40–48. Springer (2024)

2024

Later among the works it cites.

Ebrahimi, S., Khodabakhsh, M., Arabzadeh, N., Bagheri, E.: Estimating query performance through rich contextualized query representations. In: European Conference on Information Retrieval. pp. 49–58. Springer (03 2024). https://doi.org/10.1007/978-3-031-56066-8_6

2024

Later among the works it cites.

Feng, Z., Zhou, H., Zhu, Z., Qian, J., Mao, K.: Unveiling and manipulating prompt influence in large language models. ArXiv abs/2405.11891

Original