Interpreting search result rankings through intent modeling

Understand

Given the recent interest in arguably accurate yet non-interpretable neural models, even with textual features, for document ranking we try to answer questions relating to how to interpret rankings.

• In this paper we take first steps towards a framework for the interpretability of retrieval models with the aim of answering 3 main questions "What is the intent of the query according to the ranker?", "Why is a document ranked higher than another for the query?" and "Why is a document relevant to the query?" Our framework is predicated on the assumption that text based retrieval model behavior can be estimated using query expansions in conjunction with a simpler retrieval model irrespective of the underlying ranker.
• We conducted experiments with the Clueweb test collection.
• We show how our approach performs for both simpler models with a closed form notation (which allows us to measure the accuracy of the interpretation) and neural ranking models.