Efficient Document Re-Ranking for Transformers by Precomputing Term Representations

Understand

Deep pretrained transformer networks are effective at various ranking tasks, such as question answering and ad-hoc document ranking.

• However, their computational expenses deem them cost-prohibitive in practice.
• Our proposed approach, called PreTTR (Precomputing Transformer Term Representations), considerably reduces the query-time latency of deep transformer networks (up to a 42x speedup on web document ranking) making these networks more practical to use in a real-time ranking scenario.
• Specifically, we precompute part of the document term representations at indexing time (without a query), and merge them with the query representation at query time to compute the final ranking score.