Balancing Efficiency and Effectiveness Trade-offs in Large Scale Multistage Retrieval Systems
In this talk, we will discuss recent work on managing tradeoffs between efficiency and effectiveness in modern multi-stage ranking architectures which are comprised of a candidate generation stage followed by one or more reranking stages. In such an architecture, the quality of the final ranked list may not be sensitive to the quality of initial candidate pool, especially in terms of early precision. We will briefly discuss two recent related papers from my group. In the first work, we explore dynamic cutoff prediction in early stage retrieval using query difficulty pre-retrieval features. We will then turn our attention to efficiency and effectiveness trade-offs in cascaded learning-to-rank algorithms. Specifically, we re-examine the importance of tightly integrating feature costs into multi-stage learning-to-rank (LTR) IR systems, and we present a novel approach to optimizing cascaded ranking models which can directly leverage a variety of different state-of-the-art LTR rankers such as LambdaMART and Gradient Boosted Decision Trees.