ELIAS: End-to-End Learning to Index and Search in Large Output Spaces

Understand

Extreme multi-label classification (XMC) is a popular framework for solving many real-world problems that require accurate prediction from a very large number of potential output choices.

• A popular approach for dealing with the large label space is to arrange the labels into a shallow tree-based index and then learn an ML model to efficiently search this index via beam search.
• Existing methods initialize the tree index by clustering the label space into a few mutually exclusive clusters based on pre-defined features and keep it fixed throughout the training procedure.
• This approach results in a sub-optimal indexing structure over the label space and limits the search performance to the quality of choices made during the initialization of the index.