Evaluating Embedding APIs for Information Retrieval

Understand

The ever-increasing size of language models curtails their widespread availability to the community, thereby galvanizing many companies into offering access to large language models through APIs.

• One particular type, suitable for dense retrieval, is a semantic embedding service that builds vector representations of input text.
• With a growing number of publicly available APIs, our goal in this paper is to analyze existing offerings in realistic retrieval scenarios, to assist practitioners and researchers in finding suitable services according to their needs.
• Specifically, we investigate the capabilities of existing semantic embedding APIs on domain generalization and multilingual retrieval.