UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities

Understand

Retrieval-Augmented Generation (RAG) has shown substantial promise in improving factual accuracy by grounding model responses with external knowledge relevant to queries.

• However, most existing approaches are limited to a text-only corpus, and while recent efforts have extended RAG to other modalities such as images and videos, they typically operate over a single modality-specific corpus.
• In contrast, real-world queries vary widely in the type of knowledge they require, which a single type of knowledge source cannot address.
• To address this, we introduce UniversalRAG, an any-to-any RAG framework designed to retrieve and integrate knowledge from heterogeneous sources with diverse modalities and granularities.