RelevAI-Reviewer: A Benchmark on AI Reviewers for Survey Paper Relevance

Understand

Recent advancements in Artificial Intelligence (AI), particularly the widespread adoption of Large Language Models (LLMs), have significantly enhanced text analysis capabilities.

• This technological evolution offers considerable promise for automating the review of scientific papers, a task traditionally managed through peer review by fellow researchers.
• Despite its critical role in maintaining research quality, the conventional peer-review process is often slow and subject to biases, potentially impeding the swift propagation of scientific knowledge.
• In this paper, we propose RelevAI-Reviewer, an automatic system that conceptualizes the task of survey paper review as a classification problem, aimed at assessing the relevance of a paper in relation to a specified prompt, analogous to a "call for papers".