IPEval: A Bilingual Intellectual Property Agency Consultation Evaluation Benchmark for Large Language Models

Understand

The rapid development of Large Language Models (LLMs) in vertical domains, including intellectual property (IP), lacks a specific evaluation benchmark for assessing their understanding, application, and reasoning abilities.

• To fill this gap, we introduce IPEval, the first evaluation benchmark tailored for IP agency and consulting tasks.
• IPEval comprises 2657 multiple-choice questions across four major dimensions: creation, application, protection, and management of IP.
• These questions span patent rights (inventions, utility models, designs), trademarks, copyrights, trade secrets, and other related laws.