MaScQA: A Question Answering Dataset for Investigating Materials Science Knowledge of Large Language Models

Understand

Information extraction and textual comprehension from materials literature are vital for developing an exhaustive knowledge base that enables accelerated materials discovery.

• Language models have demonstrated their capability to answer domain-specific questions and retrieve information from knowledge bases.
• However, there are no benchmark datasets in the materials domain that can evaluate the understanding of the key concepts by these language models.
• In this work, we curate a dataset of 650 challenging questions from the materials domain that require the knowledge and skills of a materials student who has cleared their undergraduate degree.