Evaluation of medium-large Language Models at zero-shot closed book generative question answering

Understand

Large language models (LLMs) have garnered significant attention, but the definition of "large" lacks clarity.

• This paper focuses on medium-sized language models (MLMs), defined as having at least six billion parameters but less than 100 billion.
• The study evaluates MLMs regarding zero-shot generative question answering, which requires models to provide elaborate answers without external document retrieval.
• The paper introduces an own test dataset and presents results from human evaluation.