Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession

Understand

This article reports the results of a study examining the ability of legal and non-legal Large Language Models to perform legal analysis using the Issue-Rule-Application-Conclusion framework.

• LLMs were tested on legal reasoning tasks involving rule analysis and analogical reasoning.
• The results show that LLMs can conduct basic IRAC analysis, but are limited by brief responses lacking detail, an inability to commit to answers, false confidence, and hallucinations.
• The study compares legal and nonlegal LLMs, identifies shortcomings, and explores traits that may hinder their ability to think like a lawyer.