Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools

A Coefficient of Agreement for Nominal Scales

Jacob Cohen. 1960 · 1960

Earlier work this paper cites.

Law’s Empire

Ronald Dworkin. 1986 · 1986

Earlier work this paper cites.

Remarks: Clerks of Judge Luther A. Wilgarten, Jr

Roger J. Miner. 1989 · 1989

Earlier work this paper cites.

Is the Rule of Law an Essentially Contested Concept (in Florida)?

Jeremy Waldron. 2002 · 2002

Earlier work this paper cites.

Open Access in a Closed Universe: Lexis, Westlaw, Law Schools, and the Legal Information Market

Olufunmilayo B. Arewa. 2006 · 2006

Earlier work this paper cites.

Network Analysis and the Law: Measuring the Legal Importance of Precedents at the U.S. Supreme Court

James H. Fowler, Timothy R. Johnson, James F. Spriggs, II, Sangick Jeon, and Paul J. Wahlbeck. 2007 · 2007

Earlier work this paper cites.

Some (what) grand challenges for information retrieval

Nicholas J Belkin. 2008 · 2008

Earlier work this paper cites.

Automation bias: a systematic review of frequency, effect mediators, and mitigators

K. Goddard, A. Roudsari, and J. C. Wyatt. 2012 · 2012

Earlier work this paper cites.

Multistate Testing Practice Questions

BARBRI, Inc. 2013 · 2013

Earlier work this paper cites.

The Citation and Depreciation of U.S. Supreme Court Precedent

Ryan C. Black and James F. Spriggs, II. 2013 · 2013

Earlier work this paper cites.

How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016 · 2016

Earlier work this paper cites.

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Earlier work this paper cites.

Alphabetical List of Jurisdictions Adopting Model Rules

American Bar Association. 2018 · 2018

Earlier work this paper cites.

HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018 · 2018

Earlier work this paper cites.

Legal Uniformity in American Courts

Deborah Beim and Kelly Rader. 2019 · 2019

Earlier work this paper cites.

Unifying Human and Statistical Evaluation for Natural Language Generation

Tatsunori B. Hashimoto, Hugh Zhang, and Percy Liang. 2019 · 2019

Earlier work this paper cites.

Westlaw tip of the week: Checking cases with keycite

Thomson Reuters. 2019 · 2019

Earlier work this paper cites.

The Model Rules of Autonomous Conduct: Ethical Responsibilities of Lawyers and Artificial Intelligence

Ed Walters. 2019 · 2019

Earlier work this paper cites.

REALM: Retrieval-augmented language model pre-training

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020 · 2020

Earlier work this paper cites.

Measuring Massive Multitask Language Understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2020

Earlier work this paper cites.

Dense Passage Retrieval for Open-Domain Question Answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020

Earlier work this paper cites.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020 · 2020

Earlier work this paper cites.

Artificial Intelligence in the Legal Field and the Indispensable Human Element Legal Ethics Demands

Nicole Yamane. 2020 · 2020

Earlier work this paper cites.

A Human Being Wrote This Law Review Article: GPT-3 and the Practice of Law

Amy B. Cyphert. 2021 · 2021

Earlier work this paper cites.

AI and the Everything in the Whole Wide World Benchmark

Inioluwa Deborah Raji, Emily Denton, Emily M Bender, Alex Hanna, and Amandalynne Paullada. 2021 · 2021

Earlier work this paper cites.

When does pretraining help? assessing self-supervised learning for law and the casehold dataset of 53,000+ legal holdings

Lucia Zheng, Neel Guha, Brandon R Anderson, Peter Henderson, and Daniel E Ho. 2021 · 2021

Earlier work this paper cites.

On the Opportunities and Risks of Foundation Models

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2022 · 2022

Earlier work this paper cites.

Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset

Peter Henderson, Mark S. Krass, Lucia Zheng, Neel Guha, Christopher D. Manning, Dan Jurafsky, and Daniel E. Ho. 2022 · 2022

Earlier work this paper cites.

Important opinions on courtlistener are now summarized by the top experts — judges

Michael Lissner. 2022 · 2022

Earlier work this paper cites.

The fallacy of ai functionality

Inioluwa Deborah Raji, I. Elizabeth Kumar, Aaron Horowitz, and Andrew Selbst. 2022 · 2022

Earlier work this paper cites.

Human Evaluation of Conversations is an Open Problem: Comparing the sensitivity of various methods for evaluating dialogue agents

Eric Smith, Orion Hsu, Rebecca Qian, Stephen Roller, Y-Lan Boureau, and Jason Weston. 2022 · 2022

Earlier work this paper cites.

Do Language Models Know When They’re Hallucinating References?

Ayush Agrawal, Mirac Suzgun, Lester Mackey, and Adam Tauman Kalai. 2023 · 2023

Earlier work this paper cites.

Major Thomson Reuters News: Westlaw Gets Generative AI Research Plus Integration with Casetext CoCounsel; Gen AI Coming Soon to Practical Law

Bob Ambrogi. 2023 · 2023

Earlier work this paper cites.

Keep your AI claims in check

Michael Atleson. 2023 · 2023

Earlier work this paper cites.

ChatGPT, Esq.: Recasting Unauthorized Practice of Law in the Era of Generative AI

Joseph J. Avery, Patricia Sánchez Abril, and Alissa del Riego. 2023 · 2023

Earlier work this paper cites.

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

BIG-bench Authors. 2023 · 2023

Earlier work this paper cites.

Bots Behaving Badly: A Products Liability Approach to Chatbot-Generated Defamation

Nina Brown. 2023 · 2023

Cited alongside, same era.

GPT-4 alone is not a reliable legal solution—but it does enable one: CoCounsel harnesses GPT-4’s power to deliver results that legal professionals can rely on

Casetext. 2023 · 2023

Cited alongside, same era.

How is chatgpt’s behavior changing over time?

Original

Lingjiao Chen, Matei Zaharia, and James Zou. 2023 · 2023

Cited alongside, same era.

Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

Fabrizio Dell’Acqua, Edward McFowland, Ethan R. Mollick, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim R. Lakhani. 2023 · 2023

Cited alongside, same era.

Investigating data contamination in modern benchmarks for large language models

Generative AI and Legal Aid: Results from a Field Study and 100 Use Cases to Bridge the Access to Justice Gap

Colleen V. Chien and Miriam Kim. 2024 · 2024

Closest in time.

How Generative AI Can Help Address the Access to Justice Gap Through the Courts

Colleen V. Chien, Miriam Kim, Raj Akhil, and Rohit Rathish. 2024 · 2024

Closest in time.

Lawyering in the Age of Artificial Intelligence

Jonathan H. Choi, Amy Monahan, and Daniel Schwarcz. 2024 · 2024

Closest in time.

AI Survey: Where Artificial Intelligence Stands in the Legal Industry

Jack Collens, Rachel Reimer, Gerald Schifman, and Pamela Wilkinson. 2024 · 2024

Closest in time.

The Power of Noise: Redefining Retrieval for RAG Systems

Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024 · 2024

Closest in time.

Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models