Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?

Language models are few-shot learners,

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., · 1901

Earlier work this paper cites.

Inter-coder agreement for computational linguistics,

R. Artstein, M. Poesio, · 2008

Earlier work this paper cites.

Human-aided computer cognition for e-discovery,

C. Hogan, R. Bauer, D. Brassil, · 2009

Earlier work this paper cites.

Computing krippendorff’s alpha-reliability (2011)

K. Krippendorff, · 2011

Earlier work this paper cites.

Applying an interactive machine learning approach to statutory analysis,

J. Šavelka, G. Trivedi, K. D. Ashley, · 2015

Earlier work this paper cites.

Autonomy and reliability of continuous active learning for technology-assisted review,

Original

G. V. Cormack, M. R. Grossman, · 2015

Earlier work this paper cites.

Scalability of continuous active learning for reliable high-recall text classification,

G. V. Cormack, M. R. Grossman, · 2016

Earlier work this paper cites.

Extracting case law sentences for argumentation about the meaning of statutory terms,

J. Šavelka, K. D. Ashley, · 2016

Earlier work this paper cites.

Sentence boundary detection in adjudicatory decisions in the united states,

J. Savelka, V. R. Walker, M. Grabmair, K. D. Ashley, · 2017

Earlier work this paper cites.

Classifying legal norms with active machine learning.,

B. Waltl, J. Muhr, I. Glaser, G. Bonczek, E. Scepankova, F. Matthes, · 2017

Earlier work this paper cites.

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, · 2017

Earlier work this paper cites.

Improving language understanding by generative pre-training (2018)

A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, · 2018

Earlier work this paper cites.

Computer-assisted creation of boolean search rules for text classification in the legal domain.,

H. Westermann, J. Savelka, V. R. Walker, K. D. Ashley, K. Benyekhlef, · 2019

Earlier work this paper cites.

Improving sentence retrieval from case law for statutory interpretation,

J. Savelka, H. Xu, K. D. Ashley, · 2019

Earlier work this paper cites.

Language models are unsupervised multitask learners (2019)

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, · 2019

Earlier work this paper cites.

Sentence embeddings and high-speed similarity search for fast computer assisted annotation of legal documents,

H. Westermann, J. Savelka, V. R. Walker, K. D. Ashley, K. Benyekhlef, · 2020

Cited alongside, same era.

Learning to rank sentences for explaining statutory terms.,

J. Savelka, K. D. Ashley, · 2020

Cited alongside, same era.

J. Savelka, Discovering sentences for argumentation about the meaning of statutory terms, Ph.D. thesis, University of Pittsburgh, 2020

2020

Cited alongside, same era.

S. Wang, Y. Liu, Y. Xu, C. Zhu, M. Zeng, Want To Reduce Labeling Cost? GPT-3 Can Help, 2021. URL: http://arxiv.org/abs/2108.13487 , arXiv:2108.13487 [cs]

Original

2021

Cited alongside, same era.

Discovering explanatory sentences in legal case decisions using pre-trained language models,

J. Šavelka, K. D. Ashley, · 2021

Cited alongside, same era.

T. Kuzman, I. Mozetič, N. Ljubešić, ChatGPT: Beginning of an End of Manual Linguistic Data Annotation? Use Case of Automatic Genre Identification, 2023. URL: http://arxiv.org/abs/2303.03953 , arXiv:2303.03953 [cs]

Original

2023

Closest in time.

Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech,

Original

F. Huang, H. Kwak, J. An, · 2023

Closest in time.

C. Ziems, W. Held, O. Shaikh, J. Chen, Z. Zhang, D. Yang, Can large language models transform computational social science?, 2023. arXiv:2305.03514

Original

2023

Closest in time.

Y. Zhu, P. Zhang, E.-U. Haq, P. Hui, G. Tyson, Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks, 2023. URL: http://arxiv.org/abs/2304.10145 , arXiv:2304.10145 [cs]

Original

2023

Closest in time.