Fetching the paper…

Reading the bibliography…

2023

Mind meets machine: Unravelling GPT-4's cognitive psychology

Dhingra, Sifatkaur, Singh, Manmeet, SB, Vaisakh et al.

Original
PDF
Post on X

Understand

Cognitive psychology delves on understanding perception, attention, memory, language, problem-solving, decision-making, and reasoning.

Large language models (LLMs) are emerging as potent tools increasingly capable of performing human-level tasks.
The recent development in the form of GPT-4 and its demonstrated success in tasks complex to humans exam and complex problems has led to an increased confidence in the LLMs to become perfect instruments of intelligence.
Although GPT-4 report has shown performance on some cognitive psychology tasks, a comprehensive assessment of GPT-4, via the existing well-established datasets is required.

Mind meets machine: Unravelling GPT-4's cognitive psychology · Around

Built on

A basis for a mathematical theory of computation
John McCarthy · 1959
Earlier work this paper cites.
Understanding natural language
Terry Winograd · 1972
Earlier work this paper cites.
Commonsenseqa: A question answering challenge targeting commonsense knowledge
Original
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant · 2018
Earlier work this paper cites.
Hellaswag: Can a machine really finish your sentence?
Original
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi · 2019
Earlier work this paper cites.
Superglue: A stickier benchmark for general-purpose language understanding systems
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman · 2019
Earlier work this paper cites.

Similar

Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference
Original
R Thomas McCoy, Ellie Pavlick, and Tal Linzen · 2019
Cited alongside, same era.
Winogrande: An adversarial winograd schema challenge at scale
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi · 2021
Cited alongside, same era.
Measuring mathematical problem solving with the math dataset
Original
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt · 2021
Cited alongside, same era.
Text and patterns: For effective chain of thought, it takes two to tango
Original
Aman Madaan and Amir Yazdanbakhsh · 2022
Cited alongside, same era.

Then

Systematic generalization and emergent structures in transformers trained on structured tasks
Original
Yuxuan Li and James L McClelland · 2022
Later among the works it cites.
Using large language models to simulate multiple humans
Original
Gati Aher, Rosa I Arriaga, and Adam Tauman Kalai · 2022
Later among the works it cites.
Gpt-4 technical report
OpenAI · 2023
Closest in time.
Probing the psychology of ai models
Richard Shiffrin and Melanie Mitchell · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…