Fetching the paper…

Reading the bibliography…

2021

On Semantic Cognition, Inductive Generalization, and Language Models

Misra, Kanishka

Original
PDF
Post on X

Understand

My doctoral research focuses on understanding semantic knowledge in neural network models trained solely to predict natural language (referred to as language models, or LMs), by drawing on insights from the study of concepts and categories grounded in cognitive science.

I propose a framework inspired by 'inductive reasoning,' a phenomenon that sheds light on how humans utilize background knowledge to make inductive leaps and generalize from new pieces of information about concepts and their properties.
Drawing from experiments that study inductive reasoning, I propose to analyze semantic inductive generalization in LMs using phenomena observed in human-induction literature, investigate inductive behavior on tasks such as implicit reasoning and emergent feature recognition, and analyze and relate induction dynamics to the learned conceptual representation space.

On Semantic Cognition, Inductive Generalization, and Language Models · Around

Built on

Conceptual Change in Childhood
Carey, S. 1985 · 1985
Earlier work this paper cites.
Category-based induction
Osherson, D. N.; Smith, E. E.; Wilkie, O.; Lopez, A.; and Shafir, E. 1990 · 1990
Earlier work this paper cites.
Semantic cognition: A parallel distributed processing approach
Rogers, T. T.; and McClelland, J. L. 2004 · 2004
Earlier work this paper cites.
Structured statistical models of inductive reasoning
Kemp, C.; and Tenenbaum, J. B. 2009 · 2009
Earlier work this paper cites.

Similar

Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop
Alishahi, A.; Chrupała, G.; and Linzen, T. 2019 · 2019
Cited alongside, same era.
Comet: Commonsense transformers for automatic knowledge graph construction
Bosselut, A.; Rashkin, H.; Sap, M.; Malaviya, C.; Celikyilmaz, A.; and Choi, Y. 2019 · 2019
Cited alongside, same era.
Language models as knowledge bases?
Petroni, F.; Rocktäschel, T.; Lewis, P.; Bakhtin, A.; Wu, Y.; Miller, A. H.; and Riedel, S. 2019 · 2019
Cited alongside, same era.

Then

CLUTRR: A diagnostic benchmark for inductive reasoning from text
Sinha, K.; Sodhani, S.; Dong, J.; Pineau, J.; and Hamilton, W. L. 2019 · 2019
Later among the works it cites.
Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge
Talmor, A.; Tafjord, O.; Clark, P.; Goldberg, Y.; and Berant, J. 2020 · 2020
Later among the works it cites.
Do language models learn typicality judgments from text?
Misra, K.; Ettinger, A.; and Rayz, J. T. 2021 · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…