Entropy, Thermodynamics and the Geometrization of the Language Model

Description of language design

Martin Joos · 1950

Earlier work this paper cites.

Distributional structure

Zellig S Harris · 1954

Earlier work this paper cites.

A synopsis of linguistic theory, 1930-1955

John Firth · 1957

Earlier work this paper cites.

Naive set theory

Paul Richard Halmos · 1960

Earlier work this paper cites.

Mechanics: Volume 1

L.D. Landau, E.M. Lifshitz, and J.B. Sykes · 1976

Earlier work this paper cites.

A course in mathematical logic

John Lane Bell and Moshe Machover · 1977

Earlier work this paper cites.

Information Theory, Inference and Learning Algorithms

David J. C. MacKay · 2003

Earlier work this paper cites.

Real analysis: measure theory, integration, and Hilbert spaces

Elias M. Stein and Rami Shakarchi · 2009

Earlier work this paper cites.

A course in mathematical logic for mathematicians

Yuri I Manin and Boris Zilber · 2010

Cited alongside, same era.

Basic topology

Mark Anthony Armstrong · 2013

Cited alongside, same era.

Efficient estimation of word representations in vector space, 2013

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean · 2013

Cited alongside, same era.

Glove: Global vectors for word representation

Jeffrey Pennington, Richard Socher, and Christopher D Manning · 2014

Cited alongside, same era.

A latent variable model approach to pmi-based word embeddings

Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski · 2016

Cited alongside, same era.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017

Cited alongside, same era.

An introduction to quantum field theory

Michael E Peskin · 2018

Later among the works it cites.

Group Theory (v4.00), 2021

James S. Milne · 2021

Later among the works it cites.

Manifolds and differential geometry

Jeffrey M. Lee · 2022

Later among the works it cites.

Basics of statistical physics

Harald J. W. Muller-Kirsten · 2022

Later among the works it cites.

Emergent abilities of large language models

Original

Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al · 2022

Later among the works it cites.

Large language models, 2023

Michael R. Douglas · 2023

Later among the works it cites.