Fetching the paper…
Reading the bibliography…
Transformers are neural networks that revolutionized natural language processing and machine learning.
R. B. Potts, Some generalized order-disorder transformations, Mathematical Proceedings of the Cambridge Philosophical Society 48
1952
Earlier work this paper cites.
J. Besag, Statistical analysis of non-lattice data, Journal of the Royal Statistical Society. Series D (The Statistician) 24
1975
Earlier work this paper cites.
F. Y. Wu, The potts model, Rev. Mod. Phys. 54
1982
Earlier work this paper cites.
E. Gardner and B. Derrida, Three unfinished works on the optimal storage capacity of networks, Journal of Physics A: Mathematical and General 22
1989
Earlier work this paper cites.
H. S. Seung, H. Sompolinsky, and N. Tishby, Statistical mechanics of learning from examples, Phys. Rev. A 45
1992
Earlier work this paper cites.
D. Saad and S. A. Solla, Exact solution for on-line learning in multilayer neural networks, Phys. Rev. Lett. 74
1995
Earlier work this paper cites.
M. Biehl and H. Schwarze, Learning by on-line gradient descent, Journal of Physics A: Mathematical and General 28
1995
Earlier work this paper cites.
M. Opper and W. Kinzel, Statistical mechanics of generalization, Models of Neural Networks III: Association, Generalization, and Representation (1996)
1996
Earlier work this paper cites.
A. Engel and C. Van den Broeck, Statistical Mechanics of Learning (Cambridge University Press, 2001)
2001
Earlier work this paper cites.
2005
Earlier work this paper cites.
A. Hyvärinen, Consistency of Pseudolikelihood Estimation of Fully Visible Boltzmann Machines, Neural Computation 18
2006
Earlier work this paper cites.
2010
Earlier work this paper cites.
P. Ravikumar, M. J. Wainwright, and J. D. Lafferty, High-dimensional Ising model selection using l1-regularized logistic regression, The Annals of Statistics 38
2010
Earlier work this paper cites.
S. Cocco and R. Monasson, Adaptive cluster expansion for inferring boltzmann machines with noisy data, Phys. Rev. Lett. 106
2011
Earlier work this paper cites.
F. Morcos, A. Pagnani, B. Lunt, A. Bertolino, D. S. Marks, C. Sander, R. Zecchina, J. N. Onuchic, T. Hwa, and M. Weigt, Direct-coupling analysis of residue coevolution captures native contacts across many protein families, Proceedings of the National Academy of Sciences 108
2011
Earlier work this paper cites.
F. Ricci-Tersenghi, The bethe approximation for solving the inverse ising problem: a comparison with other inference methods, Journal of Statistical Mechanics: Theory and Experiment 2012
2012
Earlier work this paper cites.
E. Aurell and M. Ekeberg, Inverse ising inference using all the data, Phys. Rev. Lett. 108
2012
Earlier work this paper cites.
R. D. Finn, A. Bateman, et al. , Pfam: the protein families database, Nucleic Acids Research 42
2013
Cited alongside, same era.
M. Ekeberg, C. Lövkvist, Y. Lan, M. Weigt, and E. Aurell, Improved contact prediction in proteins: Using pseudolikelihoods to infer potts models, Phys. Rev. E 87
2013
Cited alongside, same era.
2016
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need (2017)
2017
Cited alongside, same era.
2017
S. Goldt, M. S. Advani, A. M. Saxe, F. Krzakala, and L. Zdeborová, Dynamics of stochastic gradient descent for two-layer neural networks in the teacher–student setup, Journal of Statistical Mechanics: Theory and Experiment 2020
2020
Later among the works it cites.
J. Jumper, R. Evans, et al. , Highly accurate protein structure prediction with alphafold, Nature 596
2021
Later among the works it cites.
F. Gerace, B. Loureiro, F. Krzakala, M. Mézard, and L. Zdeborová, Generalisation error in learning with random features and the hidden manifold model, Journal of Statistical Mechanics: Theory and Experiment 2021
2021
Later among the works it cites.
P. Nakkiran, G. Kaplun, Y. Bansal, T. Yang, B. Barak, and I. Sutskever, Deep double descent: where bigger models and more data hurt, Journal of Statistical Mechanics: Theory and Experiment 2021
2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
H. C. Nguyen, R. Zecchina, and J. Berg, Inverse statistical problems: from the inverse ising problem to data science, Advances in Physics 66
2017
Cited alongside, same era.
2018
Cited alongside, same era.
A. Radford, K. Narasimhan, et al. , Improving language understanding by generative pre-training (2018)
2018
Cited alongside, same era.
S. Chung, D. D. Lee, and H. Sompolinsky, Classification and geometry of general perceptual manifolds, Phys. Rev. X 8
2018
Cited alongside, same era.
S. Cocco, C. Feinauer, M. Figliuzzi, R. Monasson, and M. Weigt, Inverse statistical physics of protein sequences: a key issues review, Reports on Progress in Physics 81
2018
Cited alongside, same era.
G. Livan, M. Novaes, and P. Vivo, Introduction to Random Matrices (Springer International Publishing, 2018)
2018
Cited alongside, same era.
J. Bradbury, R. Frostig, et al. , JAX: composable transformations of Python+NumPy programs (2018)
2018
Cited alongside, same era.
A. Favero, F. Cagnetta, and M. Wyart, Locality defeats the curse of dimensionality in convolutional teacher–student scenarios, Journal of Statistical Mechanics: Theory and Experiment 2022
2022
Later among the works it cites.
B. Loureiro, C. Gerbelot, H. Cui, S. Goldt, F. Krzakala, M. Mézard, and L. Zdeborová, Learning curves of generic features maps for realistic datasets with a teacher-student model, Journal of Statistical Mechanics: Theory and Experiment 2022
2022
Later among the works it cites.
A. Ingrosso and S. Goldt, Data-driven emergence of convolutional structure in neural networks, Proceedings of the National Academy of Sciences 119
2022
Later among the works it cites.
M. Refinetti and S. Goldt, The dynamics of representation learning in shallow, non-linear autoencoders, in International Conference on Machine Learning (PMLR, 2022) pp. 18499–18519
2022
Later among the works it cites.
S. Goldt, B. Loureiro, G. Reeves, F. Krzakala, M. Mézard, and L. Zdeborová, The gaussian equivalence of generative models for learning with shallow neural networks, in Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference (2022)
2022
Later among the works it cites.
2022
Later among the works it cites.
2022
Later among the works it cites.
2023
Closest in time.
2023
Closest in time.
2023
Closest in time.
2023
Closest in time.
2023
Closest in time.
OpenAI, Gpt-4 technical report (2024), arXiv:2303.08774 [cs.CL]
2024
Closest in time.