Fetching the paper…
Reading the bibliography…
We study the optimal memorization capacity of modern Hopfield models and Kernelized Hopfield Models (KHMs), a transformer-compatible class of Dense Associative Memories.
Resultats sur lempilement de calottes egales sur une perisphere de rn et correction a un travail anterieur
Claude Chabauty · 1953
Earlier work this paper cites.
Probability of error for optimal codes in a gaussian channel
Claude E Shannon · 1959
Earlier work this paper cites.
Capabilities of bounded discrepancy decoding
Aaron D Wyner · 1965
Earlier work this paper cites.
Non-holographic associative memory
David J Willshaw, O Peter Buneman, and Hugh Christopher Longuet-Higgins · 1969
Earlier work this paper cites.
Error-correcting codes
William Wesley Peterson and Edward J Weldon · 1972
Earlier work this paper cites.
Vector packing in finite dimensional vector spaces
Michael H Moore · 1974
Earlier work this paper cites.
On bounds for packings on a sphere and in space
Grigorii Anatol’evich Kabatiansky and Vladimir Iosifovich Levenshtein · 1978
Earlier work this paper cites.
Neural networks and physical systems with emergent collective computational abilities
John J Hopfield · 1982
Earlier work this paper cites.
Neurons with graded response have collective computational properties like those of two-state neurons
John J Hopfield · 1984
Earlier work this paper cites.
Data compression
Debra A Lelewer and Daniel S Hirschberg · 1987
Earlier work this paper cites.
Sparse distributed memory
Pentti Kanerva · 1988
Earlier work this paper cites.
Spherical codes and designs
Philippe Delsarte, Jean-Marie Goethals, and Johan Jacob Seidel · 1991
Earlier work this paper cites.
Acceleration of stochastic approximation by averaging
Boris T Polyak and Anatoli B Juditsky · 1992
Earlier work this paper cites.
The concave-convex procedure (cccp)
Alan L Yuille and Anand Rangarajan · 2001
Earlier work this paper cites.
On the convergence properties of the projected gradient method for convex optimization
Alfredo N Iusem · 2003
Earlier work this paper cites.
Grassmannian frames with applications to coding and communication
Thomas Strohmer and Robert W Heath Jr · 2003
Earlier work this paper cites.
A handbook of γ \gamma -convergence
Andrea Braides · 2006
Earlier work this paper cites.
Brain–machine interfaces: past, present and future
Mikhail A Lebedev and Miguel AL Nicolelis · 2006
Earlier work this paper cites.
Brain-computer interfaces, virtual reality, and videogames
Anatole Lécuyer, Fabien Lotte, Richard B Reilly, Robert Leeb, Michitaka Hirose, and Mel Slater · 2008
Earlier work this paper cites.
Variational analysis , volume 317
R Tyrrell Rockafellar and Roger J-B Wets · 2009
Earlier work this paper cites.
On the convergence of the concave-convex procedure
Bharath K Sriperumbudur and Gert RG Lanckriet · 2009
Earlier work this paper cites.
Finding and investigating exact spherical codes
Jeffrey Wang · 2009
Earlier work this paper cites.
Application of competitive hopfield neural network to brain-computer interface systems
Wei-Yen Hsu · 2012
Earlier work this paper cites.
Sphere packings, lattices and groups , volume 290
John Horton Conway and Neil James Alexander Sloane · 2013
Earlier work this paper cites.
Brain–machine interface in chronic stroke rehabilitation: a controlled study
Ander Ramos-Murguialday, Doris Broetz, Massimiliano Rea, Leonhard Läer, Özge Yilmaz, Fabricio L Brasil, Giulia Liberati, Marco R Curado, Eliana Garcia-Cossio, Alexandros Vyziotis, et al · 2013
Earlier work this paper cites.
The pennbmbi: Design of a general purpose wireless brain-machine-brain interface system
Xilin Liu, Milin Zhang, Basheer Subei, Andrew G Richardson, Timothy H Lucas, and Jan Van der Spiegel · 2015
Earlier work this paper cites.
Dense associative memory for pattern recognition
Dmitry Krotov and John J Hopfield · 2016
Earlier work this paper cites.
On a model of associative memory with huge storage capacity
Mete Demircigil, Judith Heusel, Matthias Löwe, Sven Upgang, and Franck Vermet · 2017
Earlier work this paper cites.
Preventing neurodegenerative memory loss in hopfield neuronal networks using cerebral organoids or external microelectronics
Megan Morrison, Pedro D Maia, J Nathan Kutz, et al · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin · 2018
Cited alongside, same era.
On kissing numbers and spherical codes in high dimensions
Matthew Jenssen, Felix Joos, and Will Perkins · 2018
Cited alongside, same era.
Averaging stochastic gradient descent on riemannian manifolds
Nilesh Tripuraneni, Nicolas Flammarion, Francis Bach, and Michael I Jordan · 2018
Cited alongside, same era.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Later among the works it cites.
Conformal prediction for time series with modern hopfield networks
Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter · 2023
Later among the works it cites.
Transformers as statisticians: Provable in-context learning with in-context algorithm selection
Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, and Song Mei · 2023
Later among the works it cites.
Sparks of artificial general intelligence: Early experiments with gpt-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ben Peters, Vlad Niculae, and André FT Martins · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Cited alongside, same era.
Optimization on the surface of the (hyper)-sphere
Parameswaran Raman and Jiasen Yang · 2019
Cited alongside, same era.
Brain–machine interfaces from motor to mood
Maryam M Shanechi · 2019
Cited alongside, same era.
Prevalence of neural collapse during the terminal phase of deep learning training
Vardan Papyan, XY Han, and David L Donoho · 2020
Cited alongside, same era.
Hopfield networks is all you need
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, et al · 2020
Cited alongside, same era.
Learning a minimax optimizer: A pilot study
Jiayi Shen, Xiaohan Chen, Howard Heaton, Tianlong Chen, Jialin Liu, Wotao Yin, and Zhangyang Wang · 2020
Cited alongside, same era.
Thomas F Burns and Tomoki Fukai · 2023
Later among the works it cites.
Scaling laws for associative memories
Vivien Cabannes, Elvis Dohmatob, and Alberto Bietti · 2023
Later among the works it cites.
On sparse modern hopfield model
Jerry Yao-Chieh Hu, Donglin Yang, Dennis Wu, Chenwei Xu, Bo-Yu Chen, and Han Liu · 2023
Later among the works it cites.
Generalized neural collapse for a large number of classes
Jiachen Jiang, Jinxin Zhou, Peng Wang, Qing Qu, Dustin Mixon, Chong You, and Zhihui Zhu · 2023
Later among the works it cites.
Building transformers from neurons and astrocytes
Leo Kozachkov, Ksenia V Kastanenka, and Dmitry Krotov · 2023
Later among the works it cites.
A new frontier for hopfield networks
Dmitry Krotov · 2023
Later among the works it cites.
Sparse modern hopfield networks
Andre Martins, Vlad Niculae, and Daniel C McNamee · 2023
Later among the works it cites.
Learning with partial forgetting in modern hopfield networks
Toshihiro Ota, Ikuro Sato, Rei Kawakami, Masayuki Tanaka, and Nakamasa Inoue · 2023
Later among the works it cites.
Scalable diffusion models with transformers
William Peebles and Saining Xie · 2023
Later among the works it cites.
End-to-end differentiable clustering with associative memories
Bishwajit Saha, Dmitry Krotov, Mohammed J Zaki, and Parikshit Ram · 2023
Later among the works it cites.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023
Later among the works it cites.
Improving text embeddings with large language models
Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei · 2023
Later among the works it cites.
Dnabert-2: Efficient foundation model and benchmark for multi-species genome
Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu · 2023
Later among the works it cites.
Birth of a transformer: A memory viewpoint
Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, and Leon Bottou · 2024
Closest in time.
Semantically-correlated memories in a dense associative model
Thomas F Burns · 2024
Closest in time.
Camelot: Towards large language models with training-free consolidated associative memory
Zexue He, Leonid Karlinsky, Donghyun Kim, Julian McAuley, Dmitry Krotov, and Rogerio Feris · 2024
Closest in time.
Energy-based hopfield boosting for out-of-distribution detection
Claus Hofmann, Simon Schmid, Bernhard Lehner, Daniel Klotz, and Sepp Hochreiter · 2024
Closest in time.
Exponential capacity of dense associative memories
Carlo Lucibello and Marc Mézard · 2024
Closest in time.
Sora: A video generative model based on transformer diffusion
OpenAI · 2024
Closest in time.
Movie gen: A cast of media foundation models
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al · 2024
Closest in time.
Sparse and structured hopfield networks
Saul Santos, Vlad Niculae, Daniel McNamee, and Andre FT Martins · 2024
Closest in time.
Chenwei Xu, Yu-Chao Huang, Jerry Yao-Chieh Hu, Weijian Li, Ammar Gilani, Hsi-Sheng Goan, and Han Liu · 2024
Closest in time.
Dnabert-s: Learning species-aware dna embedding with genome foundation models
Zhihan Zhou, Winmin Wu, Harrison Ho, Jiayi Wang, Lizhen Shi, Ramana V Davuluri, Zhong Wang, and Han Liu · 2024
Closest in time.