Fetching the paper…
Reading the bibliography…
We propose a two-stage memory retrieval dynamics for modern Hopfield models, termed $\mathtt{U\text{-}Hop}$, with enhanced memory capacity.
Adaptively sparse transformers
Gonçalo M Correia, Vlad Niculae, and André FT Martins · 1909
Earlier work this paper cites.
Meta-learning deep energy-based memory models
Sergey Bartunov, Jack W Rae, Simon Osindero, and Timothy P Lillicrap · 1910
Earlier work this paper cites.
Non-holographic associative memory
David J Willshaw, O Peter Buneman, and Hugh Christopher Longuet-Higgins · 1969
Earlier work this paper cites.
Nonlinear programming: a unified approach , volume 52
Willard I Zangwill · 1969
Earlier work this paper cites.
Neural networks and physical systems with emergent collective computational abilities
John J Hopfield · 1982
Earlier work this paper cites.
Neurons with graded response have collective computational properties like those of two-state neurons
John J Hopfield · 1984
Earlier work this paper cites.
Sparse distributed memory
Pentti Kanerva · 1988
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Reformer: The efficient transformer
Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya · 2001
Earlier work this paper cites.
The concave-convex procedure (cccp)
Alan L Yuille and Anand Rangarajan · 2001
Earlier work this paper cites.
Low-rank bottleneck in multi-head attention models
Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank Reddi, and Sanjiv Kumar · 2002
Earlier work this paper cites.
A simple framework for contrastive learning of visual representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton · 2002
Earlier work this paper cites.
Kyungwoo Song, Yohan Jung, Dongjun Kim, and Il-Chul Moon · 2006
Earlier work this paper cites.
Modern hopfield networks and attention for immune repertoire classification
Michael Widrich, Bernhard Schäfl, Milena Pavlović, Hubert Ramsauer, Lukas Gruber, Markus Holzleitner, Johannes Brandstetter, Geir Kjetil Sandve, Victor Greiff, Sepp Hochreiter, et al · 2007
Earlier work this paper cites.
Large associative memory problem in neurobiology and machine learning
Dmitry Krotov and John J. Hopfield · 2008
Earlier work this paper cites.
Hopfield networks is all you need
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, et al · 2008
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
On the convergence of the concave-convex procedure
Bharath K Sriperumbudur and Gert RG Lanckriet · 2009
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2010
Cited alongside, same era.
Informer: Beyond efficient transformer for long sequence time-series forecasting
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang · 2012
Cited alongside, same era.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Cited alongside, same era.
Dense associative memory for pattern recognition
Dmitry Krotov and John J Hopfield · 2016
Cited alongside, same era.
From softmax to sparsemax: A sparse model of attention and multi-label classification
Quantizable transformers: Removing outliers by helping attention heads do nothing
Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort · 2023
Later among the works it cites.
Simplicial hopfield networks
Thomas F Burns and Tomoki Fukai · 2023
Later among the works it cites.
Yeqi Gao, Zhao Song, Weixin Wang, and Junze Yin · 2023
Later among the works it cites.
On sparse modern hopfield model
Jerry Yao-Chieh Hu, Donglin Yang, Dennis Wu, Chenwei Xu, Bo-Yu Chen, and Han Liu · 2023
Later among the works it cites.
Building transformers from neurons and astrocytes
Leo Kozachkov, Ksenia V Kastanenka, and Dmitry Krotov · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Andre Martins and Ramon Astudillo · 2016
Cited alongside, same era.
On a model of associative memory with huge storage capacity
Mete Demircigil, Judith Heusel, Matthias Löwe, Sven Upgang, and Franck Vermet · 2017
Cited alongside, same era.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Learning with kernels: support vector machines, regularization, optimization, and beyond
Bernhard Scholkopf and Alexander J Smola · 2018
Cited alongside, same era.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Tongzhou Wang and Phillip Isola · 2020
Cited alongside, same era.
Understanding and overcoming the challenges of efficient transformer quantization, 2021
Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort · 2021
Cited alongside, same era.
Associative memories via predictive coding
Tommaso Salvatori, Yuhang Song, Yujian Hong, Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bogacz, and Thomas Lukasiewicz · 2021
Cited alongside, same era.
Biological learning in key-value memory networks
Danil Tyulmankov, Ching Fang, Annapurna Vadaparty, and Guangyu Robert Yang · 2021
Cited alongside, same era.
Sparse modern hopfield networks
Andre Martins, Vlad Niculae, and Daniel C McNamee · 2023
Later among the works it cites.
Storage and learning phase transitions in the random-features hopfield model
Matteo Negri, Clarissa Lauditi, Gabriele Perugini, Carlo Lucibello, and Enrico Malatesta · 2023
Later among the works it cites.
Feature programming for multivariate time series prediction
Alex Reneau, Jerry Yao-Chieh Hu, Chenwei Xu, Weijian Li, Ammar Gilani, and Han Liu · 2023
Later among the works it cites.
Controlling the bifurcations of attractors in modern hopfield networks
Maria Yampolskaya and Pankaj Mehta · 2023
Later among the works it cites.
Conformal prediction for time series with modern hopfield networks
Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter · 2024
Closest in time.
Semantically-correlated memories in a dense associative model
Thomas F Burns · 2024
Closest in time.
Energy-based hopfield boosting for out-of-distribution detection
Claus Hofmann, Simon Schmid, Bernhard Lehner, Daniel Klotz, and Sepp Hochreiter · 2024
Closest in time.
Zhenyu Pan, Haozheng Luo, Manling Li, and Han Liu · 2024
Closest in time.
Bridging associative memory and probabilistic modeling
Rylan Schaeffer, Nika Zahedi, Mikail Khona, Dhruv Pai, Sang Truong, Yilun Du, Mitchell Ostrow, Sarthak Chandra, Andres Carranza, Ila Rani Fiete, et al · 2024
Closest in time.
Massive activations in large language models
Mingjie Sun, Xinlei Chen, J Zico Kolter, and Zhuang Liu · 2024
Closest in time.
STanhop: Sparse tandem hopfield model for memory-enhanced time series prediction
Dennis Wu, Jerry Yao-Chieh Hu, Weijian Li, Bo-Yu Chen, and Han Liu · 2024
Closest in time.
Chenwei Xu, Yu-Chao Huang, Jerry Yao-Chieh Hu, Weijian Li, Ammar Gilani, Hsi-Sheng Goan, and Han Liu · 2024
Closest in time.