Memory Is All You Need: An Overview of Compute-in-Memory Architectures for Accelerating Large Language Model Inference · Around

Fetching the paper…

Memory Is All You Need: An Overview of Compute-in-Memory Architectures for Accelerating Large Language Model Inference · Around

Built on

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997
1997
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998
1998
Earlier work this paper cites.
M. Iyyer, J. Boyd-Graber, L. Claudino, R. Socher, and H. Daumé III, “A neural network for factoid question answering over paragraphs,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , 2014, pp. 633–644
2014
Earlier work this paper cites.
S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” Advances in neural information processing systems , vol. 28, 2015
2015
Earlier work this paper cites.
S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149 , 2015
Original
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015
Original
2015
Earlier work this paper cites.
S. Zhou, Y. Wu, Z. Ni, X. Zhou, H. Wen, and Y. Zou, “Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,” arXiv preprint arXiv:1606.06160 , 2016
Original
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 2704–2713
2018
Earlier work this paper cites.
R. Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepaper,” arXiv preprint arXiv:1806.08342 , 2018
Original
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.
A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Glue: A multi-task benchmark and analysis platform for natural language understanding,” arXiv preprint arXiv:1804.07461 , 2018
Original
2018
Earlier work this paper cites.
M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro, “Megatron-lm: Training multi-billion parameter language models using model parallelism,” arXiv preprint arXiv:1909.08053 , 2019
Original
2019
Earlier work this paper cites.
R. Sutton, “The bitter lesson,” Incomplete Ideas (blog) , vol. 13, no. 1, p. 38, 2019
2019
Earlier work this paper cites.
S. Zhang, G. L. Zhang, B. Li, H. H. Li, and U. Schlichtmann, “Aging-aware lifetime enhancement for memristor-based neuromorphic computing,” in 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE) . IEEE, 2019, pp. 1751–1756
2019
Earlier work this paper cites.
A. Ankit, I. E. Hajj, S. R. Chalamalasetti, G. Ndu, M. Foltin, R. S. Williams, P. Faraboschi, W.-m. W. Hwu, J. P. Strachan, K. Roy et al. , “Puma: A programmable ultra-efficient memristor-based accelerator for machine learning inference,” in Proceedings of the twenty-fourth international conference on architectural support for programming languages and operating systems , 2019, pp. 715–731
2019
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020
2020
Earlier work this paper cites.
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361 , 2020
Original
2020
Earlier work this paper cites.
C. E. Leiserson, N. C. Thompson, J. S. Emer, B. C. Kuszmaul, B. W. Lampson, D. Sanchez, and T. B. Schardl, “There’s plenty of room at the top: What will drive computer performance after moore’s law?” Science , vol. 368, no. 6495, p. eaam9744, 2020
2020
Earlier work this paper cites.
A. Sebastian, M. Le Gallo, R. Khaddam-Aljameh, and E. Eleftheriou, “Memory devices and applications for in-memory computing,” Nature nanotechnology , vol. 15, no. 7, pp. 529–544, 2020
2020
Earlier work this paper cites.

Similar

X. Yang, B. Yan, H. Li, and Y. Chen, “Retransformer: Reram-based processing-in-memory architecture for transformer acceleration,” in Proceedings of the 39th International Conference on Computer-Aided Design , 2020, pp. 1–9
2020
Cited alongside, same era.
H. Guo, L. Peng, J. Zhang, Q. Chen, and T. D. LeCompte, “Att: A fault-tolerant reram accelerator for attention-based neural networks,” in 2020 IEEE 38th International Conference on Computer Design (ICCD) . IEEE, 2020, pp. 213–221
2020
Cited alongside, same era.
A. F. Laguna, A. Kazemi, M. Niemier, and X. S. Hu, “In-memory computing based accelerator for transformer networks for long sequences,” in 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE) . IEEE, 2021, pp. 1839–1844
2021
Cited alongside, same era.
M. Kang, H. Shin, and L.-S. Kim, “A framework for accelerating transformer-based language model on reram-based architecture,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 41, no. 9, pp. 3026–3039, 2021

Then

Y. Wu, Z. Wang, and W. D. Lu, “Pim-gpt: A hybrid process-in-memory accelerator for autoregressive transformers,” arXiv preprint arXiv:2310.09385 , 2023
Original
2023
Later among the works it cites.
G. Burr, H. Tsai, W. Simon, I. Boybat, S. Ambrogio, C.-E. Ho, Z.-W. Liou, M. Rasch, J. Büchel, P. Narayanan et al. , “Design of analog-ai hardware accelerators for transformer-based language models,” in 2023 International Electron Devices Meeting (IEDM) . IEEE, 2023, pp. 1–4
2023
Later among the works it cites.
S. Sridharan, J. R. Stevens, K. Roy, and A. Raghunathan, “X-former: In-memory acceleration of transformers,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems , 2023
2023
Later among the works it cites.
Z. Yang, K. Liu, Y. Duan, M. Fan, Q. Zhang, and Z. Jin, “Three challenges in reram-based process-in-memory for neural network,” in 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS) . IEEE, 2023, pp. 1–5
2023

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…