DECA: A Near-Core LLM Decompression Accelerator Grounded on a 3D Roofline Model

Bitblade: Energy-efficient variable bit-precision hardware accelerator for quantized neural networks

Sungju Ryu, Hyungjun Kim, Wooseok Yi, Eunhwan Kim, Yulhwa Kim, Taesu Kim, and Jae-Joon Kim. 2022 · 1935

Earlier work this paper cites.

Optimal brain damage

Yann LeCun, John Denker, and Sara Solla. 1989 · 1989

Earlier work this paper cites.

Complexity-effective superscalar processors. In Proceedings of the 24th annual international symposium on Computer architecture . 206–218

Subbarao Palacharla, Norman P Jouppi, and James E Smith. 1997 · 1997

Earlier work this paper cites.

An Evaluation of High-Level Mechanistic Core Models

Trevor E. Carlson, Wim Heirman, Stijn Eyerman, Ibrahim Hur, and Lieven Eeckhout. 2014 · 2014

Earlier work this paper cites.

Modeling and design of high-radix on-chip crossbar switches. In Proceedings of the 9th International Symposium on Networks-on-Chip . 1–8

Cagla Cakir, Ron Ho, Jon Lexau, and Ken Mai. 2015 · 2015

Earlier work this paper cites.

Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding

Original

Song Han, Huizi Mao, and William J Dally. 2015 · 2015

Earlier work this paper cites.

Optimizing FPGA-based accelerator design for deep convolutional neural networks. In Proceedings of the 2015 ACM/SIGDA international symposium on field-programmable gate arrays . 161–170

Chen Zhang, Peng Li, Guangyu Sun, Yijin Guan, Bingjun Xiao, and Jason Cong. 2015 · 2015

Earlier work this paper cites.

EIE: Efficient inference engine on compressed deep neural network

Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A Horowitz, and William J Dally. 2016 · 2016

Earlier work this paper cites.

Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016 · 2016

Earlier work this paper cites.

LIBXSMM: accelerating small matrix multiplications by runtime code generation. In SC’16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 981–991

Alexander Heinecke, Greg Henry, Maxwell Hutchinson, and Hans Pabst. 2016 · 2016

Earlier work this paper cites.

Cambricon-X: An accelerator for sparse neural networks. In 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 1–12

Shijin Zhang, Zidong Du, Lei Zhang, Huiying Lan, Shaoli Liu, Ling Li, Qi Guo, Tianshi Chen, and Yunji Chen. 2016 · 2016

Earlier work this paper cites.

CACTI 7: New tools for interconnect exploration in innovative off-chip memories

Rajeev Balasubramonian, Andrew B Kahng, Naveen Muralimanohar, Ali Shafiee, and Vaishnav Srinivas. 2017 · 2017

Earlier work this paper cites.

SCNN: An accelerator for compressed-sparse convolutional neural networks

Angshuman Parashar, Minsoo Rhu, Anurag Mukkara, Antonio Puglielli, Rangharajan Venkatesan, Brucek Khailany, Joel Emer, Stephen W Keckler, and William J Dally. 2017 · 2017

Earlier work this paper cites.

Scaling equations for the accurate prediction of CMOS device performance from 180 nm to 7 nm

Aaron Stillmaker and Bevan Baas. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Nvidia tensor core programmability, performance & precision. In 2018 IEEE international parallel and distributed processing symposium workshops (IPDPSW) . IEEE, 522–531

Stefano Markidis, Steven Wei Der Chien, Erwin Laure, Ivy Bo Peng, and Jeffrey S Vetter. 2018 · 2018

Earlier work this paper cites.

SparTen: A sparse tensor accelerator for convolutional neural networks. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture . 151–165

Ashish Gondimalla, Noah Chesnut, Mithuna Thottethodi, and TN Vijaykumar. 2019 · 2019

Earlier work this paper cites.

ExTensor: An Accelerator for Sparse Tensor Algebra. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (Columbus, OH, USA) (MICRO ’52) . Association for Computing Machinery, New York, NY, USA, 319–333

Kartik Hegde, Hadi Asghari-Moghaddam, Michael Pellauer, Neal Crago, Aamer Jaleel, Edgar Solomonik, Joel Emer, and Christopher W. Fletcher. 2019 · 2019

Earlier work this paper cites.

A study of BFLOAT16 for deep learning training

Original

Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey. 2019 · 2019

Earlier work this paper cites.

An efficient hardware accelerator for sparse convolutional neural networks on FPGAs. In 2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) . IEEE, 17–25

Liqiang Lu, Jiaming Xie, Ruirui Huang, Jiansong Zhang, Wei Lin, and Yun Liang. 2019 · 2019

Earlier work this paper cites.

New flexible multiple-precision multiply-accumulate unit for deep neural network training and inference

Hao Zhang, Dongdong Chen, and Seok-Bum Ko. 2019 · 2019

Earlier work this paper cites.

What is the state of neural network pruning?

Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. 2020 · 2020

Earlier work this paper cites.

Model compression and hardware acceleration for neural networks: A comprehensive survey

Lei Deng, Guoqi Li, Song Han, Luping Shi, and Yuan Xie. 2020 · 2020

Earlier work this paper cites.

Save: Sparsity-aware vector engine for accelerating dnn training and inference on CPUs. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 796–810

Zhangxiaowen Gong, Houxiang Ji, Christopher W Fletcher, Christopher J Hughes, Sara Baghsorkhi, and Josep Torrellas. 2020 · 2020

Earlier work this paper cites.

Aˆ 3: Accelerating attention mechanisms in neural networks with approximation. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) . IEEE, 328–341

Tae Jun Ham, Sung Jun Jung, Seonghak Kim, Young H. Oh, Yeonhong Park, Yoonho Song, Jung-Hun Park, Sanghee Lee, Kyoung Park, Jae W. Lee, and Deog-Kyoon Jeong. 2020 · 2020

Earlier work this paper cites.

Tensaurus: A Versatile Accelerator for Mixed Sparse-Dense Tensor Computations. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) . 689–702

Nitish Srivastava, Hanchen Jin, Shaden Smith, Hongbo Rong, David Albonesi, and Zhiru Zhang. 2020 · 2020

Earlier work this paper cites.

Matrix-multiply Assist Best Practices Guide

Puneeth Bhat, José Moreira, and Satish Kumar Sadasivam. 2021 · 2021

Earlier work this paper cites.

Next-Gen Intel Xeon CPU-Sapphire Rapids. In Hot Chips , Vol. 33

Arijit Biswas and Sailesh Kottapalli. 2021 · 2021

Earlier work this paper cites.

Tensor processing primitives: A programming abstraction for efficiency and portability in deep learning workloads. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis . 1–14

Evangelos Georganas, Dhiraj Kalamkar, Sasikanth Avancha, Menachem Adelman, Cristina Anderson, Alexander Breuer, Jeremy Bruestle, Narendra Chaudhary, Abhisek Kundu, Denise Kutnick, Frank Laub, Vasimuddin Md, Sanchit Misra, Ramanarayan Mohanty, Hans Pabst, Barukh Ziv, and Alexander Heinecke. 2021 · 2021

Earlier work this paper cites.

ELSA: Hardware-software co-design for efficient, lightweight self-attention mechanism in neural networks. In 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) . IEEE, 692–705

Tae Jun Ham, Yejin Lee, Seong Hoon Seo, Soosung Kim, Hyunji Choi, Sung Jun Jung, and Jae W Lee. 2021 · 2021

Cited alongside, same era.

Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks

Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. 2021 · 2021

Cited alongside, same era.

Rasa: Efficient register-aware systolic array matrix engine for cpu. In 2021 58th ACM/IEEE Design Automation Conference (DAC) . IEEE, 253–258

Geonhwa Jeong, Eric Qin, Ananda Samajdar, Christopher J Hughes, Sreenivas Subramoney, Hyesoon Kim, and Tushar Krishna. 2021 · 2021

Cited alongside, same era.

Pruning and quantization for deep neural network acceleration: A survey

Tailin Liang, John Glossner, Lei Wang, Shaobo Shi, and Xiaotong Zhang. 2021 · 2021

Cited alongside, same era.

Splitwise: Efficient generative LLM inference using phase splitting

Original

Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Aashaka Shah, Saeed Maleki, and Ricardo Bianchini. 2023 · 2023

Later among the works it cites.

A Tensor Marshaling Unit for Sparse Tensor Algebra on General-Purpose Processors. In Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture . 1332–1346

Marco Siracusa, Víctor Soria-Pardos, Francesco Sgherzi, Joshua Randall, Douglas J Joseph, Miquel Moretó Planas, and Adrià Armejach. 2023 · 2023

Later among the works it cites.

The synergy of speculative decoding and batching in serving large language models

Original

Qidong Su, Christina Giannoula, and Gennady Pekhimenko. 2023 · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023 · 2023