LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901

Earlier work this paper cites.

BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

Original

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019 · 1905

Earlier work this paper cites.

HellaSwag: Can a Machine Really Finish Your Sentence?

Original

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 1905

Earlier work this paper cites.

WinoGrande: An Adversarial Winograd Schema Challenge at Scale

Original

Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019 · 1907

Earlier work this paper cites.

PIQA: Reasoning about Physical Commonsense in Natural Language

Original

Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2019 · 1911

Earlier work this paper cites.

BitBlade: Energy-efficient variable bit-precision hardware accelerator for quantized neural networks

Sungju Ryu, Hyungjun Kim, Wooseok Yi, Eunhwan Kim, Yulhwa Kim, Taesu Kim, and Jae-Joon Kim. 2022 · 1935

Earlier work this paper cites.

Measuring Massive Multitask Language Understanding

Original

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2009

Earlier work this paper cites.

Stripes: Bit-serial deep neural network computing. In 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 1–12

Patrick Judd, Jorge Albericio, Tayler Hetherington, Tor M Aamodt, and Andreas Moshovos. 2016 · 2016

Earlier work this paper cites.

Pointer Sentinel Mixture Models

Original

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016 · 2016

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

TVM: end-to-end optimization stack for deep learning

Original

Tianqi Chen, Thierry Moreau, Ziheng Jiang, Haichen Shen, Eddie Q Yan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018 · 2018

Earlier work this paper cites.

UNPU: An Energy-Efficient Deep Neural Network Accelerator With Fully Variable Weight Bit Precision

Jinmook Lee, Changhyeon Kim, Sanghoon Kang, Dongjoo Shin, Sangyeob Kim, and Hoi-Jun Yoo. 2019 · 2018

Earlier work this paper cites.

Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering

Original

Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018 · 2018

Earlier work this paper cites.

Design Compiler User Guide

Synopsys Inc. 2018 · 2018

Earlier work this paper cites.

Timeloop: A systematic approach to dnn accelerator evaluation. In 2019 IEEE international symposium on performance analysis of systems and software (ISPASS) . IEEE, 304–315

Angshuman Parashar, Priyanka Raina, Yakun Sophia Shao, Yu-Hsin Chen, Victor A Ying, Anurag Mukkara, Rangharajan Venkatesan, Brucek Khailany, Stephen W Keckler, and Joel Emer. 2019 · 2019

Earlier work this paper cites.

Sparse tensor core: Algorithm and hardware co-design for vector-wise sparse neural networks on modern gpus. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture . 359–371

Maohua Zhu, Tao Zhang, Zhenyu Gu, and Yuan Xie. 2019 · 2019

Earlier work this paper cites.

Biqgemm: matrix multiplication with lookup table for binary-coding-based quantized dnns. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 1–14

Yongkweon Jeon, Baeseong Park, Se Jung Kwon, Byeongwook Kim, Jeongin Yun, and Dongsoo Lee. 2020 · 2020

Earlier work this paper cites.

Scaling laws for neural language models

Original

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020 · 2020

Earlier work this paper cites.

Accel-Sim: An extensible simulation framework for validated GPU modeling. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) . IEEE, 473–486

Mahmoud Khairy, Zhesheng Shen, Tor M Aamodt, and Timothy G Rogers. 2020 · 2020

Earlier work this paper cites.

Maestro: A data-centric approach to understand reuse, performance, and hardware cost of dnn mappings

Hyoukjun Kwon, Prasanth Chatarasi, Vivek Sarkar, Tushar Krishna, Michael Pellauer, and Angshuman Parashar. 2020 · 2020

Earlier work this paper cites.

GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE

Ali Hadi Zadeh, Isak Edo, Omar Mohamed Awad, and Andreas Moshovos. 2020 · 2020

Earlier work this paper cites.

Ansor: Generating High-Performance Tensor Programs for Deep Learning. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20) . USENIX Association, 863–879

Lianmin Zheng, Chengfan Jia, Minmin Sun, Zhao Wu, Cody Hao Yu, Ameer Haj-Ali, Yida Wang, Jun Yang, Danyang Zhuo, Koushik Sen, Joseph E. Gonzalez, and Ion Stoica. 2020 · 2020

Earlier work this paper cites.

Nvidia a100 tensor core gpu: Performance and innovation

Jack Choquette, Wishwesh Gandhi, Olivier Giroux, Nick Stam, and Ronny Krashinsky. 2021 · 2021

Earlier work this paper cites.

Need for speed: Experiences building a trustworthy system-level gpu simulator. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . IEEE, 868–880

Oreste Villa, Daniel Lustig, Zi Yan, Evgeny Bolotin, Yaosheng Fu, Niladrish Chatterjee, Nan Jiang, and David Nellans. 2021 · 2021

Earlier work this paper cites.

Dual-side sparse tensor core. In 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) . IEEE, 1083–1095

Yang Wang, Chen Zhang, Zhiqiang Xie, Cong Guo, Yunxin Liu, and Jingwen Leng. 2021 · 2021

Earlier work this paper cites.

Fusekna: Fused kernel convolution based accelerator for deep neural networks. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . IEEE, 894–907

Jianxun Yang, Zhao Zhang, Zhuangzhi Liu, Jing Zhou, Leibo Liu, Shaojun Wei, and Shouyi Yin. 2021 · 2021

Earlier work this paper cites.

Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022 · 2022

Earlier work this paper cites.

Gptq: Accurate post-training quantization for generative pre-trained transformers

Original

Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022 · 2022

Earlier work this paper cites.

Ant: Exploiting adaptive numerical data type for low-bit deep neural network quantization. In 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 1414–1433

Cong Guo, Chen Zhang, Jingwen Leng, Zihan Liu, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. 2022 · 2022

Cited alongside, same era.

Training compute-optimal large language models

Original

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al · 2022

Cited alongside, same era.

Fp8 quantization: The power of the exponent

Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort. 2022 · 2022

Cited alongside, same era.

Fp8 formats for deep learning

Original

Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, et al · 2022

Cited alongside, same era.

Bitnet: Scaling 1-bit transformers for large language models

Original

Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. 2023 · 2023

Later among the works it cites.

Training transformers with 4-bit integers

Haocheng Xi, Changhao Li, Jianfei Chen, and Jun Zhu. 2023 · 2023

Later among the works it cites.

Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning . PMLR, 38087–38099

Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023 · 2023

Later among the works it cites.

Integer or floating point? new outlooks for low-bit quantization on large language models

Original

Yijia Zhang, Lingran Zhao, Shijie Cao, Wenqiang Wang, Ting Cao, Fan Yang, Mao Yang, Shanghang Zhang, and Ningyi Xu. 2023 · 2023

Later among the works it cites.