Bitnet.cpp: Efficient Edge Inference for Ternary LLMs

Hellaswag: Can a machine really finish your sentence?

Original

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 1905

Earlier work this paper cites.

Pointer sentinel mixture models

Original

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016 · 2016

Earlier work this paper cites.

Multiplying matrices without multiplying

John Guttag Davis Blalock. 2021 · 2021

Earlier work this paper cites.

Winogrande: an adversarial winograd schema challenge at scale

Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021 · 2021

Earlier work this paper cites.

Flashattention: Fast and memory-efficient exact attention with io-awareness

Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022 · 2022

Earlier work this paper cites.

Gpt3.int8(): 8-bit matrix multiplication for transformers at scale

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022 · 2022

Earlier work this paper cites.

Quip: 2-bit quantization of large language models with guarantees

Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M De Sa. 2023 · 2023

Earlier work this paper cites.

Flashattention-2: Faster attention with better parallelism and work partitioning

Original

Tri Dao. 2023 · 2023

Earlier work this paper cites.

Spqr: A sparse-quantized representation for near-lossless llm weight compression

Original

Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. 2023 · 2023

Earlier work this paper cites.

Gptq: Accurate post-training quantization for generative pre-trained transformers

Original

Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023 · 2023

Earlier work this paper cites.

Deepgemm: Accelerated ultra low-precision inference on cpu architectures using lookup tables

Darshan C. Ganji, Saad Ashfaq, Ehsan Saboori, Sudhakar Sah, Saptarshi Mitra, MohammadHossein AskariHemmat, Alexander Hoffman, Ahmed Hassanien, and Mathieu Léonardon. 2023 · 2023

Earlier work this paper cites.

Efficient memory management for large language model serving with pagedattention

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023 · 2023

Cited alongside, same era.

Llm-qat: Data-free quantization aware training for large language models

Original

Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023 · 2023

Cited alongside, same era.

Look-up mai gemm: Increasing ai gemms performance by nearly 2.5x via msgemm

Original

Saeed Maleki. 2023 · 2023

Cited alongside, same era.

Lut-nn: Empower efficient neural network inference with centroid learning and table lookup

Xiaohu Tang, Yang Wang, Ting Cao, Li Lyna Zhang, Qi Chen, Deng Cai, Yunxin Liu, and Mao Yang. 2023 · 2023

Cited alongside, same era.

Bitnet: Scaling 1-bit transformers for large language models

The era of 1-bit llms: All large language models are in 1.58 bits

Original

Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. 2024 · 2024

Later among the works it cites.

1.58-bit llm: A new era of extreme quantization

Mohamed Mekkouri, Marc Sun, Leandro von Werra, and Thomas Wolf. 2024 · 2024

Later among the works it cites.

Lut tensor core: Lookup table enables efficient low-bit llm inference acceleration

Original

Zhiwen Mo, Lei Wang, Jianyu Wei, Zhichen Zeng, Shijie Cao, Lingxiao Ma, Naifeng Jing, Ting Cao, Jilong Xue, Fan Yang, and Mao Yang. 2024 · 2024

Later among the works it cites.

Lut-gemm: Quantized matrix multiplication based on luts for efficient inference in large-scale generative language models

Original

Gunho Park, Baeseong Park, Minsub Kim, Sungjae Lee, Jeonghoon Kim, Beomseok Kwon, Se Jung Kwon, Byeongwook Kim, Youngjoo Lee, and Dongsoo Lee. 2024 · 2024

Later among the works it cites.