Fetching the paper…
Reading the bibliography…
Transformers have attained superior performance in natural language processing and computer vision.
Some mathematical notes on three-mode factor analysis
L.R. Tucker · 1966
Earlier work this paper cites.
Tensor rank and the ill-posedness of the best low-rank approximation problem
Vin de Silva and Lek-Heng Lim · 2008
Earlier work this paper cites.
Tensor decompositions and applications
Tamara G Kolda and Brett W Bader · 2009
Earlier work this paper cites.
Tensor-train decomposition
Ivan V Oseledets · 2011
Earlier work this paper cites.
Nfea: Tensor toolbox for feature extraction and applications
A.H. NFEA Phan · 2011
Earlier work this paper cites.
Energy Delay Product , pp. 51–55
James H. Laros III, Kevin Pedretti, Suzanne M. Kelly, Wei Shu, Kurt Ferreira, John Vandyke, and Courtenay Vaughan · 2013
Earlier work this paper cites.
Tensor decompositions for learning latent variable models
Animashree Anandkumar et al · 2014
Earlier work this paper cites.
Tensorizing neural networks
Alexander Novikov, Dmitry Podoprikhin, Anton Osokin, , and Dmitry Vetrov · 2015
Earlier work this paper cites.
Tensor decomposition for signal processing and machine learning
Nicholas D. Sidiropoulos, Lieven De Lathauwer, Xiao Fu, Kejun Huang, Evangelos E. Papalexakis, and Christos Faloutsos · 2017
Earlier work this paper cites.
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
Patrick H. Chen, Si Si, Yang Li, et al · 2018
Earlier work this paper cites.
Deep contextualized word representations
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer · 2018
Earlier work this paper cites.
opt_einsum - a python package for optimizing contraction order for einsum-like expressions
Daniel G. A. Smith and Johnnie Gray · 2018
Cited alongside, same era.
Tensor decomposition for compressing recurrent neural network
Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura · 2018
Cited alongside, same era.
TIE: Energy-efficient Tensor Train-based Inference Engine for Deep Neural Network
Chunhua Deng, Fangxuan Sun, Xuehai Qian, et al · 2019
Cited alongside, same era.
BERT: pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Tensorly: Tensor learning in python
Jean Kossaifi, Yannis Panagakis, Anima Anandkumar, and Maja Pantic · 2019
Cited alongside, same era.
A Tensorized Transformer for Language Modeling
Xindian Ma, Peng Zhang, Shuai Zhang, et al · 2019
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, et al · 2020
Later among the works it cites.
Once for all: Train one network and specialize it for efficient deployment
Han Cai, Chuang Gan, and Song Han · 2020
Later among the works it cites.
Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination
Cole Hawkins, Xing Liu, and Zheng Zhang · 2020
Later among the works it cites.
Tensorized Embedding Layers
Oleksii Hrinchuk, Valentin Khrulkov, Leyla Mirvakhabova, et al · 2020
Later among the works it cites.
Tensor regression networks
Jean Kossaifi, Zachary C. Lipton, Arinbjörn Kolbeinsson, Aran Khanna, Tommaso Furlanello, and Anima Anandkumar · 2020
Later among the works it cites.
Towards memory-efficient neural networks via multi-level in situ generation
Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen, and David Z. Pan · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Timeloop: A systematic approach to dnn accelerator evaluation
Angshuman Parashar, Priyanka Raina, Yakun Sophia Shao, Yu-Hsin Chen, Victor A. Ying, Anurag Mukkara, Rangharajan Venkatesan, Brucek Khailany, Stephen W. Keckler, and Joel Emer · 2019
Cited alongside, same era.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, et al · 2019
Cited alongside, same era.
Simba: Scaling deep-learning inference with multi-chip-module-based architecture
Yakun Sophia Shao, Jason Clemons, Rangharajan Venkatesan, Brian Zimmer, Matthew Fojtik, Nan Jiang, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, Stephen G. Tell, Yanqing Zhang, William J. Dally, Joel Emer, C. Thomas Gray, Brucek Khailany, and Stephen W. Keckler · 2019
Cited alongside, same era.
FBNet: Hardware-aware Efficient Convnet Design via Differentiable Neural Architecture Search
Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, et al · 2019
Cited alongside, same era.
Universally slimmable networks and improved training techniques
Jiahui Yu and Thomas S. Huang · 2019
Cited alongside, same era.
Tt-rec: Tensor train compression for deep learning recommendation models
Chunxing Yin, Bilge Acun, Carole-Jean Wu, and Xing Liu
Cited in the paper.
Later among the works it cites.
ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques
Yuanxin Liu, Zheng Lin, and Fengcheng Yuan · 2021
Later among the works it cites.
Tensor methods in computer vision and deep learning
Yannis Panagakis, Jean Kossaifi, Grigorios G Chrysos, James Oldfield, Mihalis A Nicolaou, Anima Anandkumar, and Stefanos Zafeiriou · 2021
Later among the works it cites.
MiniViT: Compressing Vision Transformers with Weight Multiplexing
Jinnian Zhang, Houwen Peng, Kan Wu, et al · 2022
Closest in time.
Deeply Tensor Compressed Transformers for End-to-End Object Detection
Peining Zhen, Ziyang Gao, Tianshu Hou, et al · 2022
Closest in time.