Understand
The Deep Learning (DL) community sees many novel topologies published each year.
- Achieving high performance on each new topology remains challenging, as each requires some level of manual effort.
- This issue is compounded by the proliferation of frameworks and hardware platforms.
- The current approach, which we call "direct optimization", requires deep changes within each framework to improve the training performance for each hardware backend (CPUs, GPUs, FPGAs, ASICs) and requires $\mathcal{O}(fp)$ effort; where $f$ is the number of frameworks and $p$ is the number of platforms.
Built on
LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO’04) . Palo Alto, California
Chris Lattner and Vikram Adve. 2004 · 2004
Earlier work this paper cites.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015 · 2015
Earlier work this paper cites.
TVM: An End to End IR Stack for Deploying Deep Learning Workloads on Hardware Platforms
Tianqi Chen, Thierry Moreau, Ziheng Jiang, and Haichen Shen. 2017 · 2017
Earlier work this paper cites.
Similar
(Oct 2017)
Amazon Web Service AI team. 2017 · 2017
Cited alongside, same era.
DLVM: A modern compiler infrastructure for deep learning systems
Richard Wei, Vikram S. Adve, and Lane Schwartz. 2017 · 2017
Cited alongside, same era.
Apache MXNet
2017 · 2018
Cited alongside, same era.
Then
NVIDIA cuDNN
2018 · 2018
Closest in time.
XLA Overview
Google. 2017 · 2018
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…