Fetching the paper…
Reading the bibliography…
Tensor kernels in machine learning (ML) often correspond to pure mathematical expressions, making term rewriting an attractive strategy for optimization and mapping to specialized hardware accelerators.
Relay: A High-Level IR for Deep Learning
Jared Roesch, Steven Lyubomirsky, Marisa Kirisame, Josh Pollock, Logan Weber, Ziheng Jiang, Tianqi Chen, Thierry Moreau, and Zachary Tatlock. 2019 · 1904
Earlier work this paper cites.
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019 · 1912
Earlier work this paper cites.
SPORES: Sum-Product Optimization via Relational Equality Saturation for Large Scale Linear Algebra
Yisu Remy Wang, Shana Hutchison, Dan Suciu, Bill Howe, and Jonathan Leang. 2020 · 1932
Earlier work this paper cites.
An Approach for Exploring Code Improving Transformations
Deborah L. Whitfield and Mary Lou Soffa. 1997 · 1997
Earlier work this paper cites.
Term Rewriting and All That
Franz Baader and Tobias Nipkow. 1998 · 1998
Earlier work this paper cites.
MLIR: A Compiler Infrastructure for the End of Moore’s Law
Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Tatiana Shpeisman, Nicolas Vasilache, and Oleksandr Zinenko. 2020 · 2002
Earlier work this paper cites.
Fireiron: A Scheduling Language for High-Performance Linear Algebra on GPUs
Bastian Hagedorn, Archibald Samuel Elliott, Henrik Barthels, Rastislav Bodik, and Vinod Grover. 2020a · 2003
Earlier work this paper cites.
High Performance Convolutional Neural Networks for Document Processing. In Tenth International Workshop on Frontiers in Handwriting Recognition , Guy Lorette (Ed.). Université de Rennes 1, Suvisoft, La Baule (France)
Kumar Chellapilla, Sidd Puri, and Patrice Simard. 2006 · 2006
Earlier work this paper cites.
Equality Saturation: A New Approach to Optimization. In Proceedings of the 36th Annual ACM Symposium on Principles of Programming Languages (POPL ’09) . 264–276
Ross Tate, Michael Stepp, Zachary Tatlock, and Sorin Lerner. 2009 · 2009
Earlier work this paper cites.
Accelerating Haskell array codes with multicore GPUs. In DAMP ’11: The 6th workshop on Declarative Aspects of Multicore Programming . ACM
Manuel M T Chakravarty, Gabriele Keller, Sean Lee, Trevor L. McDonell, and Vinod Grover. 2011 · 2011
Earlier work this paper cites.
Bitcoin’s future carbon footprint
Shize Qin, Lena Klaaßen, Ulrich Gallersdörfer, Christian Stoll, and Da Zhang. 2020 · 2011
Earlier work this paper cites.
Learning to Schedule Halide Pipelines for the GPU
Luke Anderson, Andrew Adams, Karima Ma, Tzu-Mao Li, and Jonathan Ragan-Kelley. 2020 · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012 · 2012
Earlier work this paper cites.
Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines
Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. 2013b · 2013
Earlier work this paper cites.
The potential of polyhedral optimization: An empirical study. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) . 508–518
A. Simbürger, S. Apel, A. Größlinger, and C. Lengauer. 2013 · 2013
Cited alongside, same era.
Learning Semantic Image Representations at a Large Scale
Yangqing Jia. 2014 · 2014
Cited alongside, same era.
Automatically Improving Accuracy for Floating Point Expressions
Pavel Panchekha, Alex Sanchez-Stern, James R. Wilcox, and Zachary Tatlock. 2015 · 2015
Cited alongside, same era.
TensorFlow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) . 265–283
Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016 · 2016
Cited alongside, same era.
A Hardware-Software Blueprint for Flexible Deep Learning Specialization
T. Moreau, T. Chen, L. Vega, J. Roesch, L. Zheng, E. Yan, J. Fromm, Z. Jiang, L. Ceze, C. Guestrin, and A. Krishnamurthy. 2019 · 2019
Later among the works it cites.
Survey and Benchmarking of Machine Learning Accelerators
Albert Reuther, Peter Michaleas, Michael Jones, Vijay Gadepally, Siddharth Samsi, and Jeremy Kepner. 2019 · 2019
Later among the works it cites.
How to Bring Your Own Codegen to TVM
Zhi Chen and Cody Yu. 2020 · 2020
Later among the works it cites.
Achieving High-Performance the Functional Way: A Functional Pearl on Expressing High-Performance Optimizations as Rewrite Strategies
Bastian Hagedorn, Johannes Lenfers, Thomas Kundefinedhler, Xueying Qin, Sergei Gorlatch, and Michel Steuwer. 2020b · 2020
Later among the works it cites.
Synthesizing Structured CAD Models with Equality Saturation and Inverse Transformations. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (London, UK) (PLDI 2020) . Association for Computing Machinery, New York, NY, USA, 31–44
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Chen, Yu-Hsin and Krishna, Tushar and Emer, Joel and Sze, Vivienne. 2016 · 2016
Cited alongside, same era.
In-Datacenter Performance Analysis of a Tensor Processing Unit
Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Daniel Killebrew, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, and Doe Hyun Yoon. 2017 · 2017
Cited alongside, same era.
The Tensor Algebra Compiler
Fredrik Kjolstad, Shoaib Kamil, Stephen Chou, David Lugato, and Saman Amarasinghe. 2017 · 2017
Cited alongside, same era.
Lift: A Functional Data-Parallel IR for High-Performance GPU Code Generation. In Proceedings of the 2017 International Symposium on Code Generation and Optimization (Austin, USA) (CGO ’17) . IEEE Press, 74–85
Michel Steuwer, Toomas Remmelg, and Christophe Dubach. 2017 · 2017
Cited alongside, same era.
Benchmarking Implementations of Term Rewriting and Pattern Matching in Algebraic, Functional, and Object-Oriented Languages - The 4th Rewrite Engines Competition. In Proceedings of the 12th International Workshop on Rewriting Logic and its Applications (WRLA’18) . Thessaloniki, Greece
Hubert Garavel, Mohammad-Ali Tabikh, and Imad-Seddik Arrada. 2018 · 2018
Cited alongside, same era.
NVIDIA Tensor Core Programmability, Performance & Precision
Stefano Markidis, Steven Wei Der Chien, Erwin Laure, Ivy Bo Peng, and Jeffrey S. Vetter. 2018 · 2018
Cited alongside, same era.
The NVIDIA Deep Learning Accelerator (NVDLA)
Nvidia. 2018 · 2018
Cited alongside, same era.
Tensor comprehensions: Framework-agnostic high-performance machine learning abstractions
Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. 2018 · 2018
Cited alongside, same era.
Chandrakana Nandi, Max Willsey, Adam Anderson, James R. Wilcox, Eva Darulova, Dan Grossman, and Zachary Tatlock. 2020 · 2020
Later among the works it cites.
Verifying and Improving Halide’s Term Rewriting System with Program Synthesis
Julie L. Newcomb, Andrew Adams, Steven Johnson, Rastislav Bodik, and Shoaib Kamil. 2020 · 2020
Later among the works it cites.
Convolutional Layers User Guide
NVIDIA. 2020 · 2020
Later among the works it cites.
Interstellar: Using Halide’s Scheduling Language to Analyze DNN Accelerators. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzerland) (ASPLOS ’20) . Association for Computing Machinery, New York, NY, USA, 369–383
Xuan Yang, Mingyu Gao, Qiaoyi Liu, Jeff Setter, Jing Pu, Ankita Nayak, Steven Bell, Kaidi Cao, Heonjae Ha, Priyanka Raina, Christos Kozyrakis, and Mark Horowitz. 2020 · 2020
Later among the works it cites.
Ansor: Generating High-Performance Tensor Programs for Deep Learning. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20) . USENIX Association, Banff, Canada, 863–879
Lianmin Zheng, Chengfan Jia, Minmin Sun, Zhao Wu, Cody Hao Yu, Ameer Haj-Ali, Yida Wang, Jun Yang, Danyang Zhuo, Koushik Sen, Joseph E. Gonzalez, and Ion Stoica. 2020 · 2020
Later among the works it cites.
Kernel Operations on the GPU, with Autodiff, without Memory Overflows
Benjamin Charlier, Jean Feydy, Joan Alexis Glaunès, François-David Collin, and Ghislain Durif. 2021 · 2021
Closest in time.
A High-Performance Sparse Tensor Algebra Compiler in Multi-Level IR
Ruiqin Tian, Luanzheng Guo, Jiajia Li, Bin Ren, and Gokcen Kestor. 2021 · 2021
Closest in time.
Vectorization for Digital Signal Processors via Equality Saturation
Alexa VanHattum, Rachit Nigam, Vincent T Lee, James Bornholt, and Adrian Sampson. 2021 · 2021
Closest in time.
egg: fast and extensible equality saturation
Max Willsey, Chandrakana Nandi, Yisu Remy Wang, Oliver Flatt, Zachary Tatlock, and Pavel Panchekha. 2021 · 2021
Closest in time.