Fetching the paper…
Reading the bibliography…
As deep learning models continue to increase in size, the memory requirements for training have surged.
J. Bruno and R. Sethi, “Code generation for a one-register machine,” J. ACM
1976
Earlier work this paper cites.
M. Garey, D. Johnson, and L. Stockmeyer, “Some simplified np-complete graph problems,” Theoretical Computer Science
1976
Earlier work this paper cites.
D. Bernstein, M. Rodeh, and I. Gertner, “On the complexity of scheduling problems for parallel/pipelined machines,” IEEE Transactions on Computers
1989
Earlier work this paper cites.
A. Sllame and V. Drabek, “An efficient list-based scheduling algorithm for high-level synthesis,” in Proceedings Euromicro Symposium on Digital System Design. Architectures, Methods and Tools
2002
Earlier work this paper cites.
S.-I. Han, X. Guerin, S.-I. Chae, and A. A. Jerraya, “Buffer memory optimization for video codec application modeled in simulink,” in Proceedings of the 43rd annual Design Automation Conference
2006
Earlier work this paper cites.
G. Huang, “Analysis of solution methods for interval linear programming,” Journal of Environmental Informatics
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems
2012
Earlier work this paper cites.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2014
Earlier work this paper cites.
2014
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR
2014
Earlier work this paper cites.
2014
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2015
Earlier work this paper cites.
H. Ashayerinasab, H. Mishmast Nehi, and M. Allahdadi, “Overview of solution methods for solving interval linear programming and new method,” pp. 1–5, 09 2015
2015
Earlier work this paper cites.
2016
Earlier work this paper cites.
2016
Earlier work this paper cites.
S. Ruder, “An overview of gradient descent optimization algorithms,” CoRR
2016
Earlier work this paper cites.
PhD thesis, 12 2017
A. Neuenfeldt Júnior, The Two-Dimensional Rectangular Strip Packing Problem · 2017
Earlier work this paper cites.
C. Meng, M. Sun, J. Yang, M. Qiu, and Y. Gu, “Training deeper models by gpu memory optimization on tensorflow,” 2017
2017
Earlier work this paper cites.
2017
Earlier work this paper cites.
2017
Earlier work this paper cites.
Z. Jia, M. Zaharia, and A. Aiken, “Beyond data and model parallelism for deep neural networks,” CoRR
2018
Cited alongside, same era.
J. Feng and D. Huang, “Cutting down training memory by re-fowarding,” ArXiv
2018
Cited alongside, same era.
L. Wang, J. Ye, Y. Zhao, W. Wu, A. Li, S. L. Song, Z. Xu, and T. Kraska, “Superneurons: dynamic gpu memory management for training deep neural networks,” Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
2018
Cited alongside, same era.
2018
Cited alongside, same era.
2019
Later among the works it cites.
X. Peng, X. Shi, H. Dai, H. Jin, W. Ma, Q. Xiong, F. Yang, and X. Qian, “Capuchin: Tensor-based gpu memory management for deep learning,” in Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems
2020
Later among the works it cites.
2020
Later among the works it cites.
Y. Pisarchyk and J. Lee, “Efficient memory management for deep neural net inference,” ArXiv
2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A. Jain, A. Phanishayee, J. Mars, L. Tang, and G. Pekhimenko, “Gist: Efficient data encoding for deep neural network training,” in Proceedings of the 45th Annual International Symposium on Computer Architecture
2018
Cited alongside, same era.
T. Chen, T. Moreau, Z. Jiang, L. Zheng, E. Yan, H. Shen, M. Cowan, L. Wang, Y. Hu, L. Ceze, C. Guestrin, and A. Krishnamurthy, “TVM: An automated End-to-End optimizing compiler for deep learning,” in 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)
2018
Cited alongside, same era.
2018
Cited alongside, same era.
2018
Cited alongside, same era.
2018
Cited alongside, same era.
A. Brutzkus and A. Globerson, “Why do larger models generalize better? A theoretical perspective via the XOR problem,” in Proceedings of the 36th International Conference on Machine Learning
2019
Cited alongside, same era.
2019
Cited alongside, same era.
2019
Cited alongside, same era.
H. M. Nehi, H. A. Ashayerinasab, and M. Allahdadi, “Solving methods for interval linear programming problem: a review and an improved method,” Operational Research
2020
Later among the works it cites.
2020
Later among the works it cites.
2020
Later among the works it cites.
A. Sabne, “Xla : Compiling machine learning for peak performance,” 2020
2020
Later among the works it cites.
2020
Later among the works it cites.
2021
Later among the works it cites.
J. Ren, S. Rajbhandari, R. Y. Aminabadi, O. Ruwase, S. Yang, M. Zhang, D. Li, and Y. He, “Zero-offload: Democratizing billion-scale model training,” in USENIX Annual Technical Conference
2021
Later among the works it cites.
J. Bae, J. Lee, Y. Jin, S. Son, S. Kim, H. Jang, T. J. Ham, and J. W. Lee, “Flashneuron: Ssd-enabled large-batch training of very deep neural networks.,” in FAST
2021
Later among the works it cites.
S. Jin, C. Zhang, X. Jiang, Y. Feng, H. Guan, G. Li, S. L. Song, and D. Tao, “Comet: A novel memory-efficient deep learning training framework by using error-bounded lossy compression,” Proc. VLDB Endow
2021
Later among the works it cites.
Y. Li, A. Phanishayee, D. G. Murray, J. Tarnawski, and N. S. Kim, “Harmony: Overcoming the hurdles of gpu memory capacity to train massive dnn models on commodity servers,” Proc. VLDB Endow
2022
Later among the works it cites.
X. Jia, L. Jiang, A. Wang, W. Xiao, Z. Shi, J. Zhang, X. Li, L. Chen, Y. Li, Z. Zheng, et al
2022
Later among the works it cites.
M. Maas, U. Beaugnon, A. Chauhan, and B. Ilbeyi, “Telamalloc: Efficient on-chip memory allocation for production machine learning accelerators,” pp. 123–137, 12 2022
2022
Later among the works it cites.
2022
Later among the works it cites.
M. Levental, “Memory planning for deep neural networks,” ArXiv
2022
Later among the works it cites.
2023
Closest in time.
Steiner, Benoit, Elhoushi, Mostafa, Kahn, Jacob, Hegarty, and James, “Model: Memory optimizations for deep learning,” Accepted in International Conference on Machine Learning, ICML 2023
2023
Closest in time.