2021

RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference

Wilkening, Mark, Gupta, Udit, Hsia, Samuel et al.

Understand

Neural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment.

State-of-the-art models comprise large embedding tables that have billions of parameters requiring large memory capacities.
Unfortunately, large and fast DRAM-based memories levy high infrastructure costs.
Conventional SSD-based storage solutions offer an order of magnitude larger capacity, but have worse read latency and bandwidth, degrading inference performance.

Built on

Deep Learning Recommendation Model for Personalization and Recommendation Systems
Original
Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, and Misha Smelyanskiy. 2019 · 1906
Earlier work this paper cites.
The Design and Implementation of a Log-Structured File System
Mendel Rosenblum and John K. Ousterhout. 1992 · 1992
Earlier work this paper cites.
Design Tradeoffs for SSD Performance. In USENIX 2008 Annual Technical Conference (Boston, Massachusetts) (ATC’08) . USENIX Association, USA, 57–70
Nitin Agrawal, Vijayan Prabhakaran, Ted Wobber, John D. Davis, Mark Manasse, and Rina Panigrahy. 2008 · 2008
Earlier work this paper cites.
Rethinking flash in the data center
David G Andersen and Steven Swanson. 2010 · 2010
Earlier work this paper cites.
Understanding Capacity-Driven Scale-Out Neural Recommendation Inference
Original
Michael Lui, Yavuz Yetim, Özgür Özkan, Zhuoran Zhao, Shin-Yeh Tsai, Carole-Jean Wu, and Mark Hempstead. 2020 · 2011
Earlier work this paper cites.
Query Processing on Smart SSDs: Opportunities and Challenges. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (New York, New York, USA) (SIGMOD ’13) . Association for Computing Machinery, New York, NY, USA, 1221–1230
Jaeyoung Do, Yang-Suk Kee, Jignesh M. Patel, Chanik Park, Kwanghyun Park, and David J. DeWitt. 2013 · 2013
Earlier work this paper cites.
Active Flash: Towards Energy-efficient, In-situ Data Analytics on Extreme-scale Machines. In Proceedings of the 11th USENIX Conference on File and Storage Technologies (San Jose, CA) (FAST’13) . USENIX Association, Berkeley, CA, USA, 119–132
Devesh Tiwari, Simona Boboila, Sudharshan S. Vazhkudai, Youngjae Kim, Xiaosong Ma, Peter J. Desnoyers, and Yan Solihin. 2013 · 2013
Earlier work this paper cites.
SDF: software-defined flash for web-scale internet storage systems. In Proceedings of the 19th international conference on Architectural support for programming languages and operating systems . 471–484
Jian Ouyang, Shiding Lin, Song Jiang, Zhenyu Hou, Yong Wang, and Yuanzheng Wang. 2014 · 2014
Earlier work this paper cites.
Willow: A User-programmable SSD. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (Broomfield, CO) (OSDI’14) . USENIX Association, Berkeley, CA, USA, 67–80
Sudharsan Seshadri, Mark Gahagan, Sundaram Bhaskaran, Trevor Bunker, Arup De, Yanqin Jin, Yang Liu, and Steven Swanson. 2014 · 2014
Earlier work this paper cites.
An Efficient Design and Implementation of LSM-tree Based Key-value Store on Open-channel SSD. In Proceedings of the Ninth European Conference on Computer Systems (Amsterdam, The Netherlands) (EuroSys ’14) . ACM, New York, NY, USA, Article 16, 14 pages
Peng Wang, Guangyu Sun, Song Jiang, Jian Ouyang, Shiding Lin, Chen Zhang, and Jason Cong. 2014 · 2014
Earlier work this paper cites.
Biscuit: A Framework for Near-data Processing of Big Data Workloads. In Proceedings of the 43rd International Symposium on Computer Architecture (Seoul, Republic of Korea) (ISCA ’16) . IEEE Press, Piscataway, NJ, USA, 153–165
Boncheol Gu, Andre S. Yoon, Duck-Ho Bae, Insoon Jo, Jinyoung Lee, Jonghyun Yoon, Jeong-Uk Kang, Moonsang Kwon, Chanho Yoon, Sangyeun Cho, Jaeheon Jeong, and Duckhyun Chang. 2016 · 2016
Earlier work this paper cites.

Similar

Flash reliability in production: The expected and the unexpected. In 14th { \{ USENIX } \} Conference on File and Storage Technologies ( { \{ FAST } \} 16) . 67–80
Bianca Schroeder, Raghav Lagisetty, and Arif Merchant. 2016 · 2016
Cited alongside, same era.
Ssd in-storage computing for search engines
Jianguo Wang, Dongchul Park, Yannis Papakonstantinou, and Steven Swanson. 2016 · 2016
Cited alongside, same era.
Neural Collaborative Filtering. In Proceedings of the 26th International Conference on World Wide Web (Perth, Australia) (WWW ’17) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 173–182
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017 · 2017
Cited alongside, same era.
KAML: A Flexible, High-Performance Key-Value SSD. In 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) . 373–384
Y. Jin, H. W. Tseng, Y. Papakonstantinou, and S. Swanson. 2017 · 2017
Cited alongside, same era.
NOTES FROM THE AI FRONTIER INSIGHTS FROM HUNDREDS OF USE CASES
Michael Chui, J Manyika, M Miremadi, N Henke, R Chung, P Nel, and S Malhotra. 2018 · 2018
Cited alongside, same era.
Bandana: Using non-volatile memory for storing deep learning models
Original
Assaf Eisenman, Maxim Naumov, Darryl Gardner, Misha Smelyanskiy, Sergey Pupyrev, Kim Hazelwood, Asaf Cidon, and Sachin Katti. 2018b · 2018
Cited alongside, same era.
Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective. In 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) . 620–629
K. Hazelwood, S. Bird, D. Brooks, S. Chintala, U. Diril, D. Dzhulgakov, M. Fawzy, B. Jia, Y. Jia, A. Kalro, J. Law, K. Lee, J. Lu, P. Noordhuis, M. Smelyanskiy, L. Xiong, and X. Wang. 2018 · 2018
Cited alongside, same era.
Deep learning inference in facebook data centers: Characterization, performance optimizations and hardware implications
Original
Jongsoo Park, Maxim Naumov, Protonu Basu, Summer Deng, Aravind Kalaiah, Daya Khudia, James Law, Parth Malani, Andrey Malevich, Satish Nadathur, et al · 2018
Cited alongside, same era.
Personalized Recommendation Systems: Five Hot Research Topics You Must Know
Xing Xie, Jianxun Lian, Zheng Liu, Xiting Wang, Fangzhao Wu, Hongwei Wang, and Zhongxia Chen. 2018 · 2018
Cited alongside, same era.
Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . ACM, 1059–1068
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018 · 2018
Cited alongside, same era.
Programmable solid-state storage in future cloud datacenters
Jaeyoung Do, Sudipta Sengupta, and Steven Swanson. 2019 · 2019
Cited alongside, same era.
Caffe2 Operator Catalog
[n.d.]b
Cited in the paper.

Then

The architectural implications of Facebook’s DNN-based personalized recommendation
Original
Udit Gupta, Xiaodong Wang, Maxim Naumov, Carole-Jean Wu, Brandon Reagen, David Brooks, Bradford Cottel, Kim Hazelwood, Bill Jia, Hsien-Hsin S Lee, et al · 2019
Later among the works it cites.
NeuMMU: Architectural Support for Efficient Address Translations in Neural Processing Units
Bongjoon Hyun, Youngeun Kwon, Yujeong Choi, John Kim, and Minsoo Rhu. 2019 · 2019
Later among the works it cites.
Basic performance measurements of the intel optane DC persistent memory module
Original
Joseph Izraelevitz, Jian Yang, Lu Zhang, Juno Kim, Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R Dulloor, et al · 2019
Later among the works it cites.
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
Original
Liu Ke, Udit Gupta, Carole-Jean Wu, Benjamin Y. Cho, Mark Hempstead, Brandon Reagen, Xuan Zhang, David M. Brooks, Vikas Chandra, Utku Diril, Amin Firoozshahian, Kim Hazelwood, Bill Jia, Hsien-Hsin S. Lee, Mengxing Li, Bert Maher, Dheevatsa Mudigere, Maxim Naumov, Martin Schatz, Mikhail Smelyanskiy, and Xiu Qiao Wang. 2019 · 2019
Later among the works it cites.
TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture . 740–753
Youngeun Kwon, Yunjae Lee, and Minsoo Rhu. 2019 · 2019
Later among the works it cites.
Use Cases of Recommendation Systems in Business – Current Applications and Methods
Corinna Underwood. 2019 · 2019
Later among the works it cites.
Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 33. 5941–5948
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019 · 2019
Later among the works it cites.
DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference
Original
Udit Gupta, Samuel Hsia, Vikram Saraph, Xiu Qiao Wang, Brandon Reagen, Gu-Yeon Wei, Hsien-Hsin S. Lee, David M. Brooks, and Carole-Jean Wu. 2020 · 2020
Later among the works it cites.
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems
Original
Maxim Naumov, John Kim, Dheevatsa Mudigere, Srinivas Sridharan, Xiting Wang, Whitney Zhao, Serhat Yilmaz, Changkyu Kim, Hector Yuen, Mustafa Ozdal, Krishnakumar Nair, Isabel Gao, Bor-Yiing Su, Jiyan Yang, and Mikhail Smelyanskiy. 2020 · 2020
Later among the works it cites.
Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems
Original
Weijie Zhao, Deping Xie, Ronglai Jia, Yulei Qian, Ruiquan Ding, Ming ming Sun, and Ping Li. 2020 · 2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…