Fetching the paper…
Reading the bibliography…
While modern internet services, such as chatbots, search engines, and online advertising, demand the use of large-scale deep neural networks (DNNs), distributed training and inference over heterogeneous computing systems are desired to facilitate these DNN models.
R. Caruana, “Multitask learning,” Machine learning , vol. 28, no. 1, pp. 41–75, 1997
1997
Earlier work this paper cites.
L. B. Sokolinsky, “Lfu-k: An effective buffer management replacement algorithm,” in International Conference on Database Systems for Advanced Applications . Springer, 2004, pp. 670–681
2004
Earlier work this paper cites.
S. W. Keckler, W. J. Dally, B. Khailany, M. Garland, and D. Glasco, “Gpus and the future of parallel computing,” IEEE micro , vol. 31, no. 5, pp. 7–17, 2011
2011
Earlier work this paper cites.
Z. Ling, J. Luo, W. Yu, M. Yang, and X. Fu, “Extensive analysis and large-scale empirical evaluation of tor bridge discovery,” in 2012 Proceedings IEEE INFOCOM . IEEE, 2012, pp. 2381–2389
2012
Earlier work this paper cites.
2014
Earlier work this paper cites.
S. Gross, M. Ranzato, and A. Szlam, “Hard mixtures of experts for large scale weakly supervised vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 6865–6873
2017
Earlier work this paper cites.
2018
Earlier work this paper cites.
Intel, “Memory optimized for data-centric workloads,” https://www.intel.cn/content/www/cn/zh/architecture-and-technology/optane-dc-persistent-memory.html , 2018
2018
Earlier work this paper cites.
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2018
2018
Earlier work this paper cites.
Q. Liu, Y. Tian, J. Wu, T. Peng, and G. Wang, “Enabling verifiable and dynamic ranked search over outsourced data,” IEEE Transactions on Services Computing , vol. 15, no. 1, pp. 69–82, 2019
2019
Earlier work this paper cites.
R. Aharoni, M. Johnson, and O. Firat, “Massively multilingual neural machine translation,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 3874–3884
2019
Earlier work this paper cites.
X. Liu, S. X. Sun, and G. Huang, “Decentralized services computing paradigm for blockchain-based data governance: Programmability, interoperability, and intelligence,” IEEE Transactions on Services Computing , vol. 13, no. 2, pp. 343–355, 2019
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, “fairseq: A fast, extensible toolkit for sequence modeling,” in Proceedings of NAACL-HLT 2019: Demonstrations , 2019
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
X. Chen, M. Zhao, X. Yang, Z. Li, Y. Liu, Z. Li, and Y. Liu, “The cask effect of multi-source content delivery: Measurement and mitigation,” in 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) . IEEE, 2019, pp. 261–270
2019
Earlier work this paper cites.
J. Rasley, S. Rajbhandari, O. Ruwase, and Y. He, “Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters,” in SIGKDD , 2020, pp. 3505–3506
2020
Earlier work this paper cites.
Z. Cui, X. Xu, X. Fei, X. Cai, Y. Cao, W. Zhang, and J. Chen, “Personalized recommendation system based on collaborative filtering for iot scenarios,” IEEE Transactions on Services Computing , vol. 13, no. 4, pp. 685–695, 2020
2020
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020
2020
Cited alongside, same era.
W. Zhao, D. Xie, R. Jia, Y. Qian, R. Ding, M. Sun, and P. Li, “Distributed hierarchical gpu parameter server for massive scale deep learning ads systems,” Proceedings of Machine Learning and Systems , vol. 2, pp. 412–428, 2020
2020
Cited alongside, same era.
Y. Wang, C. Zhai, and H. Hassan, “Multi-task learning for multilingual neural machine translation,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Online: Association for Computational Linguistics, Nov. 2020, pp. 1022–1034
2020
Cited alongside, same era.
2021
Later among the works it cites.
2021
Later among the works it cites.
M. Artetxe, S. Bhosale, N. Goyal, T. Mihaylov, M. Ott, S. Shleifer, X. V. Lin, J. Du, S. Iyer, R. Pasunuru et al. , “Efficient large scale language modeling with mixtures of experts,” arXiv preprint , 2021
2021
Later among the works it cites.
Microsoft, “Tutel: An efficient mixture-of-experts implementation for large dnn model training,” https://www.microsoft.com/en-us/research/blog/tutel-an-efficient-mixture-of-experts-implementation-for-large-dnn-model-training/ , 2021
2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2020
Cited alongside, same era.
A. Yazdinejad, R. M. Parizi, A. Dehghantanha, Q. Zhang, and K.-K. R. Choo, “An energy-efficient sdn controller architecture for iot networks with blockchain-based security,” IEEE Transactions on Services Computing , vol. 13, no. 4, pp. 625–638, 2020
2020
Cited alongside, same era.
S. Tang, B. He, C. Yu, Y. Li, and K. Li, “A survey on spark ecosystem: Big data processing infrastructure, machine learning, and applications,” IEEE Transactions on Knowledge and Data Engineering , vol. 34, no. 1, pp. 71–91, 2020
2020
Cited alongside, same era.
2020
Cited alongside, same era.
P. Mattson, C. Cheng, G. Diamos, C. Coleman, P. Micikevicius, D. Patterson, H. Tang, G.-Y. Wei, P. Bailis, V. Bittorf et al. , “Mlperf training benchmark,” Proceedings of Machine Learning and Systems , vol. 2, pp. 336–349, 2020
2020
Cited alongside, same era.
L. Zhang, Y. Luo, Y. Bai, B. Du, and L.-Y. Duan, “Federated learning for non-iid data via unified feature learning and optimization objective alignment,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 4420–4428
2021
Cited alongside, same era.
H. Liu, Q. Gao, J. Li, X. Liao, H. Xiong, G. Chen, W. Wang, G. Yang, Z. Zha, D. Dong et al. , “Jizhi: A fast and cost-effective model-as-a-service system for web-scale online inference at baidu,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , 2021, pp. 3289–3298
2021
Cited alongside, same era.
C. Zhang, J. Zhou, X. Zang, Q. Xu, L. Yin, X. He, L. Liu, H. Xiong, and D. Dou, “Chase: Commonsense-enriched advertising on search engine with explicit knowledge,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management , 2021, pp. 4352–4361
2021
Cited alongside, same era.
L. Zou, S. Zhang, H. Cai, D. Ma, S. Cheng, S. Wang, D. Shi, Z. Cheng, and D. Yin, “Pre-trained language model based ranking in baidu search,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , 2021, pp. 4014–4022
2021
Cited alongside, same era.
Later among the works it cites.
S. Rajbhandari, O. Ruwase, J. Rasley, S. Smith, and Y. He, “Zero-infinity: Breaking the gpu memory wall for extreme scale deep learning,” in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , 2021, pp. 1–14
2021
Later among the works it cites.
2021
Later among the works it cites.
F. Al-Doghman, N. Moustafa, I. Khalil, Z. Tari, and A. Zomaya, “Ai-enabled secure microservices in edge computing: opportunities and challenges,” IEEE Transactions on Services Computing , 2022
2022
Closest in time.
J. Bian, H. Xiong, Z. Wang, J. Zhou, S. Ji, H. Chen, D. Zhang, and D. Dou, “Afcs: Aggregation-free spatial-temporal mobile community sensing,” IEEE Transactions on Mobile Computing , 2022
2022
Closest in time.
2022
Closest in time.
2022
Closest in time.
wikipedia, “Solid-state drive,” https://en.wikipedia.org/wiki/Solid-state_drive , 2022
2022
Closest in time.
Q. Huang, Z. Yuan, Z. Xing, Z. Zuo, C. Wang, and X. Xia, “1+ 1 > > 2: Programming know-what and know-how knowledge fusion, semantic enrichment and coherent application,” IEEE Transactions on Services Computing , 2022
2022
Closest in time.
D. Mudigere, Y. Hao, J. Huang, Z. Jia, A. Tulloch, S. Sridharan, X. Liu, M. Ozdal, J. Nie, J. Park et al. , “Software-hardware co-design for fast and scalable training of deep learning recommendation models,” in Proceedings of the 49th Annual International Symposium on Computer Architecture , 2022, pp. 993–1011
2022
Closest in time.
P. Contributors, “Paddlefleetx: An easy-to-use and high-performance one-stop tool for deep learning,” https://github.com/PaddlePaddle/PaddleFleetX , 2022
2022
Closest in time.
J. Bian, J. Huang, S. Ji, Y. Liao, X. Li, Q. Wang, J. Zhoy, Y. Wang, and D. Dou, “Feynman: Federated advertising for ecosystems-oriented mobile apps recommendation,” IEEE Transactions on Service Computing , 2023
2023
Closest in time.
J. Bian, J. Huang, S. Ji, Y. Liao, X. Li, Q. Wang, J. Zhou, D. Dou, Y. Wang, and H. Xiong, “Feynman: Federated learning-based advertising for ecosystems-oriented mobile apps recommendation,” IEEE Transactions on Services Computing , 2023
2023
Closest in time.
S. Wang, Y. Zheng, and X. Jia, “Secgnn: Privacy-preserving graph neural network training and inference as a cloud service,” IEEE Transactions on Services Computing , 2023
2023
Closest in time.
A. Shah, R. Ganesan, S. Jajodia, H. Cam, and S. Hutchinson, “A novel team formation framework based on performance in a cybersecurity operations center,” IEEE Transactions on Services Computing , 2023
2023
Closest in time.