Built on

Transformer-xl: Attentive language models beyond a fixed-length context
Original
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc V. Le, and Ruslan Salakhutdinov · 1901
Earlier work this paper cites.
Superposition of many models into one
Original
Brian Cheung, Alex Terekhov, Yubei Chen, Pulkit Agrawal, and Bruno A. Olshausen · 1902
Earlier work this paper cites.
REGAL: transfer learning for fast optimization of computation graphs
Original
Aditya Paliwal, Felix Gimeno, Vinod Nair, Yujia Li, Miles Lubin, Pushmeet Kohli, and Oriol Vinyals · 1905
Earlier work this paper cites.
Placeto: Learning generalizable device placement algorithms for distributed machine learning
Original
Ravichandra Addanki, Shaileshh Bojja Venkatakrishnan, Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh · 1906
Earlier work this paper cites.
A fast and high quality multilevel scheme for partitioning irregular graphs
George Karypis and Vipin Kumar · 1998
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Original
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le · 2014
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Original
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna · 2015
Earlier work this paper cites.
Exploring the limits of language modeling
Original
Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu · 2016
Earlier work this paper cites.
Wavenet: A generative model for raw audio
Original
Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu · 2016
Earlier work this paper cites.

Similar

Google’s neural machine translation system: Bridging the gap between human and machine translation
Original
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean · 2016
Cited alongside, same era.
Inductive representation learning on large graphs
Original
William L. Hamilton, Rex Ying, and Jure Leskovec · 2017
Cited alongside, same era.
Deep learning scaling is predictable, empirically
Original
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou · 2017
Cited alongside, same era.
Device placement optimization with reinforcement learning
Original
Azalia Mirhoseini, Hieu Pham, Quoc V. Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, and Jeff Dean · 2017

Then

Gpipe: Efficient training of giant neural networks using pipeline parallelism
Original
Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V. Le, and Zhifeng Chen · 2018
Later among the works it cites.
Beyond data and model parallelism for deep neural networks
Original
Zhihao Jia, Matei Zaharia, and Alex Aiken · 2018
Later among the works it cites.
Exploring the limits of weakly supervised pretraining
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten · 2018
Later among the works it cites.
A hierarchical model for device placement
Azalia Mirhoseini, Anna Goldie, Hieu Pham, Benoit Steiner, Quoc V. Le, and Jeff Dean · 2018
Later among the works it cites.
Regularized evolution for image classifier architecture search
Original
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le · 2018

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2019

GDP: Generalized Device Placement for Dataflow Graphs

Zhou, Yanqi, Roy, Sudip, Abdolrashidi, Amirali et al.

Understand

Runtime and scalability of large neural networks can be significantly affected by the placement of operations in their dataflow graphs on suitable devices.

With increasingly complex neural network architectures and heterogeneous device characteristics, finding a reasonable placement is extremely challenging even for domain experts.
Most existing automated device placement approaches are impractical due to the significant amount of compute required and their inability to generalize to new, previously held-out graphs.
To address both limitations, we propose an efficient end-to-end method based on a scalable sequential attention mechanism over a graph neural network that is transferable to new graphs.

Built on

Transformer-xl: Attentive language models beyond a fixed-length context
Original
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc V. Le, and Ruslan Salakhutdinov · 1901
Earlier work this paper cites.
Superposition of many models into one
Original
Brian Cheung, Alex Terekhov, Yubei Chen, Pulkit Agrawal, and Bruno A. Olshausen · 1902
Earlier work this paper cites.
REGAL: transfer learning for fast optimization of computation graphs
Original
Aditya Paliwal, Felix Gimeno, Vinod Nair, Yujia Li, Miles Lubin, Pushmeet Kohli, and Oriol Vinyals · 1905
Earlier work this paper cites.
Placeto: Learning generalizable device placement algorithms for distributed machine learning
Original
Ravichandra Addanki, Shaileshh Bojja Venkatakrishnan, Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh · 1906
Earlier work this paper cites.
A fast and high quality multilevel scheme for partitioning irregular graphs
George Karypis and Vipin Kumar · 1998
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Original
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le · 2014
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Original
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna · 2015
Earlier work this paper cites.
Exploring the limits of language modeling
Original
Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu · 2016
Earlier work this paper cites.
Wavenet: A generative model for raw audio
Original
Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu · 2016
Earlier work this paper cites.

Similar

Google’s neural machine translation system: Bridging the gap between human and machine translation
Original
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean · 2016
Cited alongside, same era.
Inductive representation learning on large graphs
Original
William L. Hamilton, Rex Ying, and Jure Leskovec · 2017
Cited alongside, same era.
Deep learning scaling is predictable, empirically
Original
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou · 2017
Cited alongside, same era.
Device placement optimization with reinforcement learning
Original
Azalia Mirhoseini, Hieu Pham, Quoc V. Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, and Jeff Dean · 2017

Then

Gpipe: Efficient training of giant neural networks using pipeline parallelism
Original
Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V. Le, and Zhifeng Chen · 2018
Later among the works it cites.
Beyond data and model parallelism for deep neural networks
Original
Zhihao Jia, Matei Zaharia, and Alex Aiken · 2018
Later among the works it cites.
Exploring the limits of weakly supervised pretraining
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten · 2018
Later among the works it cites.
A hierarchical model for device placement
Azalia Mirhoseini, Anna Goldie, Hieu Pham, Benoit Steiner, Quoc V. Le, and Jeff Dean · 2018
Later among the works it cites.
Regularized evolution for image classifier architecture search
Original
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le · 2018