Fetching the paper…
Reading the bibliography…
Mixtures of Experts (MoEs) have gained prominence in (self-)supervised learning due to their enhanced inference efficiency, adaptability to distributed training, and modularity.
The state of sparsity in deep neural networks
Trevor Gale, Erich Elsen, and Sara Hooker · 1902
Earlier work this paper cites.
A markovian decision process
Richard Bellman · 1957
Earlier work this paper cites.
Markov decision processes
Martin L Puterman · 1990
Earlier work this paper cites.
Adaptive mixtures of local experts
Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton · 1991
Earlier work this paper cites.
Python reference manual
Guido Van Rossum and Fred L Drake Jr · 1995
Earlier work this paper cites.
Matplotlib: A 2d graphics environment
John D Hunter · 2007
Earlier work this paper cites.
Python for scientific computing
Travis E. Oliphant · 2007
Earlier work this paper cites.
Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython
Wes McKinney · 2013
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis · 2015
Earlier work this paper cites.
Deep reinforcement learning with averaged target dqn
Oron Anschel, Nir Baram, and Nahum Shimkin · 2016
Earlier work this paper cites.
Jupyter Notebooks—a publishing format for reproducible computational workflows
Thomas Kluyver, Benjain Ragan-Kelley, Fernando Pérez, Brian Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Damián Avila, Safia Abdalla, Carol Willing, and Jupyter Development Team · 2016
Earlier work this paper cites.
Proximal policy optimization algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017
Earlier work this paper cites.
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer
Noam Shazeer, *Azalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Jax: composable transformations of python+ numpy programs
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, et al · 2018
Earlier work this paper cites.
Deep reinforcement learning in a handful of trials using probabilistic dynamics models
Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine · 2018
Earlier work this paper cites.
Model-ensemble trust-region policy optimization
Thanard Kurutach, Ignasi Clavera, Yan Duan, Aviv Tamar, and Pieter Abbeel · 2018
Earlier work this paper cites.
Reinforcement learning: An introduction
Richard S Sutton and Andrew G Barto · 2018
Earlier work this paper cites.
Dota 2 with large scale deep reinforcement learning
Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemyslaw Debiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al · 2019
Earlier work this paper cites.
Parameter-efficient transfer learning for NLP
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly · 2019
Earlier work this paper cites.
Model-based active exploration
Pranav Shyam, Wojciech Jaśkowski, and Faustino Gomez · 2019
Earlier work this paper cites.
Grandmaster level in starcraft ii using multi-agent reinforcement learning
Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al · 2019
Earlier work this paper cites.
Condconv: Conditionally parameterized convolutions for efficient inference
Brandon Yang, Gabriel Bender, Quoc V Le, and Jiquan Ngiam · 2019
Earlier work this paper cites.
Minatar: An atari-inspired testbed for thorough and reproducible reinforcement learning experiments
Kenny Young and Tian Tian · 2019
Earlier work this paper cites.
Biased mixtures of experts: Enabling computer vision inference under data transfer limitations
Alhabib Abbas and Yiannis Andreopoulos · 2020
Earlier work this paper cites.
An optimistic perspective on offline reinforcement learning
Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi · 2020
Earlier work this paper cites.
Autonomous navigation of stratospheric balloons using reinforcement learning
Marc G. Bellemare, Salvatore Candido, Pablo Samuel Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera S. Ponda, and Ziyun Wang · 2020
Earlier work this paper cites.
Rigging the lottery: Making all tickets winners
Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, and Erich Elsen · 2020
Cited alongside, same era.
Array programming with numpy
Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al · 2020
Cited alongside, same era.
Transient non-stationarity and generalisation in deep reinforcement learning
Maximilian Igl, Gregory Farquhar, Jelena Luketina, Wendelin Boehmer, and Shimon Whiteson · 2020
Cited alongside, same era.
Maxmin q-learning: Controlling the estimation bias of q-learning
Qingfeng Lan, Yangchen Pan, Alona Fyshe, and Martha White · 2020
Cited alongside, same era.
Gshard: Scaling giant models with conditional computation and automatic sharding
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen · 2020
Cited alongside, same era.
Towards continual reinforcement learning: A review and perspectives
Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup · 2022
Later among the works it cites.
gymnax: A JAX-based reinforcement learning environment library, 2022
Robert Tjarko Lange · 2022
Later among the works it cites.
Reducing variance in temporal-difference value estimation via ensemble of deep networks
Litian Liang, Yaosheng Xu, Stephen McAleer, Dailin Hu, Alexander Ihler, Pieter Abbeel, and Roy Fox · 2022
Later among the works it cites.
Understanding and preventing capacity loss in reinforcement learning
Clare Lyle, Mark Rowland, and Will Dabney · 2022
Later among the works it cites.
Dynamic sparse training for deep reinforcement learning
Ghada Sokar, Elena Mocanu, Decebal Constantin Mocanu, Mykola Pechenizkiy, and Peter Stone · 2022
Later among the works it cites.
Rlx2: Training a sparse deep reinforcement learning model from scratch
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Using mixture of expert models to gain insights into semantic segmentation
Svetlana Pavlitskaya, Christian Hubschneider, Michael Weber, Ruby Moritz, Fabian Huger, Peter Schlicht, and Marius Zollner · 2020
Cited alongside, same era.
Deep mixture of experts via shallow embedding
Xin Wang, Fisher Yu, Lisa Dunlap, Yi-An Ma, Ruth Wang, Azalia Mirhoseini, Trevor Darrell, and Joseph E Gonzalez · 2020
Cited alongside, same era.
Deep reinforcement learning at the edge of the statistical precipice
Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Bellemare · 2021
Cited alongside, same era.
Continuous action reinforcement learning from a mixture of interpretable experts
Riad Akrour, Davide Tateo, and Jan Peters · 2021
Cited alongside, same era.
Uncertainty-based offline reinforcement learning with diversified q-ensemble
Gaon An, Seungyong Moon, Jang-Hyun Kim, and Hyun Oh Song · 2021
Cited alongside, same era.
Single-shot pruning for offline reinforcement learning
Samin Yeasar Arnob, Riyasat Ohib, Sergey Plis, and Doina Precup · 2021
Cited alongside, same era.
Randomized ensembled double q-learning: Learning fast without a model
Xinyue Chen, Che Wang, Zijian Zhou, and Keith Ross · 2021
Cited alongside, same era.
Yiqin Tan, Pihe Hu, Ling Pan, Jiatai Huang, and Longbo Huang · 2022
Later among the works it cites.
Cola: consistent learning with opponent-learning awareness
Timon Willi, Alistair Hp Letcher, Johannes Treutlein, and Jakob Foerster · 2022
Later among the works it cites.
Mixture-of-experts with expert choice routing
Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew M Dai, Quoc V Le, James Laudon, et al · 2022
Later among the works it cites.
St-moe: Designing stable and transferable sparse expert models, 2022
Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus · 2022
Later among the works it cites.
Loss of plasticity in continual deep reinforcement learning
Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, and Marlos C Machado · 2023
Later among the works it cites.
Mod-squad: Designing mixtures of experts as modular multi-task learners
Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik G Learned-Miller, and Chuang Gan · 2023
Later among the works it cites.
Jiajun Fan, Yuzheng Zhuang, Yuecheng Liu, Jianye Hao, Bin Wang, Jiangcheng Zhu, Hao Wang, and Shu-Tao Xia · 2023
Later among the works it cites.
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts
Trevor Gale, Deepak Narayanan, Cliff Young, and Matei Zaharia · 2023
Later among the works it cites.
Relu to the rescue: Improve your on-policy actor-critic with positive advantages
Andrew Jesson, Chris Lu, Gunshi Gupta, Angelos Filos, Jakob Nicolaus Foerster, and Yarin Gal · 2023
Later among the works it cites.
Scaling opponent shaping to high dimensional games
Akbir Khan, Timon Willi, Newton Kwan, Andrea Tacchetti, Chris Lu, Edward Grefenstette, Tim Rocktäschel, and Jakob Foerster · 2023
Later among the works it cites.
Adversarial cheap talk
Chris Lu, Timon Willi, Alistair Letcher, and Jakob Nicolaus Foerster · 2023
Later among the works it cites.
Understanding plasticity in neural networks
Clare Lyle, Zeyu Zheng, Evgenii Nikishin, Bernardo Ávila Pires, Razvan Pascanu, and Will Dabney · 2023
Later among the works it cites.
Small batch deep reinforcement learning
Johan Obando Ceron, Marc Bellemare, and Pablo Samuel Castro · 2023
Later among the works it cites.
From sparse to soft mixtures of experts, 2023
Joan Puigcerver, Carlos Riquelme, Basil Mustafa, and Neil Houlsby · 2023
Later among the works it cites.
Jaxmarl: Multi-agent rl environments in jax
Alexander Rutherford, Benjamin Ellis, Matteo Gallici, Jonathan Cook, Andrei Lupu, Gardar Ingvarsson, Timon Willi, Akbir Khan, Christian Schroeder de Witt, Alexandra Souly, et al · 2023
Later among the works it cites.
Bigger, better, faster: Human-level Atari with human-level efficiency
Max Schwarzer, Johan Samir Obando-Ceron, Aaron Courville, Marc G Bellemare, Rishabh Agarwal, and Pablo Samuel Castro · 2023
Later among the works it cites.
The dormant neuron phenomenon in deep reinforcement learning
Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro, and Utku Evci · 2023
Later among the works it cites.
Taskexpert: Dynamically assembling multi-task representations with memorial mixture-of-experts
Hanrong Ye and Dan Xu · 2023
Later among the works it cites.
Smacv2: An improved benchmark for cooperative multi-agent reinforcement learning
Benjamin Ellis, Jonathan Cook, Skander Moalla, Mikayel Samvelyan, Mingfei Sun, Anuj Mahajan, Jakob Foerster, and Shimon Whiteson · 2024
Closest in time.
Stop regressing: Training value functions via classification for scalable deep rl
Jesse Farebrother, Jordi Orbay, Quan Vuong, Adrien Ali Taïga, Yevgen Chebotar, Ted Xiao, Alex Irpan, Sergey Levine, Pablo Samuel Castro, Aleksandra Faust, Aviral Kumar, and Rishabh Agarwal · 2024
Closest in time.
Multi-task reinforcement learning with mixture of orthogonal experts
Ahmed Hendawy, Jan Peters, and Carlo D’Eramo · 2024
Closest in time.