Built on

“Understanding the difficulty of training deep feedforward neural networks,”
Xavier Glorot and Yoshua Bengio, · 2010
Earlier work this paper cites.
“Parallel training of deep neural networks with natural gradient and parameter averaging,”
Original
Daniel Povey, Xiaohui Zhang, and Sanjeev Khudanpur, · 2014
Earlier work this paper cites.
“Adam: A method for stochastic optimization,”
Original
Diederik P. Kingma and Jimmy Ba, · 2014
Earlier work this paper cites.
“Small-footprint keyword spotting using deep neural networks,”
Guoguo Chen, Carolina Parada, and Georg Heigold, · 2014
Earlier work this paper cites.
“A time delay neural network architecture for efficient modeling of long temporal contexts,”
Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur, · 2015
Earlier work this paper cites.
“Librispeech: An ASR corpus based on public domain audio books,”
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, · 2015
Earlier work this paper cites.

Similar

“Federated learning of deep networks using model averaging,”
Original
H. Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Agüera y Arcas, · 2016
Cited alongside, same era.
“Federated optimization: Distributed machine learning for on-device intelligence,”
Original
Jakub Konecný, H. Brendan McMahan, Daniel Ramage, and Peter Richtárik, · 2016
Cited alongside, same era.
“Federated learning: Strategies for improving communication efficiency,”
Original
Jakub Konecný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon, · 2016
Cited alongside, same era.
“Hello edge: Keyword spotting on microcontrollers,”
Original
Yundong Zhang, Naveen Suda, Liangzhen Lai, and Vikas Chandra, · 2017
Cited alongside, same era.
“Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces,”

Then

“Client selection for federated learning with heterogeneous resources in mobile edge,”
Original
Takayuki Nishio and Ryo Yonetani, · 2018
Closest in time.
“When edge meets learning: Adaptive control for resource-constrained distributed machine learning,”
Original
Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, and Kevin Chan, · 2018
Closest in time.
“Efficient decentralized deep learning by dynamic model averaging,”
Original
Joachim Sicking Fabian Hüger Peter Schlicht Tim Wirtz Michael Kamp, Linara Adilova and Stefan Wrobel, · 2018
Closest in time.
“Speech commands: A dataset for limited-vocabulary speech recognition,”
Original
Pete Warden, · 2018
Closest in time.
“Efficient keyword spotting using dilated convolutions and gating,”

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2018

Federated Learning for Keyword Spotting

Leroy, David, Coucke, Alice, Lavril, Thibaut et al.

Understand

We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors.

We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mimics a federation of wake word users.
We empirically demonstrate that using an adaptive averaging strategy inspired from Adam in place of standard weighted model averaging highly reduces the number of communication rounds required to reach our target performance.
The associated upstream communication costs per user are estimated at 8 MB, which is a reasonable in the context of smart home voice assistants.

Built on

“Understanding the difficulty of training deep feedforward neural networks,”
Xavier Glorot and Yoshua Bengio, · 2010
Earlier work this paper cites.
“Parallel training of deep neural networks with natural gradient and parameter averaging,”
Original
Daniel Povey, Xiaohui Zhang, and Sanjeev Khudanpur, · 2014
Earlier work this paper cites.
“Adam: A method for stochastic optimization,”
Original
Diederik P. Kingma and Jimmy Ba, · 2014
Earlier work this paper cites.
“Small-footprint keyword spotting using deep neural networks,”
Guoguo Chen, Carolina Parada, and Georg Heigold, · 2014
Earlier work this paper cites.
“A time delay neural network architecture for efficient modeling of long temporal contexts,”
Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur, · 2015
Earlier work this paper cites.
“Librispeech: An ASR corpus based on public domain audio books,”
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, · 2015
Earlier work this paper cites.

Similar

“Federated learning of deep networks using model averaging,”
Original
H. Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Agüera y Arcas, · 2016
Cited alongside, same era.
“Federated optimization: Distributed machine learning for on-device intelligence,”
Original
Jakub Konecný, H. Brendan McMahan, Daniel Ramage, and Peter Richtárik, · 2016
Cited alongside, same era.
“Federated learning: Strategies for improving communication efficiency,”
Original
Jakub Konecný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon, · 2016
Cited alongside, same era.
“Hello edge: Keyword spotting on microcontrollers,”
Original
Yundong Zhang, Naveen Suda, Liangzhen Lai, and Vikas Chandra, · 2017
Cited alongside, same era.
“Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces,”

Then

“Client selection for federated learning with heterogeneous resources in mobile edge,”
Original
Takayuki Nishio and Ryo Yonetani, · 2018
Closest in time.
“When edge meets learning: Adaptive control for resource-constrained distributed machine learning,”
Original
Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, and Kevin Chan, · 2018
Closest in time.
“Efficient decentralized deep learning by dynamic model averaging,”
Original
Joachim Sicking Fabian Hüger Peter Schlicht Tim Wirtz Michael Kamp, Linara Adilova and Stefan Wrobel, · 2018
Closest in time.
“Speech commands: A dataset for limited-vocabulary speech recognition,”
Original
Pete Warden, · 2018
Closest in time.
“Efficient keyword spotting using dilated convolutions and gating,”

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, Maël Primet, and Joseph Dureau, · 2018

Cited alongside, same era.

Original

Alice Coucke, Mohammed Chlieh, Thibault Gisselbrecht, David Leroy, Mathieu Poumeyrol, and Thibaut Lavril, · 2018

Closest in time.