Built on

J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, and D. S. Pallett, “Darpa timit acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1.1,”
1993
Earlier work this paper cites.
R. H. Woo, A. Park, and T. J. Hazen, “The mit mobile device speaker verification corpus: data collection and preliminary experiments,” in
2006
Earlier work this paper cites.
E. Variani, X. Lei, E. McDermott, I. L. Moreno, and J. Gonzalez-Dominguez, “Deep neural networks for small footprint text-dependent speaker verification,” in
2014
Earlier work this paper cites.
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,”
Original
2014
Earlier work this paper cites.
L. Sifre and S. Mallat, “Rigid-motion scattering for image classification,”
2014
Earlier work this paper cites.
J. Jin, A. Dundar, and E. Culurciello, “Flattened convolutional neural networks for feedforward acceleration,”
Original
2014
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
Original
2014
Earlier work this paper cites.
W. Chan, N. Jaitly, Q. V. Le, and O. Vinyals, “Listen, attend and spell,”
Original
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,”
Original
2015
Earlier work this paper cites.
V. Sindhwani, T. Sainath, and S. Kumar, “Structured transforms for small-footprint deep learning,” in
2015
Earlier work this paper cites.
Z. Yang, M. Moczulski, M. Denil, N. de Freitas, A. Smola, L. Song, and Z. Wang, “Deep fried convnets,” in
2015
Earlier work this paper cites.
M.-T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,”
Original
2015
Earlier work this paper cites.
P. Safari, O. Ghahabi, and J. Hernando, “From features to speaker vectors by means of restricted boltzmann machine adaptation,” in
2016
Earlier work this paper cites.
S.-X. Zhang, Z. Chen, Y. Zhao, J. Li, and Y. Gong, “End-to-end attention based text-dependent speaker verification,” in
2016
Earlier work this paper cites.

Similar

F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size,”
Original
2016
Cited alongside, same era.
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,”
Original
2016
Cited alongside, same era.
D. Snyder, D. Garcia-Romero, D. Povey, and S. Khudanpur, “Deep neural network embeddings for text-independent speaker verification,” in
2017
Cited alongside, same era.
G. Bhattacharya, M. J. Alam, and P. Kenny, “Deep speaker embeddings for short-duration speaker verification,” in
2017
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in
2017

Then

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,”
Original
2018
Later among the works it cites.
C.-C. Chiu, T. N. Sainath, Y. Wu, and
2018
Later among the works it cites.
M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser, “Universal transformers,”
Original
2018
Later among the works it cites.
Y. Zhu, T. Ko, D. Snyder, B. Mak, and D. Povey, “Self-attentive speaker embeddings for text-independent speaker verification,” in
2018
Later among the works it cites.
W. Cai, J. Chen, and M. Li, “Exploring the encoding layer and loss function in end-to-end speaker and language recognition system,” in
2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2020

Self-attention encoding and pooling for speaker recognition

Safari, Pooyan, India, Miquel, Hernando, Javier

Understand

The computing power of mobile devices limits the end-user applications in terms of storage size, processing, memory and energy consumption.

These limitations motivate researchers for the design of more efficient deep models.
On the other hand, self-attention networks based on Transformer architecture have attracted remarkable interests due to their high parallelization capabilities and strong performance on a variety of Natural Language Processing (NLP) applications.
Inspired by the Transformer, we propose a tandem Self-Attention Encoding and Pooling (SAEP) mechanism to obtain a discriminative speaker embedding given non-fixed length speech utterances.

Built on

J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, and D. S. Pallett, “Darpa timit acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1.1,”
1993
Earlier work this paper cites.
R. H. Woo, A. Park, and T. J. Hazen, “The mit mobile device speaker verification corpus: data collection and preliminary experiments,” in
2006
Earlier work this paper cites.
E. Variani, X. Lei, E. McDermott, I. L. Moreno, and J. Gonzalez-Dominguez, “Deep neural networks for small footprint text-dependent speaker verification,” in
2014
Earlier work this paper cites.
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,”
Original
2014
Earlier work this paper cites.
L. Sifre and S. Mallat, “Rigid-motion scattering for image classification,”
2014
Earlier work this paper cites.
J. Jin, A. Dundar, and E. Culurciello, “Flattened convolutional neural networks for feedforward acceleration,”
Original
2014
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
Original
2014
Earlier work this paper cites.
W. Chan, N. Jaitly, Q. V. Le, and O. Vinyals, “Listen, attend and spell,”
Original
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,”
Original
2015
Earlier work this paper cites.
V. Sindhwani, T. Sainath, and S. Kumar, “Structured transforms for small-footprint deep learning,” in
2015
Earlier work this paper cites.
Z. Yang, M. Moczulski, M. Denil, N. de Freitas, A. Smola, L. Song, and Z. Wang, “Deep fried convnets,” in
2015
Earlier work this paper cites.
M.-T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,”
Original
2015
Earlier work this paper cites.
P. Safari, O. Ghahabi, and J. Hernando, “From features to speaker vectors by means of restricted boltzmann machine adaptation,” in
2016
Earlier work this paper cites.
S.-X. Zhang, Z. Chen, Y. Zhao, J. Li, and Y. Gong, “End-to-end attention based text-dependent speaker verification,” in
2016
Earlier work this paper cites.

Similar

F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size,”
Original
2016
Cited alongside, same era.
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,”
Original
2016
Cited alongside, same era.
D. Snyder, D. Garcia-Romero, D. Povey, and S. Khudanpur, “Deep neural network embeddings for text-independent speaker verification,” in
2017
Cited alongside, same era.
G. Bhattacharya, M. J. Alam, and P. Kenny, “Deep speaker embeddings for short-duration speaker verification,” in
2017
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in
2017

Then

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,”
Original
2018
Later among the works it cites.
C.-C. Chiu, T. N. Sainath, Y. Wu, and
2018
Later among the works it cites.
M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser, “Universal transformers,”
Original
2018
Later among the works it cites.
Y. Zhu, T. Ko, D. Snyder, B. Mak, and D. Povey, “Self-attentive speaker embeddings for text-independent speaker verification,” in
2018
Later among the works it cites.
W. Cai, J. Chen, and M. Li, “Exploring the encoding layer and loss function in end-to-end speaker and language recognition system,” in
2018
Later among the works it cites.