2019

PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition

Kong, Qiuqiang, Cao, Yin, Iqbal, Turab et al.

Understand

Audio pattern recognition is an important research topic in the machine learning area, and includes several tasks such as audio tagging, acoustic scene classification, music classification, speech emotion classification and sound event detection.

Recently, neural networks have been applied to tackle audio pattern recognition problems.
However, previous systems are built on specific datasets with limited durations.
Recently, in computer vision and natural language processing, systems pretrained on large-scale datasets have generalized well to several tasks.

Built on

J. P. Woodard, “Modeling and classification of natural sounds by product code hidden Markov models,” IEEE Transactions on Signal Processing , vol. 40, pp. 1833–1835, 1992
1992
Earlier work this paper cites.
D. P. W. Ellis, “Detecting alarm sounds,” https://academiccommons.columbia.edu/doi/10.7916/D8F19821/download , 2001
2001
Earlier work this paper cites.
D. Li, I. K. Sethi, N. Dimitrova, and T. McGee, “Classification of general audio data for content-based retrieval,” Pattern Recognition Letters , vol. 22, pp. 533–544, 2001
2001
Earlier work this paper cites.
G. Tzanetakis and P. Cook, “Musical genre classification of audio signals,” IEEE Transactions on Speech and Audio Processing , vol. 10, pp. 293–302, 2002
2002
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2009, pp. 248–255
2009
Earlier work this paper cites.
E. Law and L. Von Ahn, “Input-agreement: a new mechanism for collecting data using human computation games,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 2009, pp. 1197–1206
2009
Earlier work this paper cites.
A. Mesaros, T. Heittola, A. Eronen, and T. Virtanen, “Acoustic event detection in real life recordings,” in European Signal Processing Conference (EUSIPCO) , 2010, pp. 1267–1271
2010
Earlier work this paper cites.
V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in International Conference on Machine Learning (ICML) , 2010, pp. 807–814
2010
Earlier work this paper cites.
B. Uzkent, B. D. Barkana, and H. Cevikalp, “Non-speech environmental sound classification using SVMs with a new set of features,” International Journal of Innovative Computing, Information and Control , vol. 8, pp. 3511–3524, 2012
2012
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems (NeurIPS) , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
L. Vuegen, B. Broeck, P. Karsmakers, J. F. Gemmeke, B. Vanrumste, and H. Hamme, “An MFCC-GMM approach for event detection and classification,” in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) , 2013
2013
Earlier work this paper cites.
A. Van Den Oord, S. Dieleman, and B. Schrauwen, “Transfer learning by supervised pre-training for audio-based music classification,” in Conference of the International Society for Music Information Retrieval (ISMIR) , 2014, pp. 29–34
2014
Earlier work this paper cites.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research , vol. 15, pp. 1929–1958, 2014
2014
Earlier work this paper cites.
M. Lin, Q. Chen, and S. Yan, “Network in network,” in International Conference on Learning Representations (ICLR) , 2014
2014
Earlier work this paper cites.
J. Salamon, C. Jacoby, and J. P. Bello, “A dataset and taxonomy for urban sound research,” in Proceedings of the ACM International Conference on Multimedia , 2014, pp. 1041–1044
2014
Earlier work this paper cites.
E. Cakir, T. Heittola, H. Huttunen, and T. Virtanen, “Polyphonic sound event detection using multi label deep neural networks,” in International Joint Conference on Neural Networks (IJCNN) , 2015
2015
Earlier work this paper cites.
D. Stowell, D. Giannoulis, E. Benetos, M. Lagrange, and M. D. Plumbley, “Detection and classification of acoustic scenes and events,” IEEE Transactions on Multimedia , vol. 17, pp. 1733–1746, 2015
2015
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” International Conference on Learning Representations (ICLR) , 2015
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International Conference on Machine Learning (ICML) , 2015, pp. 448–456
2015
Earlier work this paper cites.
B. McFee, C. Raffel, D. Liang, D. P. Ellis, M. McVicar, E. Battenberg, and O. Nieto, “librosa: Audio and music signal analysis in python,” in Proceedings of the Python in Science Conference , vol. 8, 2015, pp. 18–25
2015
Earlier work this paper cites.

Similar

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR) , 2015
2015
Cited alongside, same era.
K. J. Piczak, “ESC: Dataset for environmental sound classification,” in ACM International Conference on Multimedia , 2015, pp. 1015–1018
2015
Cited alongside, same era.
K. Choi, G. Fazekas, and M. Sandler, “Automatic tagging using deep convolutional neural networks,” in Conference of the International Society for Music Information Retrieval (ISMIR) , 2016, pp. 805–811
2016
Cited alongside, same era.
A. Mesaros, T. Heittola, and T. Virtanen, “TUT database for acoustic scene classification and sound event detection,” in Conference on European Signal Processing Conference (EUSIPCO) , 2016, pp. 1128–1132
2016
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016, pp. 770–778
2016
Cited alongside, same era.
J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio Set: An ontology and human-labeled dataset for audio events,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2017, pp. 776–780
2017
Cited alongside, same era.
A. Mesaros, T. Heittola, A. Diment, B. Elizalde, A. Shah, E. Vincent, B. Raj, and T. Virtanen, “DCASE 2017 challenge setup: Tasks, datasets and baseline system,” in Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE) , 2017, pp. 85–92
2017
Cited alongside, same era.
S. Hershey, S. Chaudhuri, D. P. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold et al. , “CNN architectures for large-scale audio classification,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2017, pp. 131–135
2017
Cited alongside, same era.
K. Choi, G. Fazekas, M. Sandler, and K. Cho, “Transfer learning for music classification and regression tasks,” in Conference of the International Society of Music Information Retrieval (ISMIR) , 2017, pp. 141–149
2017
Cited alongside, same era.
J. Pons, O. Nieto, M. Prockup, E. Schmidt, A. Ehmann, and X. Serra, “End-to-end learning for music audio tagging at scale,” in Conference of the International Society for Music Information Retrieval (ISMIR) , 2017, pp. 637–644
2017
Cited alongside, same era.
A. Diment and T. Virtanen, “Transfer learning of weakly labelled audio,” in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) , 2017, pp. 6–10
2017
Cited alongside, same era.
W. Dai, C. Dai, S. Qu, J. Li, and S. Das, “Very deep convolutional neural networks for raw waveforms,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2017, pp. 421–425
2017
Cited alongside, same era.

Then

S.-Y. Chou, J.-S. R. Jang, and Y.-H. Yang, “Learning to recognize transient sound events using attentional supervision,” in International Joint Conferences on Artificial Intelligence (IJCAI) , 2018, pp. 3336–3342
2018
Later among the works it cites.
Y. Wang, “Polyphonic sound event detection with weak labeling,” PhD thesis, Carnegie Mellon University , 2018
2018
Later among the works it cites.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018, pp. 4510–4520
2018
Later among the works it cites.
H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” in International Conference on Learning Representations (ICLR) , 2018
2018
Later among the works it cites.
I.-Y. Jeong and H. Lim, “Audio tagging system for DCASE 2018: focusing on label noise data augmentation and its efficient learning,” DCASE Challenge Tech. Rep. , 2018
2018
Later among the works it cites.
T. Chen and U. Gupta, “Attention-based convolutional neural network for audio event classification with feature transfer learning,” https://cvssp.org/projects/making_sense_of_sounds/site/assets/challenge_abstracts_and_figures/Tianxiang_Chen.pdf , 2018
2018
Later among the works it cites.
E. Fonseca, M. Plakal, F. Font, D. P. W. Ellis, X. Favory, J. Pons, and X. Serra, “General-purpose tagging of freesound audio with audioset labels: Task description, dataset, and baseline,” in Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE) , November 2018, pp. 69–73
2018
Later among the works it cites.
“DCASE Challenge 2019,” http://dcase.community/challenge2019 , 2019
2019
Closest in time.
Y. Wang, J. Li, and F. Metze, “A comparison of five multiple instance learning pooling functions for sound event detection with weak labeling,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2019, pp. 31–35
2019
Closest in time.
Q. Kong, C. Yu, Y. Xu, T. Iqbal, W. Wang, and M. D. Plumbley, “Weakly labelled audioset tagging with attention neural networks,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 27, pp. 1791–1802, 2019
2019
Closest in time.
J. Pons and X. Serra, “MUSICNN: Pre-trained convolutional neural networks for music audio tagging,” arXiv preprint arXiv:1909.06654 , 2019
Original
2019
Closest in time.
Q. Kong, Y. Cao, T. Iqbal, Y. Xu, W. Wang, and M. D. Plumbley, “Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems,” arXiv preprint arXiv:1904.03476 , 2019
Original
2019
Closest in time.
D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, “SpecAugment: A simple data augmentation method for automatic speech recognition,” in INTERSPEECH , 2019
2019
Closest in time.
L. Ford, H. Tang, F. Grondin, and J. Glass, “A deep residual network for large-scale acoustic scene analysis,” INTERSPEECH , pp. 2568–2572, 2019
2019
Closest in time.
H. Chen, Z. Liu, Z. Liu, P. Zhang, and Y. Yan, “Integrating the data augmentation scheme with various classifiers for acoustic scene modeling,” DCASE2019 Challenge, Tech. Rep. , 2019
2019
Closest in time.
C. Kroos, O. Bones, Y. Cao, L. Harris, P. J. Jackson, W. J. Davies, W. Wang, T. J. Cox, and M. D. Plumbley, “Generalisation in environmental sound classification: The ‘Making Sense of Sounds’ data set and challenge,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2019, pp. 8082–8086
2019
Closest in time.
C. Liu, L. Feng, G. Liu, H. Wang, and S. Liu, “Bottom-up broadcast neural network for music genre classification,” arXiv preprint arXiv:1901.08928 , 2019
Original
2019
Closest in time.
Y. Zeng, H. Mao, D. Peng, and Z. Yi, “Spectrogram based multi-task audio classification,” Multimedia Tools and Applications , vol. 78, pp. 3705–3722, 2019
2019
Closest in time.
S. R. Livingstone, K. Peck, and F. A. Russo, “RAVDESS: The Ryerson audio-visual database of emotional speech and song,” in Annual Meeting of the Canadian Society for Brain, Behaviour and Cognitive Science (CSBBCS) , 2012, pp. 1459–1462
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…