2017

Multi-Level and Multi-Scale Feature Aggregation Using Pre-trained Convolutional Neural Networks for Music Auto-tagging

Lee, Jongpil, Nam, Juhan

Understand

Music auto-tagging is often handled in a similar manner to image classification by regarding the 2D audio spectrogram as image data.

However, music auto-tagging is distinguished from image classification in that the tags are highly diverse and have different levels of abstractions.
Considering this issue, we propose a convolutional neural networks (CNN)-based architecture that embraces multi-level and multi-scaled features.
The architecture is trained in three steps.

Built on

G. Tzanetakis and P. Cook, “Musical genre classification of audio signals,” IEEE Transactions on speech and audio processing , vol. 10, no. 5, pp. 293–302, 2002
2002
Earlier work this paper cites.
H. Lee, Y. Largman, P. Pham, and A. Y. Ng, “Unsupervised feature learning for audio classification using convolutional deep belief networks,” in Advances in Neural Information Processing Systems 22 , 2009, pp. 1096–1104
2009
Earlier work this paper cites.
E. Law, K. West, M. I. Mandel, M. Bay, and J. S. Downie, “Evaluation of algorithms using games: The case of music tagging,” in ISMIR , 2009, pp. 387–392
2009
Earlier work this paper cites.
P. Hamel and D. Eck, “Learning features from music audio with deep belief networks,” in Proceedings of the 11th International Conference on Music Information Retrieval (ISMIR) . Utrecht, The Netherlands, 2010, pp. 339–344
2010
Earlier work this paper cites.
P. Hamel, S. Lemieux, Y. Bengio, and D. Eck, “Temporal pooling and multiscale learning for automatic annotation and ranking of music audio,” in Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR) , 2011
2011
Earlier work this paper cites.
T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere, “The million song dataset,” in Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR) , vol. 2, no. 9, 2011, pp. 591–596
2011
Earlier work this paper cites.
P. Hamel, Y. Bengio, and D. Eck, “Building musically-relevant audio features through multiple timescale representations,” in Proceedings of the 13th International Conference on Music Information Retrieval (ISMIR) , 2012
2012
Earlier work this paper cites.
J. Nam, J. Herrera, M. Slaney, and J. O. Smith, “Learning sparse feature representations for music annotation and retrieval,” in Proceedings of the 13th International Conference on Music Information Retrieval (ISMIR) , 2012, pp. 565–570
2012
Earlier work this paper cites.
S. Dieleman and B. Schrauwen, “Multiscale approaches to music audio feature learning,” in 14th International Society for Music Information Retrieval Conference (ISMIR-2013) . Pontifícia Universidade Católica do Paraná, 2013, pp. 116–121
2013
Earlier work this paper cites.

Similar

P. Hamel, M. E. P. Davies, K. Yoshii, and M. Goto, “Transfer learning in mir: Sharing learned latent representations for music audio classification and similarity,” in 14th International Conference on Music Information Retrieval (ISMIR ’13) , 2013
2013
Cited alongside, same era.
S. Dieleman and B. Schrauwen, “End-to-end learning for music audio,” in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2014, pp. 6964–6968
2014
Cited alongside, same era.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Cited alongside, same era.
A. Van Den Oord, S. Dieleman, and B. Schrauwen, “Transfer learning by supervised pre-training for audio-based music classification,” in Conference of the International Society for Music Information Retrieval (ISMIR 2014) , 2014
2014
Cited alongside, same era.
J. Nam, J. Herrera, and K. Lee, “A deep bag-of-features model for music auto-tagging,” arXiv preprint arXiv:1508.04999 , 2015
Original
2015
Cited alongside, same era.
C. Kereliuk, B. L. Sturm, and J. Larsen, “Deep learning and music adversaries,” IEEE Transactions on Multimedia , vol. 17, no. 11, pp. 2059–2071, 2015
2015
Cited alongside, same era.
H. Schreiber, “Improving genre annotations for the million song dataset,” in Proceedings of the 16th International Conference on Music Information Retrieval (ISMIR) , 2015, pp. 241–247
2015
Cited alongside, same era.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167 , 2015
Original
2015
Cited alongside, same era.

Then

F. Chollet, “Keras: Deep learning library for theano and tensorflow,” https://github.com/fchollet/keras , 2015
2015
Later among the works it cites.
J. Pons, T. Lidy, and X. Serra, “Experimenting with musically motivated convolutional neural networks,” in 2016 14th International Workshop on Content-Based Multimedia Indexing (CBMI) . IEEE, 2016, pp. 1–6
2016
Later among the works it cites.
K. Choi, G. Fazekas, and M. Sandler, “Automatic tagging using deep convolutional neural networks,” in Proceedings of the 17th International Conference on Music Information Retrieval (ISMIR) , 2016, pp. 805–811
2016
Later among the works it cites.
R. Al-Rfou, G. Alain, A. Almahairi, C. Angermueller, D. Bahdanau, N. Ballas, F. Bastien, J. Bayer, A. Belikov, A. Belopolsky et al. , “Theano: A python framework for fast computation of mathematical expressions,” arXiv preprint arXiv:1605.02688 , 2016
Original
2016
Later among the works it cites.
J.-Y. Liu, S.-K. Jeng, and Y.-H. Yang, “Applying topological persistence in convolutional neural network for music audio signals,” arXiv preprint arXiv:1608.07373 , 2016
Original
2016
Later among the works it cites.
U. Güçlü, J. Thielen, M. Hanke, M. van Gerven, and M. A. van Gerven, “Brains on beats,” in Advances in Neural Information Processing Systems , 2016, pp. 2101–2109
2016
Later among the works it cites.
I.-Y. Jeong and K. Lee, “Learning temporal features using a deep neural network and its application to music genre classification,” in Proceedings of the 17th International Conference on Music Information Retrieval (ISMIR) , 2016, pp. 434–440
2016
Later among the works it cites.
K. Choi, G. Fazekas, M. Sandler, and K. Cho, “Convolutional recurrent neural networks for music classification,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2017, pp. 2392–2396
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…