2016

Composing Music with Grammar Argumented Neural Networks and Note-Level Encoding

Sun, Zheng, Liu, Jiaqi, Zhang, Zewang et al.

Understand

Creating aesthetically pleasing pieces of art, including music, has been a long-term goal for artificial intelligence research.

Despite recent successes of long-short term memory (LSTM) recurrent neural networks (RNNs) in sequential learning, LSTM neural networks have not, by themselves, been able to generate natural-sounding music conforming to music theory.
To transcend this inadequacy, we put forward a novel method for music composition that combines the LSTM with Grammars motivated by music theory.
The main tenets of music theory are encoded as grammar argumented (GA) filters on the training data, such that the machine can be trained to generate music inheriting the naturalness of human-composed pieces from the original dataset while adhering to the rules of music theory.

Built on

G. M. Rader, “A method for composing simple traditional music by computer,” Communications of the ACM , vol. 17, no. 11, pp. 631–638, 1974
1974
Earlier work this paper cites.
P. M. Todd, “A connectionist approach to algorithmic composition,” Computer Music Journal , vol. 13, no. 4, pp. 27–43, 1989
1989
Earlier work this paper cites.
D. Cope, “Computer modeling of musical intelligence in emi,” Computer Music Journal , vol. 16, no. 16, pp. 69–87, 1992
1992
Earlier work this paper cites.
M. C. MOZER, “Neural network music composition by prediction: Exploring the benefits of psychoacoustic constraints and multi-scale processing,” Connection Science , vol. 6, no. 2-3, pp. 247–280, 1994
1994
Earlier work this paper cites.
M. C. Mozer, “Neural network music composition by prediction: Exploring the benefits of psychoacoustic constraints and multi-scale processing,” Connection Science , vol. 6, no. 2-3, pp. 247–280, 1994
1994
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997
1997
Earlier work this paper cites.
THYWISSEN and KURT, “Genotator: an environment for exploring the application of evolutionary techniques in computer-assisted composition,” Organised Sound , vol. 4, no. 2, pp. 127–133, 1999
1999
Earlier work this paper cites.
M. Allan, “Harmonising chorales in the style of johann sebastian bach,” Master’s Thesis, School of Informatics, University of Edinburgh , 2002
2002
Earlier work this paper cites.
D. Eck and J. Schmidhuber, “Finding temporal structure in music: Blues improvisation with lstm recurrent networks,” in Neural Networks for Signal Processing, 2002. Proceedings of the 2002 12th IEEE Workshop on . IEEE, 2002, pp. 747–756
2002
Earlier work this paper cites.
A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional lstm and other neural network architectures,” Neural Networks , vol. 18, no. 5, pp. 602–610, 2005
2005
Earlier work this paper cites.

Similar

J. A. Franklin, “Recurrent neural networks for music computation,” Informs Journal on Computing , vol. 18, no. 3, pp. 321–338, 2006
2006
Cited alongside, same era.
J. A. Franklin, “Recurrent neural networks for music computation,” INFORMS Journal on Computing , vol. 18, no. 3, pp. 321–338, 2006
2006
Cited alongside, same era.
D. Eck and J. Lapalme, “Learning musical structure directly from sequences of music,” University of Montreal, Department of Computer Science, CP , vol. 6128, 2008
2008
Cited alongside, same era.
N. Boulanger-Lewandowski, Y. Bengio, and P. Vincent, “Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription,” arXiv preprint arXiv:1206.6392 , 2012
Original
2012
Cited alongside, same era.
A. Graves, “Generating sequences with recurrent neural networks,” arXiv preprint arXiv:1308.0850 , 2013
Original
2013
Cited alongside, same era.
J. D. Fernánd Ndez and F. Vico, “Ai methods in algorithmic composition: a comprehensive survey,” Journal of Artificial Intelligence Research , vol. 48, no. 48, pp. 513–582, 2013
2013
Cited alongside, same era.
S. Wermter, C. Weber, W. Duch, T. Honkela, and P. Koprinkovahristova, “Artificial neural networks and machine learning – icann 2014,” Lecture Notes in Computer Science , vol. 8681, 2014
2014
Cited alongside, same era.
K. Goel, R. Vohra, and J. Sahoo, “Polyphonic music generation by modeling temporal dependencies using a rnn-dbn,” in Artificial Neural Networks and Machine Learning–ICANN 2014 . Springer, 2014, pp. 217–224
2014
Cited alongside, same era.
D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” Computer Science , 2014
2014
Cited alongside, same era.
L. A. Gatys, A. S. Ecker, and M. Bethge, “A neural algorithm of artistic style,” arXiv preprint arXiv:1508.06576 , 2015
Original
2015
Cited alongside, same era.

Then

K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, and D. Wierstra, “Draw: A recurrent neural network for image generation,” arXiv preprint arXiv:1502.04623 , 2015
Original
2015
Later among the works it cites.
Q. Lyu, Z. Wu, and J. Zhu, “Polyphonic music modelling with lstm-rtrbm,” in Proceedings of the 23rd Annual ACM Conference on Multimedia Conference . ACM, 2015, pp. 991–994
2015
Later among the works it cites.
R. Jozefowicz, W. Zaremba, and I. Sutskever, “An empirical exploration of recurrent network architectures,” in Proceedings of The 32nd International Conference on Machine Learning , 2015, pp. 2342–2350
2015
Later among the works it cites.
F. Chollet, “Keras,” https://github.com/fchollet/keras
2015
Later among the works it cites.
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015, software available from tensorflow.org. [Online]. Available: http://tensorflow.org/
Original
2015
Later among the works it cites.
A. van den Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” arXiv preprint arXiv:1601.06759 , 2016
Original
2016
Closest in time.
D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, d. D. G. Van, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, and M. Lanctot, “Mastering the game of go with deep neural networks and tree search.” Nature , vol. 529, no. 7587, pp. 484–489, 2016
2016
Closest in time.
N. Jaques, S. Gu, R. E. Turner, and D. Eck, “Tuning recurrent neural networks with reinforcement learning,” arXiv preprint arXiv:1611.02796 , 2016
Original
2016
Closest in time.
J. P. Clendinning and E. W. Marvin, The Musician’s Guide to Theory and Analysis (Third Edition) . WW Norton & Company, 2016
2016
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…