Fetching the paper…

Reading the bibliography…

Conditional LSTM-GAN for Melody Generation from Lyrics · Around

2019

Conditional LSTM-GAN for Melody Generation from Lyrics

Yu, Yi, Srivastava, Abhishek, Canales, Simon

Original
PDF
Post on X

Understand

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables to learn and discover latent relationship between interesting lyrics and accompanying melody.

Unfortunately, the limited availability of paired lyrics-melody dataset with alignment information has hindered the research progress.
To address this problem, we create a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment through leveraging different music sources where alignment relationship between syllables and music attributes is extracted.
Most importantly, we propose a novel deep generative model, conditional Long Short-Term Memory - Generative Adversarial Network (LSTM-GAN) for melody generation from lyrics, which contains a deep LSTM generator and a deep LSTM discriminator both conditioned on lyrics.

Built on

L. A. Hiller and L. M. Isaacson, “Musical composition with a high-speed digital computer,” Journal of the Audio Engineering Society , vol. 6, no. 3, pp. 154–160, 1958
1958
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation , vol. 9, no. 8, pp. 1735–1780, 1997. [Online]. Available: https://doi.org/10.1162/neco.1997.9.8.1735
1997
Earlier work this paper cites.
D. Ponsford, G. Wiggins, and C. Mellish, “Statistical learning of harmonic movement,” Journal of New Music Research , vol. 28, no. 2, pp. 150–177, 1999
1999
Earlier work this paper cites.
D. Conklin, “Music generation from statistical models,” in SAISB Symposium on Artificial Intelligence and Creativity in the Arts and Sciences , 2003, pp. 30–35
2003
Earlier work this paper cites.
D. Cope, Computer Models of Musical Creativity . The MIT Press, 2005
2005
Earlier work this paper cites.
G. A. Wiggins, “A preliminary framework for description, analysis and comparison of creative systems,” Journal of Knowledge Based Systems , vol. 19, no. 7, pp. 449–458, 2006
2006
Earlier work this paper cites.
A. Smola, A. Gretton, L. Song, and B. Schölkopf, “A hilbert space embedding for distributions,” in Algorithmic Learning Theory , M. Hutter, R. A. Servedio, and E. Takimoto, Eds. Springer Berlin Heidelberg, 2007
2007
Earlier work this paper cites.
A. Eigenfeldt and P. Pasquier, “Realtime generation of harmonic progressions using controlled markov selection,” in International Conference on Computational Creativity , 2010, pp. 16–25
2010
Earlier work this paper cites.
S. Fukayama, K. Nakatsuma, S. Sako, T. Nishimoto, and S. Sagayama, “Automatic song composition from the lyrics exploiting prosody of the japanese language,” in International Conference of Sound and Music Computing , 2010, pp. 299–302
2010
Earlier work this paper cites.

Similar

K. Monteith, T. R. Martinez, and D. Ventura, “Automatic generation of melodic accompaniments for lyrics,” in Proceedings of the Third International Conference on Computational Creativity, 2012. , 2012, pp. 87–94
2012
Cited alongside, same era.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Networks,” arXiv e-prints , 2014
2014
Cited alongside, same era.
J. D. F. Rodriguez and F. J. Vico, “AI methods in algorithmic composition: A comprehensive survey,” CoRR , vol. abs/1402.0585, 2014
Original
2014
Cited alongside, same era.
M. Mirza and S. Osindero, “Conditional generative adversarial nets,” CoRR , vol. abs/1411.1784, 2014. [Online]. Available: http://arxiv.org/abs/1411.1784
Original
2014
Cited alongside, same era.

Then

O. Mogren, “C-RNN-GAN: continuous recurrent neural networks with adversarial training,” CoRR , vol. abs/1611.09904, 2016
Original
2016
Later among the works it cites.
L. Yu, W. Zhang, J. Wang, and Y. Yu, “Seqgan: Sequence generative adversarial nets with policy gradient,” 2016
2016
Later among the works it cites.
E. Jang, S. Gu, and B. Poole, “Categorical reparameterization with gumbel-softmax,” 2016
2016
Later among the works it cites.
J. Briot and F. Pachet, “Music generation by deep learning - challenges and directions,” CoRR , vol. abs/1712.04371, 2017. [Online]. Available: http://arxiv.org/abs/1712.04371
Original
2017
Later among the works it cites.
J. Wu, C. Hu, Y. Wang, X. Hu, and J. Zhu, “A hierarchical recurrent neural network for symbolic melody generation,” CoRR , vol. abs/1712.05274, 2017
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

S. J. Reddi, A. Ramdas, B. Poczos, A. Singh, and L. Wasserman, 2014

2014

Cited alongside, same era.

M. Scirea, G. A. B. Barros, N. Shaker, and J. Togelius, “Smug: Scientific music generator,” in Sixth International Conference on Computational Creativity , 2015, pp. 204–211

2015

Cited alongside, same era.

W. Bounliphone, E. Belilovsky, M. B. Blaschko, I. Antonoglou, and A. Gretton, “A Test of Relative Similarity For Model Selection in Generative Models,” arXiv e-prints , 2015

2015

Cited alongside, same era.

M. Ackerman and D. Loker, “Algorithmic songwriting with ALYSIA,” CoRR , vol. abs/1612.01058, 2016. [Online]. Available: http://arxiv.org/abs/1612.01058

2016

Cited alongside, same era.

Y. Yu, S. Tang, F. Raposo, and L. Chen, “Deep cross-modal correlation learning for audio and lyrics in music retrieval,” ACM Trans. Multimedia Comput. Commun. Appl. , vol. 15, no. 1

Cited in the paper.

https://en.wikipedia.org/wiki/Melody

Cited in the paper.

T. Anders and E. R. Miranda, “Constraint programming systems for modeling music theories and composition,” ACM Comput. Surv

Cited in the paper.

M. Delgado, W. Fajardo, and M. Molina-Solana, “Inmamusys: Intelligent multiagent music system,” Expert Syst. Appl

Cited in the paper.

2017

Later among the works it cites.

D. D. Johnson, “Generating polyphonic music using tied parallel networks,” in International Conference on Evolutionary and Biologically Inspired Music and Art , 2017, pp. 128–143

2017

Later among the works it cites.

H. Bao, S. Huang, F. Wei, L. Cui, Y. Wu, C. Tan, S. Piao, and M. Zhou, “Neural melody composition from lyrics,” CoRR , vol. abs/1809.04318, 2018. [Online]. Available: http://arxiv.org/abs/1809.04318

2018

Later among the works it cites.

H.-P. Lee, J.-S. Fang, and W.-Y. Ma, “iComposer: An automatic songwriting system for Chinese popular music,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)

2019

Closest in time.