2020

NHSS: A Speech and Singing Parallel Database

Sharma, Bidisha, Gao, Xiaoxue, Vijayan, Karthika et al.

Understand

We present a database of parallel recordings of speech and singing, collected and released by the Human Language Technology (HLT) laboratory at the National University of Singapore (NUS), that is called NUS-HLT Speak-Sing (NHSS) database.

We release this database to the public to support research activities, that include, but not limited to comparative studies of acoustic attributes of speech and singing signals, cooperative synthesis of speech and singing voices, and speech-to-singing conversion.
This database consists of recordings of sung vocals of English pop songs, the spoken counterpart of lyrics of the songs read by the singers in their natural reading manner, and manually prepared utterance-level and word-level annotations.
The audio recordings in the NHSS database correspond to 100 songs sung and spoken by 10 singers, resulting in a total of 7 hours of audio data.

Built on

doi:10.21437/Interspeech.2019-1925
B. Sharma, R. K. Das, H. Li, On the Importance of Audio-Source Separation for Singer Identification in Polyphonic Music, in: INTERSPEECH, 2019, pp. 2020–2024 · 1925
Earlier work this paper cites.
T. Saitou, N. Tsuji, M. Unoki, M. Akagi, Analysis of acoustic features affecting “singing-ness" and its application to singing-voice synthesis from speaking-voice, in: INTERSPEECH, 2004, pp. 1925–1928
1928
Earlier work this paper cites.
J. Sundberg, The level of the ‘singing formant’ and the source spectra of professional bass singers, Quarterly Progress and Status Report: STL-QPSR 11 (4) (1970) 21–39
1970
Earlier work this paper cites.
P. Oncley, Frequency, amplitude, and waveform modulation in the vocal vibrato, The Journal of the Acoustical Society of America 49 (1A) (1971) 136–136
1971
Earlier work this paper cites.
L. Rabiner, On the use of autocorrelation analysis for pitch detection, IEEE transactions on acoustics, speech, and signal processing 25 (1) (1977) 24–33
1977
Earlier work this paper cites.
J. M. Grey, J. W. Gordon, Perceptual effects of spectral modifications on musical timbres, The Journal of the Acoustical Society of America 63 (5) (1978) 1493–1500
1978
Earlier work this paper cites.
H. Fujisaki, Dynamic characteristics of voice fundamental frequency in speech and singing, in: The production of speech, Springer, 1983, pp. 39–55
1983
Earlier work this paper cites.
R. Leonard, A database for speaker-independent digit recognition, in: International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vol. 9, IEEE, 1984, pp. 328–331
1984
Earlier work this paper cites.
J. Sundberg, T. D. Rossing, The science of singing voice, Vol. 87, ASA, 1990
1990
Earlier work this paper cites.
H. Sakoe, S. Chiba, A. Waibel, K. Lee, Dynamic programming algorithm optimization for spoken word recognition, Readings in speech recognition 159 (1990) 224
1990
Earlier work this paper cites.
J. J. Godfrey, E. C. Holliman, J. McDaniel, SWITCHBOARD: Telephone speech corpus for research and development, in: International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vol. 1, 1992, pp. 517–520
1992
Earlier work this paper cites.
D. B. Paul, J. M. Baker, The design for the Wall Street Journal-based CSR corpus, in: Proceedings of the workshop on Speech and Natural Language, Association for Computational Linguistics, 1992, pp. 357–362
1992
Earlier work this paper cites.
J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett, N. LDahlgren, V. Zue, TIMIT acoustic phonetic continuous speech corpus, Linguistic Data Consortium (1993)
1993
Earlier work this paper cites.
J. Sundberg, I. R. Titze, R. Scherer, Phonatory control in male singing: A study of the effects of subglottal pressure, fundamental frequency, and mode of phonation on the voice source, Journal of Voice 7 (1) (1993) 15 – 29
1993
Earlier work this paper cites.
C. Veaux, J. Yamagishi, K. MacDonald, CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit, University of Edinburgh. The Centre for Speech Technology Research (CSTR) doi:http://dx.doi.org/10.7488/ds/1994
1994
Earlier work this paper cites.
S. Werner, E. Keller, Fundamentals of speech synthesis and speech recognition: Basic concepts, state of the art, and future challenges, chapter prosodic aspects of speech, E. Keller (1994) 23–40
1994
Earlier work this paper cites.
H.-G. Hirsch, D. Pearce, The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions, in: ASR2000-Automatic Speech Recognition: Challenges for the new Millenium ISCA Tutorial and Research Workshop (ITRW), 2000
2000
Earlier work this paper cites.
K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, T. Kitamura, Speech parameter generation algorithms for HMM-based speech synthesis, in: International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vol. 3, 2000, pp. 1315–1318
2000
Earlier work this paper cites.
G. Tzanetakis, A. Ermolinskyi, P. Cook, Pitch histograms in audio and symbolic music information retrieval, Journal of New Music Research 32 (2) (2003) 143–152
2003
Earlier work this paper cites.
J. Kominek, A. W. Black, The CMU Arctic speech databases, in: Fifth ISCA workshop on speech synthesis (SSW5), 2004, pp. 223–224
2004
Earlier work this paper cites.
P. Knees, E. Pampalk, G. Widmer, Artist classification with web-based data, in: International Society for Music Information Retrieval Conference (ISMIR), 2004, pp. 517–524
2004
Earlier work this paper cites.
J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, W. Kraaij, M. Kronenthal, et al., The AMI meeting corpus: A pre-announcement, in: International workshop on machine learning for multimodal interaction, Springer, 2005, pp. 28–39
2005
Earlier work this paper cites.
M. Schedl, P. Knees, G. Widmer, Investigating web-based approaches to revealing prototypical music artists in genre taxonomies, in: 1st International Conference on Digital Information Management, 2006, pp. 519–524
2006
Earlier work this paper cites.
C. McKay, D. McEnnis, I. Fujinaga, A large publicly accessible prototype audio database for music research, in: International Society for Music Information Retrieval Conference (ISMIR), 2006, pp. 160–163
2006
Earlier work this paper cites.
Y. Ohishi, M. Goto, K. Itou, K. Takeda, On the human capability and acoustic cues for discriminating the singing and the speaking voices, 2006, pp. 1831–1837
2006
Earlier work this paper cites.
D. P. Ellis, Classifying music audio with timbral and chroma features, in: International Society for Music Information Retrieval Conference (ISMIR), 2007, pp. 339–340
2007
Earlier work this paper cites.
A. F. Martin, C. S. Greenberg, NIST 2008 speaker recognition evaluation: Performance across telephone and room microphone channels, in: INTERSPEECH, 2009, pp. 2579–2582
2009
Earlier work this paper cites.
C. L. Hsu, J. S. R. Jang, On the improvement of singing voice separation for monaural recordings using the MIR-1K dataset, IEEE Transactions on Audio, Speech, and Language Processing 18 (2) (2009) 310–319
2009
Earlier work this paper cites.

Similar

M. A. Dayme, Dynamics of the singing voice, Springer Science & Business Media, 2009
2009
Cited alongside, same era.
S. Sing!, Smule digital archive mobile performances(damp), https://ccrma.stanford.edu/damp/ , [Online; accessed 08-January-2020] (2010)
2010
Cited alongside, same era.
A. Mesaros, T. Virtanen, Automatic recognition of lyrics in singing, EURASIP Journal on Audio, Speech, and Music Processing (2010) ArticleID : 546047
2010
Cited alongside, same era.
T. L. New, M. Dong, P. Chan, X. Wang, B. Ma, H. Li, Voice conversion: From spoken vowels to singing vowels, in: IEEE International Conference on Multimedia and Expo (ICME), 2010, pp. 1421–1426
2010
Cited alongside, same era.
N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, P. Ouellet, Front-end factor analysis for speaker verification, IEEE Transactions on Audio, Speech, and Language Processing 19 (4) (2010) 788–798
2010
Cited alongside, same era.
S. J. King, V. Karaiskos, The blizzard challenge 2011, 2011
2011
Cited alongside, same era.
T. Bertin-Mahieux, D. P. Ellis, B. Whitman, P. Lamere, The million song dataset, in: International Society for Music Information Retrieval Conference (ISMIR), 2011, pp. 591–596
2011
Cited alongside, same era.
D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, et al., The Kaldi speech recognition toolkit, in: IEEE workshop on automatic speech recognition and understanding, no. EPFL-CONF-192584, IEEE Signal Processing Society, 2011
2011
Cited alongside, same era.
L. Cen, M. Dong, P. Chan, Segmentation of speech signals in template-based speech to singing conversion, in: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, 2011, pp. 1–4
2011
Cited alongside, same era.
N. Henrich, J. Smith, J. Wolfe, Vocal tract resonances in singing: Strategies used by sopranos, altos, tenors, and baritones, The Journal of the Acoustical Society of America 129 (2) (2011) 1024–1035
2011
Cited alongside, same era.
L. Cen, M. Dong, P. Chan, Template-based personalized singing voice synthesis, in: International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2012, pp. 4509–4512
2012
Cited alongside, same era.
S. J. King, V. Karaiskos, The blizzard challenge 2013, 2013
2013
Cited alongside, same era.

Then

M. Morise, F. Yokomori, K. Ozawa, WORLD: a vocoder-based high-quality speech synthesis system for real-time applications, IEICE TRANSACTIONS on Information and Systems 99 (7) (2016) 1877–1884
2016
Later among the works it cites.
Z. Wu, O. Watts, S. King, Merlin: An open source neural network speech synthesis system., in: The 9th ISCA Speech Synthesis Workshop (SSW), 2016, pp. 202–207
2016
Later among the works it cites.
K. Ito, The LJ speech dataset, Available at, https://keithito.com/LJ-Speech-Dataset/ (2017)
2017
Later among the works it cites.
Z. Rafii, A. Liutkus, F.-R. Stöter, S. I. Mimilakis, R. Bittner, The MUSDB18 corpus for music separation (Dec. 2017). URL https://doi.org/10.5281/zenodo.1117372
2017
Later among the works it cites.
A. Aljanaki, Y.-H. Yang, M. Soleymani, Developing a benchmark for emotional analysis of music, PloS one 12 (3) (2017) 1–22
2017
Later among the works it cites.
M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, M. Sonderegger, Montreal forced aligner: Trainable text-speech alignment using kaldi., in: INTERSPEECH, 2017, pp. 498–502
2017
Later among the works it cites.
B. Sharma, S. M. Prasanna, Vowel onset point detection using sonority information., in: INTERSPEECH, 2017, pp. 444–448
2017
Later among the works it cites.
B. Sharma, S. M. Prasanna, Enhancement of spectral tilt in synthesized speech, IEEE Signal Processing Letters 24 (4) (2017) 382–386
2017
Later among the works it cites.
G. Meseguer-Brocal, A. Cohen-Hadria, G. Peeters, DALI: A large Dataset of synchronized Audio, LyrIcs and notes, automatically created using teacher-student machine learning paradigm, in: International Society for Music Information Retrieval Conference (ISMIR), 2018, pp. 431–437
2018
Later among the works it cites.
C. Gupta, R. Tong, H. Li, Y. Wang, Semi-supervised lyrics and solo-singing alignment, in: International Society for Music Information Retrieval Conference (ISMIR), 2018, pp. 600–607
2018
Later among the works it cites.
X. Gao, B. Sisman, R. K. Das, K. Vijayan, NUS-HLT spoken lyrics and singing (SLS) corpus, in: International Conference on Orange Technologies (ICOT), 2018, pp. 1–6
2018
Later among the works it cites.
K. Vijayan, H. Li, T. Toda, Speech-to-singing voice conversion: The challenges and strategies for improving vocal conversion processes, IEEE Signal Processing Magazine 36 (1) (2018) 95–102
2018
Later among the works it cites.
Montreal-forced-aligner, https://github.com/MontrealCorpusTools/Montreal-Forced-Aligner , [Online; accessed 26-Nobvember-2018]
2018
Later among the works it cites.
B. Sharma, Improving quality of statistical parametric speech synthesis using sonority information, Ph.D. thesis (2018)
2018
Later among the works it cites.
K. Vijayan, X. Gao, H. Li, Analysis of speech and singing signals for temporal alignment, in: IEEE Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2018, pp. 1893–1898
2018
Later among the works it cites.
D. Bogdanov, A. Porter, H. Schreiber, J. Urbano, S. Oramas, The AcousticBrainz genre dataset: Multi-source, multi-level, multi-label, and large-scale, in: International Society for Music Information Retrieval Conference (ISMIR), 2019, pp. 360–367
2019
Later among the works it cites.
Y. Shi, J. Zhou, Y. Long, Y. Li, H. Mao, Addressing text-dependent speaker verification using singing speech, Applied Sciences 9 (13) (2019) 2636
2019
Later among the works it cites.
doi:10.5281/zenodo.3241566
Y. Shi, J. Zhou, Y. Long, Y. Li, H. Mao, Rss database (Jun. 2019) · 2019
Later among the works it cites.
C. Gupta, K. Vijayan, B. Sharma, X. Gao, H. Li, NUS speak-to-sing : A web platform for personalized speech-to-singing conversion, in: Show & Tell, INTERSPEECH, 2019, pp. 2376–2377
2019
Later among the works it cites.
X. Gao, X. Tian, R. K. Das, Y. Zhou, H. Li, Speaker-independent spectral mapping for speech-to-singing conversion, in: IEEE Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2019, pp. 159–164
2019
Later among the works it cites.
B. Sharma, C. Gupta, H. Li, Y. Wang, Automatic lyrics-to-audio alignment on polyphonic music using singing-adapted acoustic models, in: International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019, pp. 396–400
2019
Later among the works it cites.
B. Sharma, Y. Wang, Automatic evaluation of song intelligibility using singing adapted stoi and vocal-specific features, IEEE/ACM Transactions on Audio, Speech, and Language Processing 28 (2019) 319–331
2019
Later among the works it cites.
B. Sharma, H. Li, A combination of model-based and feature-based strategy for speech-to-singing alignment, in: INTERSPEECH, 2019, pp. 624–628
2019
Later among the works it cites.
A. Nagrani, J. S. Chung, W. Xie, A. Zisserman, Voxceleb: Large-scale speaker verification in the wild, Computer Speech & Language 60 (2020) 101027
2020
Closest in time.
P. Boersma, D. Weenink, Praat: doing phonetics by computer, http://www.fon.hum.uva.nl/praat/ , [Online; accessed 08-April-2020]
2020
Closest in time.
C. Gupta, E. Yılmaz, H. Li, Automatic lyrics alignment and transcription in polyphonic music: Does background music help?, in: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 496–500
2020
Closest in time.
C. Gupta, H. Li, Y. Wang, Automatic leaderboard: Evaluation of singing quality without a standard reference, IEEE/ACM Transactions on Audio, Speech, and Language Processing 28 (2020) 13–26
2020
Closest in time.
G. Seshadri, B. Yegnanarayana, Perceived loudness of speech based on the characteristics of glottal excitation source, The Journal of the Acoustical Society of America 126 (4) (2009) 2061–2071
2071
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…