Fetching the paper…

Reading the bibliography…

2024

The Interspeech 2024 Challenge on Speech Processing Using Discrete Units

Chang, Xuankai, Shi, Jiatong, Tian, Jinchuan et al.

Original
PDF
Post on X

Understand

Representing speech and audio signals in discrete units has become a compelling alternative to traditional high-dimensional feature vectors.

Numerous studies have highlighted the efficacy of discrete units in various applications such as speech compression and restoration, speech recognition, and speech generation.
To foster exploration in this domain, we introduce the Interspeech 2024 Challenge, which focuses on new speech processing benchmarks using discrete units.
It encompasses three pivotal tasks, namely multilingual automatic speech recognition, text-to-speech, and singing voice synthesis, and aims to assess the potential applicability of discrete units in these tasks.

The Interspeech 2024 Challenge on Speech Processing Using Discrete Units · Around

Built on

A. Graves et al. , “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proc. ICML , 2006, pp. 369–376
2006
Earlier work this paper cites.
G. Hinton et al. , “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal Process. Mag. , vol. 29, no. 6, pp. 82–97, 2012
2012
Earlier work this paper cites.
A. Graves, “Sequence transduction with recurrent neural networks,” arXiv preprint arXiv:1211.3711 , 2012
Original
2012
Earlier work this paper cites.
J. Chorowski et al. , “Attention-based models for speech recognition,” Proc. NeurIPS , vol. 2015, pp. 577–585, 2015
2015
Earlier work this paper cites.
T. N. Sainath et al. , “Learning the speech front-end with raw waveform CLDNNs,” Proc. Interspeech , 2015
2015
Earlier work this paper cites.
V. Panayotov et al. , “Librispeech: an asr corpus based on public domain audio books,” in Proc. IEEE ICASSP , 2015, pp. 5206–5210
2015
Earlier work this paper cites.
Y. Qian et al. , “Very deep convolutional neural networks for noise robust speech recognition,” IEEE/ACM Trans. ASLP. , vol. 24, no. 12, pp. 2263–2276, 2016
2016
Earlier work this paper cites.
K. Ito and L. Johnson, “The LJ speech dataset,” https://keithito.com/LJ-Speech-Dataset/ , 2017
2017
Earlier work this paper cites.
L. Dong et al. , “Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition,” in Proc. IEEE ICASSP , 2018
2018
Earlier work this paper cites.
S. Watanabe et al. , “ESPnet: End-to-end speech processing toolkit,” in Proc. Interspeech , 2018, pp. 2207–2211
2018
Earlier work this paper cites.
A. Gulati et al. , “Conformer: Convolution-augmented Transformer for speech recognition,” in Proc. Interspeech . ISCA, 2020, pp. 5036–5040
2020
Earlier work this paper cites.
A. Baevski et al. , “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Proc. NeurIPS , vol. 33, pp. 12 449–12 460, 2020
2020
Earlier work this paper cites.
T. Hayashi and S. Watanabe, “Discretalk: Text-to-speech as a machine translation problem,” arXiv preprint arXiv:2005.05525 , 2020
Original
2020
Earlier work this paper cites.
T. Hayashi et al. , “ESPnet-TTS: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit,” in Proc. IEEE ICASSP . IEEE, 2020, pp. 7654–7658
2020
Earlier work this paper cites.
P. Lu et al. , “XiaoiceSing: A high-quality and integrated singing voice synthesis system,” Proc. Interspeech , 2020
2020
Earlier work this paper cites.
Y. Ren et al. , “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in Proc. ICLR , 2020
2020
Earlier work this paper cites.
P. Guo et al. , “Recent developments on espnet toolkit boosted by conformer,” in Proc. IEEE ICASSP , 2021, pp. 5874–5878
2021
Earlier work this paper cites.
W.-N. Hsu et al. , “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Trans. ASLP. , vol. 29, pp. 3451–3460, 2021
2021
Earlier work this paper cites.

Similar

K. Lakhotia et al. , “On generative spoken language modeling from raw audio,” Trans. ACL. , vol. 9, pp. 1336–1354, 2021
2021
Cited alongside, same era.
E. Kharitonov et al. , “Text-free prosody-aware generative spoken language modeling,” arXiv preprint arXiv:2109.03264 , 2021
Original
2021
Cited alongside, same era.
F. Kreuk et al. , “Textless speech emotion conversion using discrete and decomposed representations,” arXiv preprint arXiv:2111.07402 , 2021
Original
2021
Cited alongside, same era.
——, “ESPnet2-TTS: Extending the edge of tts research,” arXiv preprint arXiv:2110.07840 , 2021
Original
2021
Cited alongside, same era.
A. Polyak et al. , “Speech Resynthesis from Discrete Disentangled Self-Supervised Representations,” in Proc. Interspeech , 2021, pp. 3615–3619

Then

P. K. Rubenstein et al. , “Audiopalm: A large language model that can speak and listen,” arXiv preprint arXiv:2306.12925 , 2023
Original
2023
Later among the works it cites.
S. Maiti et al. , “Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks,” arXiv preprint arXiv:2309.07937 , 2023
Original
2023
Later among the works it cites.
M. Hassid et al. , “Textually pretrained speech language models,” arXiv preprint arXiv:2305.13009 , 2023
Original
2023
Later among the works it cites.
T. A. Nguyen et al. , “Generative spoken dialogue language modeling,” Trans. ACL. , vol. 11, pp. 250–266, 2023
2023
Later among the works it cites.
M. Kim et al. , “Many-to-many spoken language translation via unified speech and text representation learning with unit-to-unit translation,” arXiv preprint arXiv:2308.01831 , 2023

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2021

Cited alongside, same era.

S. Chen et al. , “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE JSTSP , vol. 16, no. 6, pp. 1505–1518, 2022

2022

Cited alongside, same era.

G. Maimon and Y. Adi, “Speaking style conversion with discrete self-supervised units,” arXiv preprint arXiv:2212.09730 , 2022

2022

Cited alongside, same era.

F. Kreuk et al. , “Audiogen: Textually guided audio generation,” arXiv preprint arXiv:2209.15352 , 2022

2022

Cited alongside, same era.

T. Saeki et al. , “UTMOS: Utokyo-sarulab system for VoiceMOS challenge 2022,” arXiv preprint arXiv:2204.02152 , 2022

2022

Cited alongside, same era.

W. C. Huang et al. , “The VoiceMOS Challenge 2022,” in Proc. Interspeech , 2022, pp. 4536–4540

2022

Cited alongside, same era.

Y. Zhang et al. , “VISinger: Variational inference with adversarial learning for end-to-end singing voice synthesis,” in Proc. IEEE ICASSP , 2022

2022

Cited alongside, same era.

Y. Wang et al. , “Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis,” in Proc. Interspeech , 2022, pp. 4242–4246

2022

Cited alongside, same era.

2023

Later among the works it cites.

J. Copet et al. , “Simple and controllable music generation,” arXiv preprint arXiv:2306.05284 , 2023

2023

Later among the works it cites.

W.-N. Hsu et al. , “Revise: Self-supervised speech resynthesis with visual input for universal and generalized speech regeneration,” in Proc. IEEE/CVF CVPR , 2023, pp. 18 795–18 805

2023

Later among the works it cites.

J. Shi et al. , “ML-SUPERB: Multilingual Speech Universal PERformance Benchmark,” in Proc. Interspeech , 2023, pp. 884–888

2023

Later among the works it cites.

X. Chang et al. , “Exploring speech recognition, translation, and understanding with discrete speech units: A comparative study,” arXiv preprint arXiv:2309.15800 , 2023

2023

Later among the works it cites.

T. A. Nguyen et al. , “Expresso: A benchmark and analysis of discrete expressive speech resynthesis,” arXiv preprint arXiv:2308.05725 , 2023

2023

Later among the works it cites.

B. Yan et al. , “ESPnet-ST-v2: Multipurpose spoken language translation toolkit,” in Proc. ACL , Jul. 2023

2023

Later among the works it cites.

L. Barrault et al. , “Seamless: Multilingual expressive and streaming speech translation,” arXiv preprint arXiv:2312.05187 , 2023

2023

Later among the works it cites.

C. Wang et al. , “Neural codec language models are zero-shot text to speech synthesizers,” arXiv preprint arXiv:2301.02111 , 2023

2023

Later among the works it cites.

D. Yang et al. , “Uniaudio: An audio foundation model toward universal audio generation,” arXiv preprint arXiv:2310.00704 , 2023

2023

Later among the works it cites.

J. Shi et al. , “Enhancing speech-to-speech translation with multiple TTS targets,” in Proc. IEEE ICASSP , 2023, pp. 1–5

2023

Later among the works it cites.

R. Kumar et al. , “High-fidelity audio compression with improved rvqgan,” Proc. NeurIPS , vol. 36, 2024

2024

Closest in time.

Y. Ai et al. , “APCodec: A neural audio codec with parallel amplitude and phase spectrum encoding and decoding,” arXiv preprint arXiv:2402.10533 , 2024

2024

Closest in time.