2022

SepIt: Approaching a Single Channel Speech Separation Bound

Lutati, Shahar, Nachmani, Eliya, Wolf, Lior

Understand

We present an upper bound for the Single Channel Speech Separation task, which is based on an assumption regarding the nature of short segments of speech.

Using the bound, we are able to show that while the recent methods have made significant progress for a few speakers, there is room for improvement for five and ten speakers.
We then introduce a Deep neural network, SepIt, that iteratively improves the different speakers' estimation.
At test time, SpeIt has a varying number of iterations per test sample, based on a mutual information criterion that arises from our analysis.

Built on

C. R. Rao, Information and the Accuracy Attainable in the Estimation of Statistical Parameters . New York, NY: Springer New York, 1992, pp. 235–247. [Online]. Available: https://doi.org/10.1007/978-1-4612-0919-5_16
1992
Earlier work this paper cites.
J. Garofolo, D. Graff, D. Paul, and D. Pallett, “Csr-i (wsj0) complete ldc93s6a,” Web Download. Philadelphia: Linguistic Data Consortium , vol. 83, 1993
1993
Earlier work this paper cites.
H. Sahlin and U. Lindgren, “The asymptotic cramer-rao lower bound for blind signal separation,” in Proceedings of 8th Workshop on Statistical Signal and Array Processing , 1996, pp. 328–331
1996
Earlier work this paper cites.
N. Brunel and J.-P. Nadal, “Mutual information, fisher information, and population coding,” Neural computation , vol. 10, pp. 1731–57, 11 1998
1998
Earlier work this paper cites.
J. Jensen, I. Batina, R. Hendriks, and H. Richard, “A study of the distribution of time-domain speech samples and discrete fourier coefficients,” 01 2005
2005
Earlier work this paper cites.
E. Doron, A. Yeredor, and P. Tichavsky, “Cramér–rao-induced bound for blind separation of stationary parametric gaussian sources,” IEEE Signal Processing Letters , vol. 14, no. 6, pp. 417–420, 2007
2007
Earlier work this paper cites.
H. Erdogan, J. R. Hershey, S. Watanabe, and J. Le Roux, “Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2015, pp. 708–712
2015
Earlier work this paper cites.
J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clustering: Discriminative embeddings for segmentation and separation,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2016, pp. 31–35
2016
Earlier work this paper cites.

Similar

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2017
2017
Cited alongside, same era.
Y. Luo and N. Mesgarani, “Tasnet: time-domain audio separation network for real-time, single-channel speech separation,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2018, pp. 696–700
2018
Cited alongside, same era.
J. L. Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “Sdr - half-baked or well done?” 2018
2018
Cited alongside, same era.
——, “Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation,” IEEE/ACM transactions on audio, speech, and language processing , vol. 27, no. 8, pp. 1256–1266, 2019
2019
Cited alongside, same era.
Y. Luo, Z. Chen, and T. Yoshioka, “Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech separation,” arXiv preprint arXiv:1910.06379 , 2019
Original
2019
Cited alongside, same era.
Y. Luo, Z. Chen, and T. Yoshioka, “Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech separation,” 2020
2020
Cited alongside, same era.
E. Nachmani, Y. Adi, and L. Wolf, “Voice separation with an unknown number of multiple speakers,” 2020
2020
Cited alongside, same era.

Then

J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vincent, “Librimix: An open-source dataset for generalizable speech separation,” arXiv preprint arXiv:2005.11262 , 2020
Original
2020
Later among the works it cites.
N. Zeghidour and D. Grangier, “Wavesplit: End-to-end speech separation by speaker clustering,” arXiv preprint arXiv:2002.08933 , 2020
Original
2020
Later among the works it cites.
V. Kautsky, Z. Koldovsky, P. Tichavsky, and V. Zarzoso, “Cramér–rao bounds for complex-valued independent component extraction: Determined and piecewise determined mixing models,” IEEE Transactions on Signal Processing , vol. 68, p. 5230–5243, 2020. [Online]. Available: http://dx.doi.org/10.1109/TSP.2020.3022827
2020
Later among the works it cites.
C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, and J. Zhong, “Attention is all you need in speech separation,” 2021
2021
Later among the works it cites.
X. Hu, K. Li, W. Zhang, Y. Luo, J.-M. Lemercier, and T. Gerkmann, “Speech separation using an asynchronous fully recurrent convolutional neural network,” Advances in Neural Information Processing Systems , vol. 34, 2021
2021
Later among the works it cites.
S. Dovrat, E. Nachmani, and L. Wolf, “Many-speakers single channel speech separation with optimal permutation training,” 2021
2021
Later among the works it cites.
Z. Yao, W. Pei, F. Chen, G. Lu, and D. Zhang, “Stepwise-refining speech separation network via fine-grained encoding in high-order latent domain,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2022
2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…