Restorative Speech Enhancement: A Progressive Approach Using SE and Codec Modules

Understand

In challenging environments with significant noise and reverberation, traditional speech enhancement (SE) methods often lead to over-suppressed speech, creating artifacts during listening and harming downstream tasks performance.

• To overcome these limitations, we propose a novel approach called Restorative SE (RestSE), which combines a lightweight SE module with a generative codec module to progressively enhance and restore speech quality.
• The SE module initially reduces noise, while the codec module subsequently performs dereverberation and restores speech using generative capabilities.
• We systematically explore various quantization techniques within the codec module to optimize performance.