Causal Diffusion Models for Generalized Speech Enhancement

Abstract

In this work, we present a causal speech enhancement system that is designed to handledifferent types of corruptions. This paper is an extended version of our contribution to the “ICASSP 2023Speech Signal Improvement Challenge”. The method is based on a generative diffusion model which hasbeen shown to work well in scenarios beyond speech-in-noise, such as missing data and non-additivecorruptions. We guarantee causal processing with an algorithmic latency of 20 ms by modifying the networkarchitecture and removing non-causal normalization techniques. To train and test our model, we generate anew corrupted speech dataset which includes additive background noise, reverberation, clipping, packet loss,bandwidth reduction, and codec artifacts. We compare the causal and non-causal versions of our method toinvestigate the impact of causal processing and we assess the gap between specialized models trained on aparticular corruption type and the generalized model trained on all corruptions. Although specialized modelsand non-causal models have a small advantage, we show that the generalized causal approach does not sufferfrom a significant performance penalty, while it can be flexibly employed for real-world applications wheredifferent types of distortions may occur.