An attention based densely connected U-NET with convolutional GRU for speech enhancement

Abstract

Speech enhancement performance is highly reliant on on the efficacy of representative features extracted from noisy speech. However, SE frequently experiences problems with semantically irrelevant as well as silence frames. This work aims to provide a hybrid structure in time domain. This framework enables temporal context aggregation using densely connected U-Net with attention-based skip links. The attention mechanism makes the model focus on the semantically relevant and important parts of the raw waveform while keeping more focus on the voice activity region. Dilated convolutions help with context aggregation and dense connection provides detailed target information by passing through a variety of layers at various dilation rates. To represent the correlation between neighboring noisy speech frames, a two Layer GRU is added in the bottleneck of the encoder-decoder. Extensive experimentations have demonstrated that the proposed framework consistently advances the performance over existing baselines across two widely used objective metrics such as STOI (short-time objective intelligibility), and PESQ (perceptual evaluation of the speech quality).