Max-Pooling Based Self-Attention with Transformer for Speaker Verification

Abstract

Transformer has become predominant in many natural language processing (NLP) tasks for its powerful long-term sequence processing abilities. As the central idea of Transformer, self-attention mechanism is originally proposed to model global information for textual sequences. However, discriminating acoustic feature sequences from different speakers mostly rely on local information, which makes Transformer less competitive in the speaker verification task. We alleviate this limitation with a max-pooling based self-attention mechanism to enlarge the receptive field of the attention heads thus to better capture local information. Besides, we also introduce and compare position-based and content-based self-attention mechanism to self-attention and explore different frame-level pooling methods for speaker embeddings. Experiments conducted on AISHEL-1 and LibriSpeech datasets demonstrate that the method we proposed accomplishes the most excellent performance with statistic attentive pooling (SAP) compared with the original Transformer baseline systems.