Modeling Irregular Voice in End-to-End Speech Synthesis via Speaker Adaptation

Abstract

End-to-end text-to-speech (TTS) synthesizers may not create a speech similar to the target speaker when the adaptation data is limited or/and chosen randomly. Creaky voice might occur frequently, depending on the speaker and the context. This paper uses speaker adaptation to model creaky voice in speech synthesis. We adapted FastSpeech 2 with four target speakers by selecting the adaptation data based on the occurrence of creaky phonation: 1) sentences with frequent creaky voice, 2) randomly chosen sentences, and 3) sentences with few creaky voice. In an objective evaluation, the proposed model successfully modeled creaky voice using data selection (1), producing speech with more creakiness than the other data selections. A subjective test revealed that these frequent creaky voice synthesized samples (for the average of four speakers) obtained slightly less preference than the synthesized speech from a few creaky voice adaptation sentences. Irregular voice models might contribute to building emotional and personalized speech synthesis.