Audio-visual perception of mandarin tone in clear speech

Abstract

Clearly enunciated speech (relative to conversational, plain speech) involves articulatory and acoustic modifications that have been shown to enhance segmental intelligibility. However, little research has explored clear-speech effects on the perception of suprasegmental properties such as lexical tone, particularly involving visual (facial) perception. Moreover, research has not determined if visual tonal cues are linguistically significant, presumably because tone production does not rely on vocal tract configurations. Investigating clear-speech effects may help validate the contribution of visual information, since improved visual tone intelligibility in clear speech would indicate the linguistic relevance of visual cues. The present study tested this hypothesis by examining the intelligibility of clear and plain Mandarin tones by native (Mandarin) and non-native (English) perceivers with auditory/visual input modalities (AO: audio-only; VO: visual-only; and AV: audio-visual). Results showed small but significant clear-speech benefits in each modality for both natives and non-natives. However, while the natives revealed an overall visual gain and auditory dominance (AV>AO>VO), visual gain appeared to be less prominent in non-native perception (AV<AO>VO in plain, AV = AO>VO in clear). These results demonstrate clear-speech facilitation in visual as well as auditory perception, particularly with native perceivers, suggesting the existence of linguistically relevant visual tonal cues.