P300 audio-visual speller

Abstract

The Farwell and Donchin matrix speller is well known as one of the highest performing brain-computer interfaces (BCIs) currently available. However, its use of visual stimulation limits its applicability to users with normal eyesight. Alternative BCI spelling systems which rely on non-visual stimulation, e.g. auditory or tactile, tend to perform much more poorly and/or can be very difficult to use. In this paper we present a novel extension of the matrix speller, based on flipping the letter matrix, which allows us to use the same interface for visual, auditory or simultaneous visual and auditory stimuli. In this way we aim to allow users to utilize the best available input modality for their situation, that is use visual + auditory for best performance and move smoothly to purely auditory when necessary, e.g. when disease causes the user's eyesight to deteriorate. Our results on seven healthy subjects demonstrate the effectiveness of this approach, with our modified visual + auditory stimulation slightly out-performing the classic matrix speller. The purely auditory system performance was lower than for visual stimulation, but comparable to other auditory BCI systems.