An Investigation on Audio-Visual BCI: The Interaction Effect between Audio and Vision

Short Communication

Austin Public Health. 2016; 1(1): 1004.

An Investigation on Audio-Visual BCI: The Interaction Effect between Audio and Vision

Minqiang Huang, Xingyu Wang and Jing Jin*

Department of Advanced Control and Optimization for Chemical Processes, East China University of Science and Technology, China

*Corresponding author: Jing Jin, Department of Advanced Control and Optimization for Chemical Processes, East China University of Science and Technology, Ministry of Education, Shanghai, 200237, China

Received: June 29, 2016; Accepted: July 15, 2016; Published: July 18, 2016

Abstract

Brain Computer Interface (BCI) technology has been used to help disabled patients communicate or control external devices through brain activity. Recently, audio-visual Brain Computer Interfaces (BCIs) were proposed and these hybrid-modal BCIs got better performance than the single-modal BCIs. In this study, three patterns were designed to explore the relation between the auditory stimuli and the visual stimuli in an audio-visual BCI system. Reversely, the results showed that the patterns combing visual and auditory stimuli did not get better performance than pattern using visual patterns.

Keywords: Audio and vision; Visual BCI; Stimuli; Single-modal

Introduction

In recent years, BCIs relying on single modal such as visual or auditory have been developed to a bottleneck period. The visual BCIs can achieve quite high performance in items of classification accuracy and information bit rate, but visual BCIs demand that users have good sight and ability to control their eyeballs. Although auditory BCIs get rid of the visual requirements, comparing with visual BCIs, low classification accuracy of auditory BCIs is not a negligible shortcoming. To change the current situation, researchers begin to explore a hybrid-modal BCI. Mostly, hybrid-modal BCIs are focus on combing visual and auditory sense.

Method and Materials

Subjects

Eleven healthy people (7 males and 4 females, ages 24-28) participated in this experiment, which was approved by the local ethics committee. All of the participants were right handed and they were given information about the experiment (without exposing experiment intention).

Stimuli

This study used visual and auditory stimuli. Three visual stimuli were presented in Figure 1. The frequency spectrums of auditory stimuli were presented in Figure 2. Each visual stimulus had a corresponding auditory stimulus. Stimulus Onset Asynchrony (SOA) was 600ms. The presentation of each stimulus lasted 400ms. The volume of auditory stimuli was unified.