Kuriakose S; Pradeep Raj KB; Lahiri U

Research Article

Austin J Autism & Relat Disabil. 2016; 2(2): 1016.

Design of Performance-Sensitive Adaptive Response Technology for Children with Autism: Usability Study

Kuriakose S*, Pradeep Raj KB and Lahiri U

Electrical Engineering, IIT Gandhinagar, Gujarat, India

*Corresponding author:Kuriakose S, Electrical Engineering, IIT Gandhinagar, Gujarat, India

Received: Febraury 18, 2016; Accepted: March 22, 2016; Published: March 28, 2016

Abstract

Autism is characterized by deficits in social communication and imagination. Conventional intervention techniques, though powerful, have limitations, e.g., limited availability of trained resources, limitations in simulating variations in social situations at the therapist’s clinic, etc. Several researchers have been studying technology-assisted techniques, e.g., Virtual Reality (VR). In our study, we have explored the potential of VR-based system that can adapt itself in an individualized manner based on one’s task performance, to offer social contexts that requires one to adhere to social etiquettes during navigation (Interaction Task-I) in a social environment and interact with virtual peers exhibiting subtle emotional expressions (Interaction Task-II) to complete a social task. Additionally, our system has been augmented with real -time physiological data acquisition capability in a synchronized manner. Results of a preliminary usability study, designed as a proof-of-concept application, indicates the potential of our VR-based system to systematically administer some of the aspects of social communication among individuals thereby indicating varying implications on both performance and physiological indices that in turn can serve as a complementary measure in the hands of the interventionist by detecting potential physiology -related biomarkers of one’s anxiety while interacting with VR-based social situations.

Keywords: Autism, Virtual Reality, Physiology, Anxiety

Abbreviations

ASD: Autism Spectrum Disorder; PART: Performance-Sensitive Adaptive Response Technology; SCQ: Social Communication Questionnaire; SRS: Social Responsiveness Scale; TD: Typically Developing; SCAS: Spence Children’s Anxiety Rating Scale; IDEA: Individuals with Disabilities Education Act; PPG: Pulseplethysmogram; EDA: Electrodermal Activity; SKT: Skin Temperature

Introduction

Autism, a behaviorally defined syndrome, is characterized by core deficits in social interaction, communication and fixed or repetitive behavior [1]. These core deficits in social communication adversely affect one’s adaptive independence. Although the etiology of autism and the characteristics responsible for differential response to treatment is not yet well understood, there is an increasing consensus that intensive behavioral and educational programs can strengthen the social communication skills and significantly improve long term outcomes for individuals with Autism Spectrum Disorder (ASD) [2,3]. For effective and intensive intervention, one-to-one therapy requires long hours from qualified therapists/clinicians, who are often rarely available or are beyond the financial resources of families [4]. Therefore, high quality behavioral intervention is often inaccessible to ASD population. Motivated by the critical need of intensive intervention services, particularly in developing countries like India, emerging technology may provide an alternative assistive solution. Given the promise of observation -based conventional intervention techniques where the therapist carefully monitors one’s performance in a task to adjust her intervention strategy, the purpose of our present research is to design a technology -assisted platform (Performancesensitive Adaptive Response Technology (PART) henceforth) that is capable of adapting itself based on one’s task performance in an individualized manner. Additionally, in conventional settings, the therapist also monitors one’s affective states, e.g., anxiety, for effective floor-time therapy [5] that is often subjective in nature. In our present research, we hypothesize that the PART system has potential to have implications on one’s task performance, thereby helping to learn at least some of the aspects of social communication and also identify biomarkers of one’s anxiety level while serving as a complementary tool for the therapist by detecting anxiety-provoking elements of social interaction in an individualized manner. Here we have designed a Usability Study with PART system using a Virtual Reality (VR) programming platform, as proof-of-concept application and not as a full-fledged intervention system.

Role of Virtual Reality (VR) in autism intervention

A wide array of technology-assisted intervention tools for adolescents with autism are explored, namely, computer technology [6], VR [7], and robotic systems [8]. Among these solutions, we chose VR, because of its malleability, controllability, modifiable sensory stimulation, individualized approach, safety and potential reduction of problematic aspects of human interaction, particularly during initial skill training [4,9]. The flexible VR-based technology with the ability to mimic real environments in terms of imagery and contexts [4] can offer an efficient generalization of skills from the VR environment to the real world. Also, advanced computing technology has made VR-based systems readily available and affordable [9]. These attributes make VR a suitable candidate for designing skill training platforms for ASD.

Literature review shows pioneering work by researchers on the applicability of VR-based platform for addressing daily living skills for adolescents with autism [4,10]. Studies have shown the use of VR addressing different aspects of social communication, e.g., social context, subtle social cues, social interaction, facial emotional expression [8], navigation in a social scenario [4] etc. in an isolated way. Presently available VR-based systems for individuals with ASD are capable of modifying tasks based only on objective performance characteristics (i.e., correct or incorrect) of responses [11,12] with no structured feedback from facilitators that can serve as cues to the participants while interacting with the social scenario. Also, some of these pioneering research studies have not devised mechanisms to identify physiology-related biomarkers of one’s anxiety that might be informative to the interventionist /caregiver regarding elements of social interaction that are anxiety -provoking for individuals with ASD. This is important, since individuals with ASD are restricted in their explicit expression of their anxiety. In fact, this might help to identify areas of social interaction where an individual would benefit from intervention/skill training not possible with currently-available VR-based studies. In our present work, we have designed a usability study with a PART system to expose our participants to different aspects of social communication in an integrated and structured manner accompanied with virtual peers who act like facilitators. Here we expose our participants to both navigation-related social norms and etiquettes, and also to subtle emotional aspects of social communication with potential to identify physiology-related biomarkers of one’s anxiety.

Choice of anxiety as target affective state

Here we wanted to understand the feasibility of using physiology -related biomarkers as potential indicators of one’s anxiety, a critical concern especially among clinical samples of autism. Increased anxiety diminishes the chances of effective learning [13]. Specifically, it might happen that a participant is performing well, but the urgency of demonstrating good performance might often lead to increased anxiety. This is particularly true for individuals with ASD who are often prone to elevated anxiety levels [14]. Also the manifestation of increased anxiety might be differentiated given the spectrum nature of Autism. This manifestation can be explicit, or implicit, or combination of these. Individuals with ASD are often restricted in explicit expression of their affective state [15], thereby placing limitations on traditional observation - based approaches where alternative technologies can be employed.

Role of physiology-based technology as viomarkers of anxiety

Many children with ASD are capable of yielding correct performance on objective task measures. However, it is their vulnerabilities surrounding elements of social communication, related to navigation or deciphering subtle emotional cues are so closely tied to their functional social impairments [16]. Added to this, individuals with ASD are often highly anxious [16]. Due to deficits in explicit expression of their anxiety level, estimation of anxiety becomes difficult. Here we tried to understand the potential of physiology-related biomarkers to act as indicators of one’s anxiety level while being exposed to both navigation-related and subtle emotional aspects of social communication.

The physiological signals being continuously available and not directly impacted by communication deficits can be alternatively used as markers of one’s anxiety level [17]. For example, recently, Kushki et al. in their non-VR based research involving both typical adults and individuals with ASD have shown that one’s skin temperature, electrodermal and cardiovascular activity [18] vary with the anxiety level. Specifically, activation of sympathetic nervous system with increased anxiety is manifested as increased heart rate (cardiovascular activity), increased activity of eccrine sweat glands (electrodermal activity) and decreased skin temperature [18]. This also holds good in VR-based interaction where researchers have shown that physiological signals can be evoked by different amounts in the presence of virtual environments [19,20] and the transition from one affective state to another is accompanied by dynamic shifts in indicators of Autonomic Nervous System activation [13]. Studies have reported the possibility of using VR -based system for treating patients with anxiety disorders where anxiety level is detected using heart rate, electrodermal activity, skin temperature, etc. [21]. However, none of these pioneering research studies have investigated the implications on one’s physiology as biomarkers of anxiety, when exposed to VR -based tasks that can address aspects of adhering to navigation-related social norms and additionally exploring the effect of subtle context -relevant emotional cues that are critical for effective social conversation.

Purpose of Study

In our present research, we have designed a usability study, as proof -of-concept application using the VR platform. In this, we exposed our participants to various aspects of social communication in a systematic and integrated manner, having components of navigation-related social etiquettes and context-relevant emotional expressions. The PART system was designed to be sensitive to one’s individualized performance. Additionally, we acquired the participant’s physiological signals in a time -synched manner along with VR-based task progression. Thus we intend to understand the manifestation of anxiety as well as variations in performance by dissociating a particular social context into subtask s of navigationrelated and emotion-related social context. Specifically, here we wanted to address following questions to help us to understand the potential of PART system:

Is the PART system acceptable to our target population? Also, what is the efficacy of the PART system to their typically developing (TD) counterparts?
Does PART system offering tasks of varying challenges while exposing participants to different social aspects have the potential to cause variations in one’s task performance?
Is the PART system capable of causing variations in one’s physiological indices and if so are these variations differentiated based on anxiety levels for different aspects of social communication?

Method

Participants

We designed a usability study as proof-of-concept application in which six adolescents with ASD (ASD1-ASD6) and six TD (TD1- TD6) adolescents participated. The participant’s ID number (e.g., ASD1) was given in the order in which they came for the study. The ASD participants (n=6; Age: 16±4.5 (M±SD)) were selected from Pearl Special Needs Foundation and KGP children’s hospital, Gujarat, India. This school and hospital situated in and around Ahmedabad, Gujarat (western part of India) is well-known for their services. They provide education/intervention services to individuals with developmental disorders coming from families of different socio-economic backgrounds with children being Hindi/Gujarati speaking, but conversant in English. The TD participants (n=6; Age: 16±3 (M±SD)) were recruited from a neighborhood regular school. The inclusion criteria for participants with ASD required them to be high-functioning with some preliminary diagnosis of autism measures. We screened the participants for core autism-related symptoms using Social Responsiveness Scale (SRS) [22], and the Social Communication Questionnaire (SCQ) [23] administered in the presence of the therapist / clinician involved in our study. Table 1 shows the participants’ characteristics. All the TD participants were below the clinical thresholds in all the SRS and SCQ measures. All the ASD participants except ASD1 were above in either/both of the clinical cutoff scores for SRS and SCQ indicating that they were in the autism range. For ASD1, the SRS score showed that he was marginally in the autism range. Again, the SCQ score indicated that ASD1 was not in the clinical category. However, for ASD1, we found from his medical report from a reputed nearby hospital which used Individuals with Disabilities Education Act (IDEA) scale [24] that he was in the clinical range. Literature shows that the IDEA scale can also be used to report autism measures [24]. Additionally, ASD1 had the highest Spence Children’s Anxiety Rating Scale (SCAS) score [25]. The T-scores Table 1 indicates that TD participants on an average were less anxious than their ASD counterparts. We did not have access to participants’ IQ-related scores. However, all the participants were enrolled in a Special Needs (for ASD) and regular (for TD) School, where teachers selected the participants based on their impression about having an IQ above average.

Table 1: Participant Characteristics.




  
    ID
    Age (y)
    SRS(T-score)
    SCQ(T-score)
    SCAS(T-score)
  
  
    ASD1(M)
    14
    58
    10
    61
  
  
    ASD2 (F)
    23
    60
    13
    46
  
  
    ASD3 (F)
    19
    75
    18
    55
  
  
    ASD4 (M)
    12
    61
    11
    58
  
  
    ASD5 (M)
    19
    65
    18
    57
  
  
    ASD6 (F)
    12
    60
    13
    40
  
  
    M (SD)
    16.5±4.5
    63.17±6.24
    13.83±3.43
    52.83± 8.08
  
  
    TD1 (F)
    13
    49
    9
    42
  
  
    TD2 (M)
    18
    45
    4
    50
  
  
    TD3 (M)
    19
    44
    4
    43
  
  
    TD4 (M)
    13
    48
    12
    41
  
  
    TD5 (M)
    19
    51
    5
    54
  
  
    TD6 (M)
    14
    50
    9
    58
  
  
    M (SD)
    16.4±3.1
    47.8± 2.8
    7.2±3.3
    48± 7.1
  
  
    Note: M and F represent Male and Female, respectively



Table 1:  Participant Characteristics.

Clinical measures used for screening participants with autism

Social Responsiveness Scale (SRS): This is 65-item rating scale which measures the ASD symptoms in social awareness, social information processing, reciprocal social communication, social anxiety/avoidance, and stereotypic behavior [22]. These are rated on a 4-point scale ranging from “Not True” to “Almost Always True.” The SRS generates total T-score [Normal Range (= 59T), Mild to Moderate ASD Range (60T-75T), or Severe Range (= 76T) [22]] reflecting severity of social deficits in the autism spectrum.

Social Communication Questionnaire (SCQ): This is a brief instrument for screening or verification of ASD symptoms in individuals and compiled into a parent report questionnaire [23]. These questions tap qualitative impairments in reciprocal social interaction, communication, and repetitive and stereotyped patterns of behavior. The SCQ (cut-off score=15) taps the symptomatology, specifically social functioning and communication skills associated with ASD [23].

Assessment of Anxiety Level of the participants: In our present study, we wanted to understand the potential of PART system to use at least some of the physiology-related biomarkers of one’s anxiety level during interaction. In order to get an estimate of one’s anxiety level prior to taking part in our study, we administered SCAS [25]. The SCAS consists of 44 items to assess anxiety symptoms related to social phobia, separation anxiety, panic attack, obsessive compulsive disorder, generalized anxiety and physical injury fears. The SCAS forms , ranging from Never (score=0) to Always (score=3) [25] were filled by TD participants and by caregivers for participants with ASD. A T-score=60 indicates elevated anxiety [25].

Design of Virtual Reality-based social communication task: For our usability study, we designed VR-based social communication scenarios set in realistic environments built around social stories describing daily-life social contexts. Social stories are short stories that describe a particular social situation in a structured and consistent manner [26]. Studies show that social stories can effectively address social communication deficits in individuals with autism [26]. We developed 12 social stories broadly categorized into a memorable day in the life, favorite sports, film, best friend, field trip and travel with family, etc. For each social story, we developed context-relevant virtual environments using Google SketchUp. These virtual environments were then exported to our VR-based programming platform using python-based Vizard software. Then these environments were augmented with 3D characters (avatars) programmed with animation, speech, blinking, gesturing and ability to move dynamically in a VR environment to create different sets of realistic social communication scenarios. For our usability study, we designed VR-based social communication tasks, namely, Interaction Task-I (IT-I) and Interaction Task-II (IT-II) set in socially credible contexts, with IT-II following IT-I. Though real-life social situations have varying combinations of different perspectives, in our usability study, we systematically structured IT-I and IT-II to address two different aspects of social communication, hence allowing the participants to understand a social scenario from two different perspectives in isolation as a step towards social communication skill training. The IT-I allowed participants to interact and navigate through virtual environment following social etiquettes. For example, Figure 1 shows a social scenario of a restaurant being visited. First Avatar1 introduced the social scenario, e.g., restaurant and a common friend (Avatar2). Later the participant is asked to navigate through virtual environment by responding to questions asked by Avatar2 using a menu-driven interface. Based on the difficulty level the participant was exposed to different social interaction scenarios, e.g., empty restaurant, for lower difficulty level. With increased difficulty, the participant was exposed to less-crowded, moderately-crowded, and fully-occupied restaurant.

Figure 1: VR-based social communication task for IT-I.

    
    
    Figure 1:  VR-based social communication task for IT-I.

The IT-II exposed participants to subtle social communication cues, e.g., facial emotional expression, while interacting with Avatar1 Figure 2. Avatar1 narrated her experience of visiting a similar restaurant-related social environment along with context-relevant facial emotional expression. The participant was asked to watch and listen to Avatar1 and respond to questions asked by Avatar1 at the end of the narration. The difficulty level of IT-II sub-task was based on the type of questions, e.g., context-relevant, projected contingent, emotion recognition and on reporting on one’s own feeling, asked to the participants. The Table 2 shows description of the type of questions asked. These questions were designed with an aim to address some of the core social communication-related deficits of individuals with autism. Specifically, since the emotion recognition and reporting on one’s own personal feeling is often difficult communication issues for individuals with ASD, these were chosen for designing tasks of higher difficulty levels.

Figure 2: VR-based social communication task for IT-II.

    
    
    Figure 2:  VR-based social communication task for IT-II.

Table 2: Types of Questions.




  
    Type of question 
    Description 
  
  
    Context-relevant
    Simple and direct questions addressing the contexts that have been referred    by the Avatar1 during the narration
  
  
    Projected contingent
    Required one to deduce certain facts mentioned in the narration,    before responding to the question
  
  
    Emotion recognition
    Recognize the emotion exhibited by the Avatar1
  
  
    Personal feeling
    To understand whether the participant could feel the situation from    the Avatar1’s perspective.



Table 2:  Types of Questions.

The PART system offered tasks that switched from lower to higher difficulty level if the participant scored adequate performance (chosen =70% as a first approximation) and offered a task of lower difficulty for inadequate performance (=70%). We also acquired his physiological signals.

Acquisition of Participants’ Physiological Signals: While our participants interacted with PART system, their Pulseplethysmogram (PPG), Electrodermal Activity (EDA) and Skin Temperature (SKT) were acquired by using Biopac MP150 (from Biopac Systems Inc) operated in wireless mode. The real-time physiological data were validated using Acknowledge 4.3 software that comes with Biopac MP150 for standalone applications. The PPG sensor placed on the distal phalanges of middle finger of one’s non-dominant hand senses pulsating blood at finger tips. Another pair of disposable sensors placed on the distal phalanges the index and ring fingers of nondominant hand measured EDA. The SKT _sensor placed on the thumb of the non-dominant hand measured one’s body temperature. We extracted physiology-related indices, e.g., mean pulse rate (PR_MEAN) from PPG signal, tonic mean (Tonic_MEAN) from EDA signal and mean skin temperature (SKTMEAN) since, these indices or their derivatives often serve as effective indicators of one’s anxiety level [27,28].

Computation of parameters used in the data analysis

Following parameters were computed:

Normalized performance score

We calculated overall weighted, normalized performance score (PS_Norm) for IT-I (PS_{Norm_IT-I}) and IT-II (PS_{Norm_IT-II}) using equation (1).

${Normalized Performance Score (PS}_{Norm})= \frac{\sum_{i = 1}^{j} X_{D L i} * W_{D L i}}{\sum_{i = 1}^{j} X_{D L i} * W_{D L i}}, Where j=I II III, IV (1)$

Here, X_DLi represents an average performance score obtained in a task trial in level i, W_DLi weight assigned to the task in that particular difficulty level, and X_DLiis the total maximum score achievable in a task belonging to level i.

Change in physiological indices w.r.t baseline

We wanted to understand the implication of task difficulty levels (Condition 1) and avatar’s facial emotional expression (Condition 2) on one’s physiological indices namely, PR_MEAN, Tonic_MEAN and SKT_MEAN

% change in PR_MEAN w.r.t baseline

${(PR}_{MEANw .r .tBaseline(Condition1/Condition2)}) = \frac{{PR}_{MEAN(Condition1/Condition2)} - {PR}_{MEANBaseline}}{{PR}_{MEANBaseline}} X100 (2)$

Likewise was the case for % change in Tonic_MEAN w.r.t baseline (Tonic_{MEANBaseline(condition1/condition2)}) and SKT_{MEAN w.r.t baseline} (SKT _{MEANBaseline(
condition1/condition2)}).

Procedure and Experimental Setup

Figure 3 shows the experimental setup in which the participant was asked to sit in front of a task computer. A therapist (for participants with ASD) helped the experimenter in the process of familiarizing the participant to the new task environment. The experimenter gave a brief introduction of the system, described tasks by using visual schedule, followed by showing the experimental setup along with sensors to the participant. The participant was told that he was free to quit from the study at any time if he felt uncomfortable during interaction. Then the experimenter placed sensors on the participant’s finger tips and confirmed proper acquisition of physiological signals. After the participant’s yes nod, the experimenter started the execution of VR-based social communication tasks. The task began with audiovisual instruction screen to inform the participant about the task. This was followed by three minutes of baseline recording of physiological signals. Then the participant started to interact with PART system. Simultaneously, his physiological signals were acquired along with event-related markers to synchronize the physiological data acquisition with task progression. The study followed institute ethics and no compensation was provided for participation.

Figure 3: Experimental Setup.

    
    
    Figure 3:  Experimental Setup.

Results

We designed usability study using PART system in which participants with ASD and their TD counterparts interacted with VR-based tasks while their physiological signals were acquired in time-synched manner. Here we present preliminary analysis of our collected data. We were interested to

(1) See acceptability of our system by our participants.

(2) Understand the implications of varying task challenge levels with focus on IT-I and IT-II sub-tasks.

(3) Understand the implications of avatar’s facial emotional expression on participant’s performance and physiology.

Acceptability of PART system

After our participants interacted with PART system, we conducted an exit survey to understand whether our system was acceptable to our target population. In exit survey, the experimenter asked the participant

(i) Whether they were comfortable

(a) In interacting with PART system and

(b) With physiological sensors

(ii) Whether they were interested in further interaction with PART system

(iii) Whether they felt that PART can be recommended to their friends and

(iv) Asked to mention the social story that they liked the most.

All the participants informed that they were comfortable in using PART system and with physiological sensors. None of them mentioned any skin irritation due to the sensors. Also, all the participants said that they were looking forward to use PART again and would also recommend their friends to use it. Each participant mentioned that they liked different stories. For example, some found, some stories on ‘film’, while others on ‘travel with family’ as interesting, etc. All the participants completed the study without quitting. Thus PART system had the potential to be accepted by the target population.

Implication of task difficulty on participants’ performance

Our PART system offered VR-based tasks of varying difficulty levels (DLI-DLIV) depending on the questions asked by the avatar in IT-I and IT-II sub-tasks. Since the participants interacted with varying number of task trails belonging to different difficulty levels, we calculated the overall weighted normalized performance score (PS_Norm) for IT-I (PS_{Norm_IT-I}) and IT-II (PS_{Norm_IT-II}). Table 3 shows the PS_Norm for our participants with ASD and their TD counterparts. From table 3, we see that group average of PS_Norm for TD participants was higher than that of their ASD counterparts for both IT-I and IT-II. This might indicate that the tasks offered by PART system were more challenging for participants with ASD than their TD counterparts. Only few ASD participants (ASD1 and ASD2) had PS_Norm comparable to that of their TD counterparts in both IT-I and IT-II tasks. From the IDEA and SRS scores, we see that ASD1 and ASD2, respectively, were in the mild autism range. However, with a small sample size, we do not want to generalize our findings and do not intend to establish a correlation of one’s performance with autism measures. Again, we can see from Table 3 that both ASD and TD participants had higher PS_Norm in IT-I compared to IT-II tasks. Specifically, the average % decrease in PS_Norm from IT-I to IT-II for ASD was higher (26%) than that for TD (15%). This might indicate that ASD participants found IT-II tasks more challenging than IT-I tasks as compared to their TD counterparts.

Table 3: PS_Norm for IT-I and IT-II.




  
     
    PS_{Norm_IT-I}
    PS_{Norm_IT-II}
    (n₁, n₂, n₃, n₄)
  
  
    ASD1
    0.935
    0.743
    (1,1,1,3)
  
  
    ASD2
    0.877
    0.604
    (3,3,3,2)
  
  
    ASD3
    0.674
    0.531
    (3,1,0,0)
  
  
    ASD4
    0.059
    0.023
    (3,0,0,0)
  
  
    ASD5
    0.018
    0.014
    (3,0,0,0)
  
  
    ASD6
    0.050
    0.009
    (3,0,0,0)
  
  
    M (SD)
    0.435±0.440
    0.321±0.341
    -
  
  
    TD1
    0.978
    0.841
    (1,1,1,3)
  
  
    TD2
    0.944
    0.833
    (1,1,1,3)
  
  
    TD3
    0.958
    0.905
    (1,1,1,3)
  
  
    TD4
    0.938
    0.721
    (1,1,2,3)
  
  
    TD5
    0.952
    0.680
    (1,1,1,3)
  
  
    TD6
    0.935
    0.873
    (1,1,1,3)
  
  
    M (SD)
    0.951±0.016
    0.809±0.089
    -
  
  
    Note: (n₁, n₂, n₃, n₄,)    indicate the no. of trials in DLI,DLII, DLIII, DLIV respectively



Table 3:  PS_Norm for IT-I and IT-II.

In addition to PS_Norm score, the number of task trials played by participants can be another measure to understand the implication of task difficulty of PART system on participants’ performance. From Table 3 we see that only ASD1 and ASD2 were able to move to tasks of higher difficulty levels, namely, DLIII and DLIV, while all the TD participants were able to move to tasks of higher difficulty levels with a smooth transition from one level to the next.

In summary, we observed that ASD participants found VR-based social communication tasks more challenging as compared to the TD participants, as expected. Also, both ASD and TD participants found IT-II task that included subtle, but sensitive components of social communication e.g., context-relevant facial emotional expressions, more challenging than that of IT-I which included navigation-related aspects. Also, PART with varying task difficulty was capable of making individualized task presentation where few ASD participants were able to reach more difficult tasks as compared to TD participants.

Implication of avatar’s facial emotional expression on participant’s performance

Even though our system was not designed to train our participants in emotion recognition task, yet, since in IT-II task, our participants were exposed to avatars demonstrating context-relevant facial emotional expression, e.g., happy or angry, we were also interested to understand the implications of avatar’s facial emotional expression on participants’ performance. Although all the IT-II subtasks exposed our participants to avatars narrating social stories along with context-relevant facial emotional expressions, our PART system asked emotion-recognition questions to the participants for task trials belonging to DLIII and DLIV. From Table 3, we can see that only ASD1 and ASD2 and TD1-TD6 could move to tasks belonging to DLIII and DLIV. We calculated the average % performance score for all ASD and TD participants corresponding to tasks with avatars’ ‘happy’ and ‘angry’ facial emotional expressions. FromFigure 4 we see that both ASD and TD participant groups had lesser % performance score for ‘angry’ compared to that for ‘happy’ emotional expression.

Figure 4: Implications of avatars’ emotional expression on participants’ performance.

    
    
    Figure 4:  Implications of avatars’ emotional expression on participants’
performance.

In summary, exposure to avatar’s emotional expression had variations in the performance scores for both ASD and TD participants. However, as expected, we find that even in a VR-based social context, the % performance score of ASD participants were lesser, being marginally at 70%, than that for TD group.

Implication of tasks of varying difficulty levels on participants’ physiology

While our participants interacted with PART system, we acquired their physiological signals in a time-synchronized manner. Our objective was to detect physiological indices that can serve as biomarkers of one’s anxiety level, which in turn can serve as objective cues to the therapist to identify at least some of the elements of social interaction that can be anxiety-provoking for the participant in an individualized manner. Thus, we carried out further analysis to understand the effect of varying task difficulty on participants’ physiology. Figures 5 and 6 show the variations in %change in PRMEAN, TonicMEAN, and SKT_MEAN from baseline for ASD and TD participants, corresponding to IT-I and IT-II sub-tasks respectively. We can see that the average %change in PR_MEAN from baseline (henceforth PR_{MEANw.r.tBaseline}) for both IT-I and IT-II sub-tasks belonging to DLI-DLIV were positive with variations that might be indicative of increased anxiety level for participants with ASD. Also, for IT-II tasks, the trend of the variation in PR_{MEANw.r.tBaseline} with difficulty level, though similar to that during IT-I was of higher magnitude than that during IT-I. This increase in magnitude of PR_{MEANw.r.tBaseline} might indicate that IT-II task was more challenging for ASD participants than IT-I, that is also reflected from their reduced PS_Norm. For TD participants, we see a different picture for both IT-I and IT-II tasks. Variations in average %change in Tonic_MEAN from baseline (Tonic_{MEANw.r.tBaseline}) for both IT-I and IT-II tasks belonging to DLI-DLIV are positive with variations that might indicate an increased anxiety level for participants with ASD. For TD participants, we see that TonicMEANw.r.t Baseline was positive with an increasing trend from DLI-DLIV which was different from that seen from their PR_{MEANw.r.tBaseline}. Thus Tonic_{MEANw.r.tBaseline} might indicate decrease in anxiety level which is contrary to that revealed from PR_{MEANw.r.tBaseline}. In order to understand the reason behind this discrepancy, we found that one’s Tonic_MEAN can be an indicator of excitement [29], besides anxiety. The SKT_MEAN being another indicator of anxiety level was also analyzed. We can see that the average %change in SKT_MEAN from baseline (SKT_{MEANw.r.tBaseline}) for ASD group was negative for tasks of DLI for both IT-I and IT-II tasks that might indicate an increase in anxiety level. However, for tasks belonging to DLII-DLIV, for ASD participants, the SKT_MEAN during the tasks was comparable to that at baseline. For the TD participants on the other hand, the SKT_{MEANw.r.tBaseline} was positive and considerably larger in magnitude for both IT-I and IT-II tasks as compared to ASD participants which might indicate reduced anxiety level.

Figure 5: Implications of difficulty level on participants (a) PR_{MEANw.r.tBaseline} (b) Tonic_{MEANw.r.tBaseline} (c) SKT_{MEANw.r.tBaseline} for ASD.

    
    
    Figure 5:  Implications of difficulty level on participants (a) PR_{MEANw.r.tBaseline} (b) Tonic_{MEANw.r.tBaseline} (c) SKT_{MEANw.r.tBaseline} for ASD.

Figure 6: Implications of difficulty level on participants (a) PR_{MEANw.r.tBaseline} (b) Tonic_{MEANw.r.tBaseline} (c) SKT_{MEANw.r.tBaseline} for TD.

    
    
    Figure 6:  Implications of difficulty level on participants (a) PR_{MEANw.r.tBaseline} (b) Tonic_{MEANw.r.tBaseline} (c) SKT_{MEANw.r.tBaseline} for TD.

In summary, we see the potential of using physiological indices as indicators of one’s anxiety level. For participants with ASD, PR_MEAN and Tonic_MEAN increased showing elevated anxiety across difficulty levels for IT-I and IT-II tasks, with IT-II being more anxiety-provoking than IT-I. In contrast, for TD participants, PR_MEAN decreased showing reduced anxiety across difficulty levels for IT-I and IT-II tasks, with IT-II again being more anxiety-provoking than IT-I. However, for Tonic_MEAN, though we see that it has increased across difficulty levels for both IT-I and IT-II tasks (similar to ASD), yet, the Tonic_MEAN for IT-II tasks for each difficulty level were not always higher than that for the IT-I tasks belonging to similar difficulty level (which is different from that for ASD) that might indicate increase in excitement among TD group to perform better. The SKT_MEAN changed minimally for ASD and increased (larger magnitude) for TD participants indicating that they were not anxious to IT-I and IT-II tasks.

Implication of emotions on participants’ physiology

While our participants interacted with IT-II sub-tasks of PART system, they were exposed to avatars narrating social stories and exhibiting context-relevant facial emotional expression. We were interested to understand the implications of avatars’ emotional expressions on participants’ physiological indices. From Figure 7 we can see that the average %change in PR_MEAN and Tonic_MEAN from baseline was positive for both ‘happy’ and ‘angry’ emotional expressions for participants with ASD with PR_{MEANw.r.tBaseline} and Tonic_{MEANw.r.tBaseline} being greater for ‘angry’ emotion than that for ‘happy’ emotion, as can be seen from literature [30] possibly inferring that ‘angry’ emotion was more anxiety-provoking. The SKT_MEAN of participants with ASD during task execution was comparable to that during baseline. This might indicate that the emotional expressions displayed by avatars were subtle enough not to cause appreciable variations in the SKT_MEAN, or, the SKT_MEAN of the ASD group in our study were less sensitive to the avatars’ emotional expressions. For TD participants, two of the three physiological indices, namely,

Figure 7: Implications of avatars’ facial emotional expression for (a) PR_{MEANw.r.tBaseline} (b) Tonic_{MEANw.r.tBaseline} (c) SKT_{MEANw.r.tBaseline} for ASD and TD.

    
    
    Figure 7:  Implications of avatars’ facial emotional expression for (a) PR_{MEANw.r.tBaseline} (b) Tonic_{MEANw.r.tBaseline} (c) SKT_{MEANw.r.tBaseline} for ASD and TD.

PR_{MEANw.r.tBaseline} and SKT_{MEANw.r.tBaseline}, had variations that might indicate reduced anxiety for both the ‘happy’ and ‘angry’ emotions. In contrast, Tonic_{MEANw.r.tBaseline} was positive which might be due to excitement

In summary, we observe that avatars’ context-relevant facial emotional expressions were capable of causing variations in participants’ physiological indices. For participants with ASD, PR_MEAN and Tonic_MEAN increased showing elevated anxiety for both emotions, with ‘Angry’ being more anxiety-provoking than ‘Happy’. In contrast, for TD participants, PR_MEAN decreased showing reduction in anxiety for both emotions, with ‘Angry’ again being more anxietyprovoking than ‘Happy’. However, for Tonic_MEAN, though we see that it has increased for both emotions (similar to ASD), yet, Tonic_MEAN for ‘Happy’ was higher than that for ‘Angry’ (which is different from that for ASD). Thus, such a variation in Tonic_MEAN corresponding to ‘Happy’ and ‘Angry’ emotions can be attributed to increase in excitement among TD group. The SKT_MEAN changed minimally for ASD and increased for TD participants showing that they were not anxious to both the emotions.

Discussion

Given rapid progress and developments in technology, it has been argued that specific VR-based applications may be effective tools to interventionists working with children with ASD [31]. Although growing number of studies have examined applications of advanced interactive VR-based technologies to address daily-living skills [4,10], exposure to different aspects of social communication, e.g., social context, subtle social cues, social interaction, emotional expression [8], navigation in social scenario [4] etc. for adolescents with ASD, these have been addressed in an isolated manner. The presently available VR-based systems as applied to tasks involving individuals with ASD are capable of modifying tasks based only on objective performance characteristics (i.e., correct or incorrect) of responses [11,12] with no structured feedback that can serve as cues to the participants while interacting with social scenarios. Also, some of these pioneering research studies have not devised mechanisms to identify physiology-related biomarkers of one’s anxiety that might be informative to interventionist/caregiver regarding anxietyprovoking elements of social interaction for individuals with ASD in individualized manner. This might help to identify areas of social interaction where the individual would benefit from intervention/ skill training.

In our present research offered here, we have designed a usability study with VR-based performance-sensitive adaptive response technology (PART) system so as to expose our participants to different aspects of social communication in an integrated and structured manner accompanied with virtual peers who act like facilitators. Here we expose our participants to both navigation-related social norms and etiquettes and also to subtle emotional aspects of social communication. Additionally, PART system has the potential to identify physiology-related biomarkers of one’s anxiety in social contexts addressing both navigation-related and emotional aspects of social communication. Though not designed as an intervention study, but as proof-of-concept application, the preliminary results show that PART system has potential to cause variations in one’s task performance and some physiological indices that in turn can be mapped to participant’s anxiety in an individualized manner.

Limitations

However, there are some limitations of the current study that warrant consideration. When considering the applicability of this technology for practical intervention of ASD, several methodological considerations limit interpretation of our findings. The present study demonstrates the potential of using PART technology for social skill learning for individuals with ASD. However, these findings are preliminary and limited in nature. These findings are weakened by our limited sample size and corresponding issues of low power for which we have removed statistical analysis. Thus, these trends underscore the need for additional research with more complex systems and larger sample sizes. Certainly a much larger study of the current system would be needed to understand how our current findings impact areas of core deficit for individuals with ASD and how such impact can be generalized to the heterogeneous population of individuals with the disorder. There also remain many questions regarding what specific mechanism may be modified in order to optimize efficient and relevant task adjustment.

Another limitation is that this study used a menu-driven conversational module using mouse. This may be a skill relevant within a computer-generated environment and is not related to actual reallife conversation skills. However, as a first step towards developing proof-of-concept application we adhered to computer-based menudriven communication. These may not be suitable for individuals with low functioning ASD. In future, we plan to incorporate a speech recognizer module with built-in natural language understanding facility for bidirectional conversation to mitigate these problems.

Conclusion

This work demonstrates proof-of-concept of the technology and the implication of our PART system along with facilitated feedback to have implications on one’s performance and physiological indices that can be mapped to one’s anxiety level. The physiological data acquisition can not only provide information on one’s physiological profile, but can also serve as a complementary tool for the therapist. However, questions about the practicality, efficacy, and ultimate benefit of the use of this and other technological tools for demonstrating clinically significant improvements in terms of ASD impairment remain. Also, there was no evidence of the capability of this technology to realize change for participants outside of the limited environment of the experiment itself or over time. However, we hope that such a system can contribute to a better living for the individuals with ASD in the long run.

References

Download PDF

Citation: Kuriakose S, Pradeep Raj KB and Lahiri U. Design of Performance-Sensitive Adaptive Response Technology for Children with Autism: Usability Study. Austin J Autism & Relat Disabil. 2016; 2(2): 1016.

Instruction for Authors

Submit Your Article

Type of question	Description
Context-relevant	Simple and direct questions addressing the contexts that have been referred by the Avatar1 during the narration
Projected contingent	Required one to deduce certain facts mentioned in the narration, before responding to the question
Emotion recognition	Recognize the emotion exhibited by the Avatar1
Personal feeling	To understand whether the participant could feel the situation from the Avatar1’s perspective.

	PS_{Norm_IT-I}	PS_{Norm_IT-II}	(n₁, n₂, n₃, n₄)
ASD1	0.935	0.743	(1,1,1,3)
ASD2	0.877	0.604	(3,3,3,2)
ASD3	0.674	0.531	(3,1,0,0)
ASD4	0.059	0.023	(3,0,0,0)
ASD5	0.018	0.014	(3,0,0,0)
ASD6	0.050	0.009	(3,0,0,0)
M (SD)	0.435±0.440	0.321±0.341	-
TD1	0.978	0.841	(1,1,1,3)
TD2	0.944	0.833	(1,1,1,3)
TD3	0.958	0.905	(1,1,1,3)
TD4	0.938	0.721	(1,1,2,3)
TD5	0.952	0.680	(1,1,1,3)
TD6	0.935	0.873	(1,1,1,3)
M (SD)	0.951±0.016	0.809±0.089	-
Note: (n₁, n₂, n₃, n₄,) indicate the no. of trials in DLI,DLII, DLIII, DLIV respectively

ID	Age (y)	SRS(T-score)	SCQ(T-score)	SCAS(T-score)
ASD1(M)	14	58	10	61
ASD2 (F)	23	60	13	46
ASD3 (F)	19	75	18	55
ASD4 (M)	12	61	11	58
ASD5 (M)	19	65	18	57
ASD6 (F)	12	60	13	40
M (SD)	16.5±4.5	63.17±6.24	13.83±3.43	52.83± 8.08
TD1 (F)	13	49	9	42
TD2 (M)	18	45	4	50
TD3 (M)	19	44	4	43
TD4 (M)	13	48	12	41
TD5 (M)	19	51	5	54
TD6 (M)	14	50	9	58
M (SD)	16.4±3.1	47.8± 2.8	7.2±3.3	48± 7.1
Note: M and F represent Male and Female, respectively