Use of Parent - and Child Self-Report in the Assessment of Social Anxiety in Autism Spectrum Disorder

Abstract

Assessment of disorders such as social anxiety often includes input from multiple informants, with reported symptoms and their severity typically varying between sources. When assessing individuals who are on the Autism Spectrum, it may be difficult to gather self-report due to difficulties in communicating, or lack insight into their experiences, especially in regards to social anxiety. A review of the literature demonstrates that while in are typically developing and ASD populations, parents report higher levels of anxiety than their children, inter-rater discrepancies are larger in the ASD population. When such differences arise, one often must decide whose report to give more weight to, as it may mean the difference between symptoms being considered clinically relevant or not, or requiring support/treatment. While many argue that due to characteristics of ASD social anxiety symptoms reported by parents should be given more credit, it is unclear if the higher ratings provided by this group reflect a more accurate account of their children’s anxiety, or if other factors, such as misinterpretation of behaviors, are at play. As current research cannot identify whether parent or children’s self-report provide are more valid, the collection of both is considered best practice when assessing those with ASD for SAD.

Keywords: Autism spectrum disorder; Assessment; Social anxiety; Interrater agreement; Parent-report; Self-report

Abbreviations

ASD: Autism Spectrum Disorder; SAD: Social Anxiety Disorder; NICE: National Institute for Health and Care Excellence; MASC-C: Multidimensional Anxiety Scale for Children; SCARED-71: Screen for Child Anxiety-Related Emotional Disorder-71; CBT: Cognitive- Behavioral Therapy; CDI-S: Children’s Depression Inventory- Short version; RCMAS: Revised Children’s Manifest Anxiety Scale; CASS-S: Conners-Wells Adolescent Self-Report Scale-Short Edition; SLOI-CV: Short Leyton Obsessional Inventory-Child Version; ACIPL: Autism Comorbidity Interview-Present and Lifetime Version; ADHD: Attention Deficit/Hyperactivity Disorder; OCD: Obsessive Compulsive Disorder; ICC: Intraclass Correlation

Introduction

When assessing an individual for a condition, such as Social Anxiety Disorder (SAD), the assessment process often includes input from significant others, such as parents or caregivers [1]. Reliance on such sources can be to varying degrees; when the individual is young, or considered to have reduced competence the clinician will generally be more reliant on others’ reports. Characteristics of Autism Spectrum Disorder (ASD) may make it difficult to provide self-reports on experiences such as social anxiety, so when assessing for SAD in the ASD population there can be a stronger reliance on information provided by parents/caregivers.

When gathering information from multiple sources, there can be discrepancies in reported symptoms, or the severity of these. Such variances can mean the difference between qualifying for a diagnosis or not, and the resulting treatment options. It is therefore important to consider what underlies such discrepancies to help establish whether one sources’ information should be given more weight over another.

This review first will explore challenges faced when collecting information from multiple sources. The potential difficulties those with ASD face when providing self-reports of their experiences, specifically their experience of social anxiety, and studies which have used such an approach will then be discussed. Finally, parent-child agreement in reports of social anxiety in the ASD population will be reviewed.

Discrepancies in informants’ reports during the assessment process

It is generally believed that different informants can provide significant, unique information during assessment, therefore best practice involves collecting reports from multiples sources when possible [2]. For example, the National Institute for Health and Care Excellence (NICE), which provides evidence based guidelines for assessment and treatment of various conditions, state that when a child or young person is assessed for SAD he/she should be seen alone at some stage, and when possible information should also be obtained from a parent/caregiver who can provide insight into both current and past functioning [3]. When gathering information from both the child themselves and a parent, consideration must be given to discrepancies between the two parties’ reports, and the potential explanations for these. Agreement between parent and child report would suggest both parties are sound sources of information. However, if it was consistently found that both parties provided the same information/ratings, it would make collection from both sources unnecessary, and inefficient. If children and their parents are found to provide different responses however, it may be that information provided complements one another, with both sources adding to the development of an overall picture of what is happening. Alternatively, discrepancies may indicate that one source of information is less accurate than the other. When this is the case it can be difficult to determine which source is more accurate [1]. Not only can disagreement between reports lower confidence in a potential diagnosis, it can be problematic when attempting to select appropriate treatment targets [4].

A review of studies assessing concordance between parent and child reports of anxiety found that agreement between the two source was not high or consistent enough to suggest that reliance on just one of these informants is sufficient [1]; therefore to ensure best assessment of anxiety both should be taken into account with neither considered the gold standard [1,5]. Unfortunately it remains unclear as to what underlies the inconsistencies in reporting and which party should be judged to be less reliable in their reports, even when the child is considered to be typically developing. Various factors can influence the level of agreement observed, and the reason for it.

Potential problems with self-report in ASD

As previously stated, the NICE guidelines [3] recommend that when assessing young people for social anxiety self-reported information should be gathered, and in addition when possible, reports from a parent or caregiver. However, this recommendation is provided for the general population, not the ASD population; the deficits and difficulties associated with ASD may make assessments using self-report less suitable. For example, the social and communication deficits associated with ASD, such as alexithymia or lack of verbal ability mean some individuals with ASD can find it difficult to recognize the emotions and feelings they are experiencing, or to verbalize these internal states to those around them. This is especially pronounced for those individuals experiencing more significant forms of ASD who may have no accessible verbal communication, or lack the communication skills to engage in diagnostic interviews [4].

Evidence to suggest that self-reports provided by children with ASD should be used with caution is demonstrated by the finding that despite physiological responses to pleasant, neutral, and unpleasant images being similar in groups of children with and without ASD, self-reported ratings of how pleasant and interesting the pictures were differed. Ratings provided by the participants’ with ASD did not significantly differ between the two measures (pleasantness and interestingness) for any picture type, whereas the participants without ASD gave different responses on these measures for two of the three picture types. This suggests that while children with ASD may experience similar physiological reactions to those without ASD, deficits associated with ASD may affect how they perceive or report such feelings [6].

Individuals with ASD may find that providing self-report can be especially difficult when discussing social anxiety as symptoms can be considered quite abstract and therefore difficult for children with ASD to understand [7]. Even when such individuals are able to acknowledge that they are experiencing this form of anxiety, young persons with ASD may lack the understanding of the relationship between the anxiety and their impaired social skills [8].

Various studies have demonstrated that caution should be used when gathering self-reports from children with ASD. One study with a sample of individuals with ASD and a comorbid anxiety disorder found that only 23% of the individuals reported anxiety symptoms at a clinically significant level on the Multidimensional Anxiety Scale for Children (MASC-C) [9], suggesting under-reporting of symptomology. Further to this, 10% showed heightened levels of inconsistent reports, meaning their responses were deemed invalid [10]. Conversely, a separate study using the Screen for Child Anxiety- Related Emotional Disorder-71 (SCARED-71) [11] which also included individuals with ASD and a comorbid anxiety disorder found that 80% of the individuals reported anxiety symptoms at a clinically significant level [12].

In another study, various measures of anxiety were taken before and after Cognitive-Behavioral Therapy (CBT) treatment. Following treatment, reductions were seen in parent reported anxiety, and changes in diagnostic status; however, the majority of individuals’ self-reported anxiety showed no change [13]. In contrast, another study using CBT for anxiety in children with ASD showed reductions in self-reported anxiety by the participants alongside parent reported reductions [14].

Mazefsky, Kao, and Oswald [2] investigated the utility of four popular self-report measures when used with a sample of children with high functioning ASD. Measures included the Children’s Depression Inventory- Short version (CDI-S) [15], the Revised Children’s Manifest Anxiety Scale (RCMAS) [16], the Conners-Wells Adolescent Self-Report Scale-Short Edition (CASS-S) [17], and the Short Leyton Obsessional Inventory-Child Version (SLOI-CV) [18]. Self-reports on these measures were compared with parent report on the Autism Comorbidity Interview-Present and Lifetime Version (ACI-PL) [19], a semi-structured interview designed to assess for a range of comorbid psychiatric disorders in the ASD population specifically.

No substantial relationship was found between the parent’s ACIPL ratings, and children’s ratings on the four measures, although good internal consistency scores suggest that the sample had a level of understanding suitable to provide consistent answers [2]. All total self-reported ratings on the CDI-S, the RCMAS, and the CASS-S fell below the cut off scores for depression, anxiety, and Attention Deficit/Hyperactivity Disorder (ADHD) respectively, despite scores on the ACI-PL indicating presence of the associated disorders. This demonstrates poor sensitivity and specificity of these measures when used with an ASD population. The SLOI-CV produced a high false positive rate with mean self-ratings falling above the cut off score for Obsessive Compulsive Disorder (OCD) for all participants, irrespective of their OCD rating on the ACI-PL. This suggests this measure may unintentionally tap ASD symptomology, such as repetitive behaviours [2]. The RCMAS was the strongest measure assessed; difference between self-reports by those with and without anxiety neared the level of significance (though both fell below the cut off score). While significant differences were seen on the social concerns subscale, mean RCMAS scores were similar among those with, and without comorbid social anxiety. The authors also hypothesized that social concerns may be what was driving the anxiety disorders experienced in the sample, regardless of type of anxiety, or that social worries were those that the individuals were most self-aware of and able to report [2].

Children judged to have a comorbid disorder based the ACIPL provided higher ratings on the various measures than those judged to have no comorbid disorder; however this difference was not significant. The authors suggested that this might demonstrate ability of this group to provide self-report, albeit not at a level suitable for diagnostic purposes [2]. It was concluded that self-reports from those with ASD could provide useful information, but that caution should be used when interpreting these; highlighting the importance of information from other sources [2]. Van Steensel, et al. [12] found that self-report on the SCARED-71 by children with ASD and a comorbid anxiety disorder, had lower discriminate validity than did the parent report for this participant group, or self-report in the group of children with an anxiety disorder and no ASD. It was not clear, however, whether this was due to difficulty identifying and expressing anxiety symptoms or sample size (n=115) [12]. To note though, the majority (80%) of children with ASD and a comorbid anxiety disorder provided self-reports on the SCARED-71 indicative of clinical levels of anxiety. As these ratings correspond with their diagnosis of an anxiety disorder, it suggests sound self-report ability by this sample for anxiety assessment [12].

In summary, review of the research suggests that relying exclusively on self-reports provided by adolescents with ASD may lead to under identification of anxiety problems [2,10]. Because of the problems associated with children with ASD providing self-reports, past research in the area has frequently used parent report, either exclusively or in combination with individuals’ self-reports, to assess children and adolescents with ASD experiences of anxiety. Following best practise guidelines, use of self-report information should still be gathered; this may in itself be beneficial to the individuals by giving them the opportunity to identify and describe their experiences [4].

Parent-child agreement in assessment

It is generally found that parents’- and children’s self-ratings of internalized problems, such as depression or anxiety, are less strongly correlated than ratings of externalizing problems, such as aggression or alcohol/drug use [20]. However, as some symptoms of anxiety are often expressed behaviourally (e.g. school refusal), parent reported measures of anxiety are thought to be a valid measure in the typically developing population [21].

The idea that parent report is also an appropriate method for assessment of anxiety in ASD has been supported by research literature which has found good correlations between parent and self-reported anxiety [22-24]. However, it is important to note that parents of those with ASD typically report symptoms of anxiety at a higher level than their children’s self-reports [8,22,25,26]. This is not unique to the ASD population, with studies finding higher parental ratings of anxiety compared to child self-report in the typically developing population [4,27].

The unique characteristics associated with the ASD population, such as the social and communication deficits, may increase the amount of disagreement seen between the parental report and selfreport [10]. For example, in Van Steensel, et al.’s [12] study using the SCARED-71, the correlations between parent and child report were lower in the ASD group than those in the non- ASD group, but not significantly so. The trend for parent reported anxiety to be rated as more severe than children’s self-reported anxiety was evident in both groups, this difference however was more exaggerated in the ASD population.

Good inter-rater agreement on anxiety measures between parental report and self-report in the ASD population is not a consistent finding though. White, et al. [10] found that overall agreement between parent and child report on the MASC anxiety measure was poor. While significant correlations were found for the Total Physical Symptoms, Somatic/Autonomic Physical Symptoms, and Separation/Panic Total subscales, correlations on all other subscales, including the Social Anxiety subscale and the Total MASC score were not significant. Only 9 of the 39 items showed significant levels of agreement, with those assessing observable behaviours showing the strongest agreement [10].

Blakeley-Smith, et al. [4] sought to investigate the agreement on the SCARED between parent and child self-report within an ASD population. Consistent with previous studies findings, the parents reported higher anxiety ratings than the children with ASD. Looking at different types of anxiety assessed by the SCARED, the correlations between the ratings by parents and children ranged from fair to substantial; separation anxiety was the only domain with substantial agreement, with an intraclass correlation (ICC) of 71; the social anxiety domain had an ICC of 59 [4]. This suggests that different types of anxiety can produce different levels of agreement when rated by the two sources. In terms of agreement on the clinical significance of anxiety symptoms between parents and children, only separation anxiety showed significant levels of agreement. The study also looked at characteristics which influenced the level of agreement; verbal IQ was found to have a significant positive correlation with the total anxiety score on the SCARED, and two subscales (separation anxiety, and school avoidance). Meta cognitive abilities were found to significantly correlate with level of agreement on the social anxiety scale exclusively. The authors suggested this may be due to an increased awareness of their own social deficits [4].

The authors concluded that self-report may not be any more problematic with the ASD population than with the general population, with fair to strong agreement between the ratings by parents and children report shown on the SCARED. The study was unable to indicate which sources’ reports were more reliable when disagreement was shown, however the authors hypothesized that children with ASD may be influenced by the wording of the question, and by more recent and/or extreme experiences than overall experiences [4]. Overall, the findings supported the idea that both parental and child ratings should continue to be included when working with this population, as is recommended for the general population

It is unclear if the increased symptom severity reported by parents is due to inaccurate perception or exaggeration, or if parents may potentially have better insight into their child’s anxieties and the higher reporting is indicative of more accurate reports than what young persons with ASD are able to provide [8,25,26]. For example, parents may attribute certain behaviour disturbances to anxiety (such as sleeping problems or irritability), whereas young people with ASD may not have awareness of this underlying cause [28]. The idea parents may have better insight has led some researchers to argue that parent report may be a better measure to use than child/ adolescent self-reports [8,26,28]. This idea is supported by the finding that children with ASD provided lower anxiety ratings than clinician reported ratings, which were more similar to parental ratings [10].

The higher ratings of social anxiety by parents than children may however occur for various other reasons. As both ASD and social anxiety have genetic elements, it could mean that similar traits in both parent and child cause the parents’ ratings to be biased [26]. Alternatively, parents may give higher reports of their child’s anxiety due to misinterpretation of their child’s preference for individual play as a sign of withdrawal due to anxiety [25]. It has also been suggested that increased ratings of anxiety by parents may be due to increased empathy for difficulties their child with ASD faces [4]. Parents of children with or without ASD may also exaggerate their child’s symptoms for treatment-seeking, in the hopes it will lead to increased attention and support for their child [27].

While parents report higher levels of anxiety than their children, it has been suggested that such ratings may still provide an underrepresentation of the problem due to inaccurate interpretation of anxious behaviors as a result of potential differences in how anxiety is presented in this population [22].

Conclusion

There is generally some level of inconsistency in parent versus child report, regardless of the disorder the individual may have or be being assessed for, however the level of disagreement may be more prominent when working with adolescents with ASD to assess for anxiety [10].Due to the inability to determine whether parent or child reports are more accurate, it is considered best to gather information from both when possible [1,10]. While individuals with severe forms of ASD may lack the communication skills to contribute self-reported experiences, it is considered best practise to obtain self-reports during assessment when they have a milder form of ASD [4].

To reduce discrepancies in reporting due to misinterpretation, Klein [1] recommends that when assessing anxiety using both parent and child report it is important to clearly define the concepts that are being enquired about, to ensure both have a clear understanding of what is being asked.

Overall, findings from various studies demonstrate that children and adolescents with ASD provide lower ratings than their parents on self-report anxiety measures. This is consistent with the typically developing population, but to a greater degree. This suggests that such individuals may under-report anxiety symptoms, while parents/ caregivers and clinician provide higher, and potentially more accurate ratings. It is unclear what causes the lower self-ratings; it may be due to misinterpretation of their symptoms or the questions, lack of insight, embarrassment, or other reasons which we are not yet aware [10].

References

1. Klein RG. Parent-Child Agreement in Clinical Assessment of Anxiety and Other Psychopathology: A Review. J Anxiety Disord. 1991; 5: 187-198.
2. Mazefsky CA, Kao J, Oswald DP. Preliminary evidence suggesting caution in the use of psychiatric self-report measures with adolescents with highfunctioning autism spectrum disorders. Res Autism Spectr Disord. 2011; 5: 164-174.
3. National Institute for Health and Care Excellence. Social anxiety disorder: recognition, assessment and treatment. NICE Clinical Guideline. 2013; 159.
4. Blakeley-Smith A, Reaven J, Ridge K, Hepburn S. Parent-child agreement of anxiety symptoms in youth with autism spectrum disorders. Res Autism Spectr Disord. 2012; 6: 707-716.
5. Silverman WK, Ollendick TH. Evidence-based assessment of anxiety and its disorders in children and adolescents. J Clin Child Adolesc Psychol. 2005; 34: 380-411.
6. Shalom DB, Mostofsky SH, Hazlett RL, Goldberg MC, Landa RJ, Faran Y, et al. Normal physiological emotions but differences in expression of conscious feelings in children with high-functioning autism. J Autism Dev Disord. 2006; 36: 395-400.
7. Renno P, Wood JJ. Discriminant and convergent validity of the anxiety construct in children with autism spectrum disorders. J Autism Dev Disord. 2013; 43: 2135-2146.
8. Russell E, Sofronoff K. Anxiety and social worries in children with Asperger syndrome. Aust N Z J Psychiatry. 2005; 39: 633-638.
9. March JS. Multidimensional Anxiety Scale for Children manual. North Tonawanda, NY: Multi-Health Systems; 1999.
10. White SW, Schry AR, Maddox BB. Brief report: The assessment of anxiety in high-functioning adolescents with autism spectrum disorder. J Autism Dev Disord. 2012; 42:1138-1145.
11. Bodden DH, Bogels SM, Muris P. The diagnostic utility of the Screen for Child Anxiety Related Emotional Disorders-71 (SCARED-71). Behav Res Ther. 2009; 47: 418-425.
12. Van Steensel FJA, Deutschman AACG, Bögels SM. Examining the Screen for Child Anxiety-Related Emotional Disorder-71 as an assessment tool for anxiety in children with high-functioning autism spectrum disorders. Autism. 2013; 17: 681-692.
13. White SW, Ollendick T, Scahill L, Oswald D, Albano AM. Preliminary efficacy of a cognitive-behavioral treatment program for anxious youth with autism spectrum disorders. J Autism Dev Disord. 2009; 39: 1652-1662.
14. Chalfant AM, Rapee R, Carroll L. Treating anxiety disorders in children with high functioning autism spectrum disorders: a controlled trial. J Autism Dev Disord. 2007; 37: 1842-1857.
15. Kovacs M. The Children’s Depression Inventory manual. Toronto, Ontario, Canada: Multi-Health Systems, Inc. 1992.
16. Reynolds CR, Richmond BO. Revised Children’s Manifest Anxiety Scale (RCMAS): Manual: WPS, Western Psychological Services. 1985.
17. Conners CK, Wells KC, Parker JD, Sitarenios G, Diamond JM, Powell JW. A new self-report scale for assessment of adolescent psychopathology: factor structure, reliability, validity, and diagnostic sensitivity. J Abnorm Child Psychol. 1997; 25: 487-497.
18. Bamber D, Tamplin A, Park RJ, Kyte ZA, Goodyer IM. Development of a short leyton obsessional inventory for children and adolescents. J Am Acad Child Adolesc Psychiatry. 2002; 41: 1246-1252.
19. Leyfer OT, Folstein SE, Bacalman S, Davis NO, Dinh E, Morgan J, et al. Comorbid psychiatric disorders in children with autism: interview development and rates of disorders. J Autism Dev Disord. 2006; 36: 849-861.
20. Rey JM, Schrader E, Morris-Yates A. Parent-child agreement on children’s behaviours reported by the Child Behaviour Checklist (CBCL). J Adolesc. 1992; 15: 219-230.
21. Wood JJ, Gadow KD. Exploring the Nature and Function of Anxiety in Youth with Autism Spectrum Disorders. Clinical Psychology: Science and Practice. 2010; 17: 281-292.
22. White SW, Oswald D, Ollendick T, Scahill L. Anxiety in children and adolescents with autism spectrum disorders. Clin Psychol Rev. 2009; 29: 216-229.
23. Sukhodolsky DG, Scahill L, Gadow KD, Arnold LE, Aman MG, McDougle CJ, et al. Parent-rated anxiety symptoms in children with pervasive developmental disorders: frequency and association with core autism symptoms and cognitive functioning. J Abnorm Child Psychol. 2008; 36: 117-128.
24. Farrugia S, Hudson J. Anxiety in adolescents with Asperger syndrome: negative thoughts, behaviour problems, and life interference. Focus Autism Other Dev Disabl. 2006; 21: 25-35.
25. Gillott A, Furniss F, Walter A. Anxiety in high-functioning children with autism. Autism. 2001; 5: 277-286.
26. Kuusikko S, Pollock-Wurman R, Jussila K, Carter AS, Mattila ML, Ebeling H, et al. Social anxiety in high-functioning children and adolescents with Autism and Asperger syndrome. J Autism Dev Disord. 2008; 38: 1697-1709.
27. Krain AL, Kendall PC. The role of parental emotional distress in parent report of child anxiety. J Clin Child Psychol. 2000; 29: 328-335.
28. Evans DW, Canavera K, Kleinpeter FL, Maccubbin E, Taga K. The fears, phobias and anxieties of children with autism spectrum disorders and Down syndrome: comparisons with developmentally and chronologically age matched children. Child Psychiatry Hum Dev. 2005; 36: 3-26.