Special Article - Autism in Children
Austin J Autism & Relat Disabil. 2017; 3(1): 1038.
Early Screening for Autism in Child Health Care Services
Haglund N¹*, Dahlgren SO², Gustafsson P¹, Råstam M¹ and Källén K3
¹Department of Clinical Sciences Lund, Child and Adolescent Psychiatry, Lund University, Sweden
²Department of Psychology, University of Gothenburg, Sweden
³Department of Clinical Sciences Lund, Obstetrics and Gynecology, Center of Reproduction Epidemiology, University of Lund, Sweden
*Corresponding author:Nils Haglund, Department of Clinical Sciences Lund, Child and Adolescent Psychiatry, Lund University, Baravägen 1C, SE-22185 Lund, Sweden
Received: April 08, 2017; Accepted: July 06, 2017; Published: July 14, 2017
Abstract
The Observation Scale for Autism (OSA) was designed to be used as a level 1 screening instrument for Autism Spectrum Disorders (ASD) in primary health care for children. The time-efficient instrument, suitable for children under 3 years, was performed as a complementary assessment for early signs of ASD, in the existing 30- month, follow- up program in the Swedish Child Health Care (CHC) services. In total, 2571 children were screened for ASD with the OSA. A majority of the children (89%) was assessed with both OSA and the Modified Checklist for Autism in Toddlers (M-CHAT). When screened at 30 months´ with the OSA, 35 children (1.4%), reached the cut- off for suspected ASD. Information on ASD diagnoses was retrieved from the Child and Adolescent Psychiatry (CAP) clinics, and all children were followed to at least 6 years (mean 7.3). Among the screened children, 26 (1.01%) were diagnosed with ASD. In this naturalistic setting, both the OSA and the M-CHAT showed low Positive Predictive Value (PPV) as “stand-alone-instruments” (PPV=14% and 7%, respectively) in detecting ASD in children. The M-CHAT showed a higher sensitivity than the OSA (41% vs 19%), but had a higher false positive rate (93% compared to 86%). If combined (screen positive OSA and screen positive M-CHAT), the test results showed acceptable PPV but very low sensitivity. When used in the current clinical setting, none of the instruments, neither used one by one nor used in combination, showed satisfying ability to identify children at risk for ASD.
Keywords: Screening; Autism; Level 1; Child Health Care; Evaluation; Recommendation
Abbreviations
OSA: Observation Scale for Autism; ASD: Autism Spectrum Disorders; CHC: Child Health Care; M-CHAT: Modified Checklist for Autism in Toddlers; CAP: Child and Adolescent Psychiatry; PPV: Positive Predictive Value; AAP: American Academy of Pediatrics; USPSTF: U.S. Preventive Services Task Force; JA-OBS: Attention Ability –Observation –test; NH: Nils Haglund; SOD: SvenOlof Dahlgren; ADOS: Autism Diagnostic Observation Schedule; ADI: Autism Diagnostic Interview; WPPSI-R: Wechsler Preschool and Primary Scale of Intelligence-Revise; DSM-IV: Diagnostic and Statistical Manual of Mental Disorders, 4th ed; DSM 5 (5th ed); NPV: Negative Predictive Value; UK NSC: UK National Screening Committee
Introduction
Early detection of Autism Spectrum Disorders (ASD) has over the last decades become of uttermost interest for Child Health Care due to the increasing evidence that early interventions improve the social and communicative skills of children with ASD [1,2]. The impact of early intervention on alleviation of autism symptoms [3,4], stress the importance to develop an efficient autism screening system for early detection of autism. Different screening instruments, tools and questionnaires have been developed, with the Modified Checklist for Autism in Toddlers (M-CHAT) [5] as one of the most frequently used in finding children at risk for ASD in early childhood. There is and has been a discussion for and against a general screening (level 1 screening) for ASD at 18 or 24 months of age. Level 1 screening include all children regardless of suspicion of ASD, and differ from level 2 screening which include children who are already identified at increased risk due to family history or concerns from parents or clinicians. The American Academy of Pediatrics (AAP) made a statement in 2007 to support a universal screening for ASD at early age [6]. The basis for this recommendation was existing evidence of ASD symptoms at 18 months, the intensive research for developing early ASD- screening tools, and the growing number of effective early intervention programs [7]. The statement made by AAP was questioned by some scientists and practitioners [8,9], who found the evidence too weak to warrant a general ASD screening, and the interest for an implementation in pediatric practice has shown to be modest.
In 2016, the U.S. Preventive Services Task Force USPSTF [10] concluded that there was still insufficient evidence to balance the benefits and harms of screening young children for ASD in early ages as a general health care service. The USPSTF found adequate evidence that current available screening tests could detect ASD in children aged 18-30 months. However, they concluded that there was not enough evidence on benefits for a general screening for ASD when no concerns are raised from family members, other care givers or care professionals. The USPSTF concluded that evidence was lacking, of poor quality or conflicting. On the other hand, the risk of harms of screening for ASD was considered as minimal [10]. The USPSTF found no study that assessed or addressed harm of screening for ASD. Still, the review state “issues that affect families in whom children receive early false-positive screens for ASD are not described in the literature and certainly warrant further consideration”. Furthermore, the UPSTF concluded that to the best of their knowledge, there was no screening method available that could be proven cost-effective, especially in the light of the high false positive rate reported. In UK, the National Health Service made a statement against universal screening for ASD in 2011 [11].
Over the last decades, the autism spectrum was expanded to include children with milder symptoms, which has resulted in an increasing number of children diagnosed with ASD [12]. This fact challenges existing screening instruments in finding the right children for further neuropsychiatric evaluation. Different studies have reported difficulties detecting children with milder forms of ASD symptoms at younger ages, especially in under-served populations where parents’ concerns might not be raised due to cultural background and ethnicity [13,14]. Recommendation of a universal screening for neurodevelopmental disorders in primary practice, when concerns from parents or family are not raised, challenge the quality of the instrument and the overall screening procedure. A screening tool with low sensitivity has difficulties to identify children at risk for ASD whereas an instrument with low specificity will result in an unnecessary large number of children to be referred for further assessment. The quality of the screening tool can be measured by its sensitivity and specificity, which both should exceed 0.70 [15]. In the usually strained primary Health Care Services, a future universal screening of all children for ASD (level 1) must be based on a nontime consuming and easy to handle screening procedure.
Although a number of screening instruments for detection of ASD has been developed over the last decades, there are still only few scales available for assessment of symptoms in children before 3 years in a general setting. Most of the existing instruments are based on parent-reported questionnaires to be used either solely [16-21], or in combination with observation scales [22,23], or are designed to be used as level 2 instruments [24,25]. Existing instruments are often expected to be used by trained nurses, and are not designed to be used in universal, level 1- child health care clinical settings. A recent review [15] evaluating existing level 1 screening instruments for ASD found support for the use of ASD- specific screening at 18 and 24 months, but concluded that screening before 24 months was associated with higher false- positive rates than screening after 24 months. The M-CHAT is the most frequently used instrument in community settings, and has been internationally evaluated. When the screening tool is used as a stand-alone instrument, the Positive Predictive Value (PPV) for the M-CHAT has been reported to be as low as 0.06 [26]. If the M-CHAT was followed by an interview, the PPV has reported to be as high as 0.57 - 0.65 in different universal clinical settings [15]. The PPV was reported to be lower in younger children, aged 16-24 months (0.28) compared to 24-30 months old ones (0.61). A Swedish study [23] evaluated the M-CHAT in combination with the Joint Attention Ability Observation –Test (JA-OBS), followed by parent interview. Only M-CHAT test-positive children were assessed by the observation tool (JA-OBS). The study yielded high PPV (90%) for detecting ASD, but was not included in the previously quoted review. To the best of our knowledge, the sensitivity of the M-CHAT has not yet been assessed in a clinical setting.
It has been reported that existing screening tools depending on parents’ observation abilities often have unsatisfactory value in discriminating between ASD and non-ASD within the group of children showing atypical development [27]. Although some appropriate screening tools for early detection of ASD have been developed and shown to have good psychometric properties, there is still a need for brief, easy to handle assessment instruments designed for use in the primary health care system. Our research team developed the OSA (Observation Scale for Autism) [26], designed to be a time- efficient observation scale, easy to administer, and suitable for children under 3 years. The instrument was designed to perform independently of social, language, and cultural background. The mentioned requirement was especially important when the diagnostic tool should be used in the multicultural city of Malmö. The instrument was developed to be a part of the existing 30- month, follow- up program in the Swedish Child Health Care (CHC) services, offered free to all children. To the best of our knowledge, we are not aware of any true level-1 screening study in which all children were tested with an observation instrument.
The results from a pilot study evaluating the OSA [28], suggested the instrument to be able to discriminate children with ASD from children with typical development, and from children with Down syndrome. The latter group was included to evaluate the performance of the OSA in a group of children with developmental delay. According to the pilot study, using a suggested cut-off (scoring negative in 3 items or more), the OSA provides high sensitivity for ASD (92%) with low false-positive rates.
Aims
The aim of the present study was to evaluate the OSA as a level-1 instrument in the Swedish primary health care universal 30 months’ follow-up to detect signs of ASD in children. Screen positive children were supposed to be referred for diagnostic evaluation at the Child and Adolescent Psychiatry (CAP) clinic. A second aim was to compare the performance of OSA with that of the M-CHAT (which was used parallel with the OSA).
Material and Methods
Measure
The OSA was developed by the two first authors (NH, SOD). Items for the observation scale were chosen according to research in early markers for ASD [29-31]. In cooperation with the nurses at two CHC-units in Malmö, the OSA was adapted to be suitable for the standard 30- month assessment of all children in the primary Child Health Care program in Sweden. The OSA was designed to be used as a part of the universal examination, and had to be easy to administer, non-time consuming, and requiring minimal formal training for the CHC-nurses.
The OSA was earlier presented and described [26] and consists of 12 observations, focusing on the observation of behavior of the child´s ability to interact with his/her parent(s) and the CHCnurse. Observations are especially focused on reciprocal behavior in communication and social interaction and play, namely; reciprocal social interaction between caregiver and child; reciprocal eye contact between nurse and child during the assessment; reciprocal play between nurse and child; the child’s spontaneous use of two word phrases during assessment. The observers were instructed to determine whether the child behave at each observation point as expected for a child with the developmental age of 30 months. If not, the observer would mark a negative score for the item in question. When evaluating the OSA instrument [28], the research team selected the 9 most discriminative items to increase specificity (excluding from the original 12 observations; adequate movements, building blocks and two-word- sentences). The 9- item version of the OSA was used in the current study.
Participants
As a first step, the OSA was tried at two CHC-units with different populations according to social and language background in Malmö. After that, the screening instrument was offered to be used at all CHCunits in the Malmö area. Malmö is a city with a diverse population regarding cultural, social, and language background. In 2015, more than 50% of all children under the age of 16 years had a mother born outside the Nordic countries, and the proportion of immigrants is increasing [32]. Out of 29 CHC-units, 21 units chose to participate, in an up to a two-year period screening of all children at their 30-months follow-up health assessment. Different CHC-units participated in the screening program for varying long periods according to agreements made. The over-all screening period lasted during January 2011to May 2013 (29 months), but few CHC centers participated during the whole screening period. In total, 2,571 children were screened with OSA out of 6,450 children who attended the 30-month follow- up at any participating CHC (Figure 1 & Table 1). A majority of the CHCnurses participated in a one-day course on the early symptoms of ASD, and received information on how to use the OSA instrument.
With M-CHAT
Without M-CHAT
N
n
%
n
%
Total
2571
2282
89
289
11
Boys
1360
1198
88
162
12
Girls
1211
1084
89
127
11
Maternal language of tough
Swedish
1753
1629
93
124
7
Other Nordic language
84
75
89
9
11
Slavic language
155
118
76
37
24
Other European language
119
103
87
16
13
Non-European language
460
370
80
90
20
Date of birth
2008*
332
317
95
15
5
2009
1450
1315
91
135
9
2010
789
650
82
139
18
Age at test (months),
mean [SD]
30.6 [1.3]
30.6[1.2]
31.1[1.6]
Age at study closure (years),
mean [SD]
7.3[0.6]
7.3[0.6]
7.0[0.6]
M-CHAT: Modified Checklist for Autism in Toddlers; SD: Standard Deviation
* February 1 to December 31
Table 1: Demographic characteristics of the study population, by availability of M-CHAT.
Screening procedure
In Sweden, all children are offered a 30 months follow-up at their local CHC-unit. The CHC-nurse makes a general assessment, including cognitive-, motor- and speech development. For the current study, parents were offered a complementary screening for ASD symptoms using the OSA and the M-CHAT. Prior to the 30 months follow–up, the parents were sent information on the ASDscreening, were asked for a written consent to participate, and were asked to fill out the M-CHAT questionnaire before the appointment at the CHC-unit. The OSA screening was estimated to take a maximum of 5-10 minutes to perform. The M-CHAT questionnaire was available in 12 different languages and, if needed, parents were offered help at the CHC-unit to complete the questionnaire. Parents who, by any reason, decided not to participate with their child in the ASD- screening were asked to fill out a drop- out report. It has been estimated that 95-98 % of all parents take their child to the free 30- month follow-up assessment in Sweden. Participating CHC-units were offered continuous guidance from the research team during the whole screening period, and recently employed CHC-nurses received information on how to use the OSA instrument. If a child reached cut-off on either the OSA or the M-CHAT, or if parents had concerns regarding their child´s development, the CHC-nurses were instructed to arrange an appointment with the local CHC-psychologist for a second opinion before referring the child for an ASD evaluation at the CAP clinic.
The screening results (both from the OSA and the M-CHAT) were continuously collected and registered in a SPSS-database.
ASD evaluation
Children raising the suspicion of ASD were referred to a CAP clinic to be assessed by a comprehensive neuropsychiatric team, including psychiatrists and psychologists, all experts in autism. Such a neuropsychiatric evaluation comprised assessment with the Autism Diagnostic Observation Schedule (ADOS) [33] and the Autism Diagnostic Interview (ADI) [34], all according to the “gold standard” for autism diagnose. An attempt to evaluate cognitive development (often the Wechsler Preschool and Primary Scale of Intelligence-Revised (WPPSI-R) [35] or Griffiths’ Developmental Scales I and II [36]) was performed. Besides the assessments with diagnostic instrument, the children were observed in their preschool environment, for social and communicative development, by autismtrained psychologists. For children diagnosed before 2014, the criteria for ASD according to DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, 4th ed) [37] had to be fulfilled, whereas for children diagnosed after 2014, to DSM 5 [38].
Data from the CAP unit (retrieved December 14, 2016) were linked to the screening data base using each child’s unique ID-code in order to assess possible ASD diagnoses among the screened children.
Statistics
The associations between a clinical ASD diagnoses (yes vs no) and the scores of the nine individual OSA items, or three or more total scores, respectively, were assessed using Fisher Exact tests. Sensitivities of the OSA and the MCHAT, respectively, were defined as the proportion of test positive children among all children who were diagnosed with ASD at the study closure (December 14, 2016). PPV was defined as the proportion of test positive children who were diagnosed with ASD, and the false positive rate was defined as the proportion of test positive children who were not given any diagnosis of ASD at the time of the study closure.
Ethics
The study was approved by the Regional Ethics Committee in Lund (Dnr 2010/366, Dnr 2011/299). Participating in the study and screening was free for all parents and a written consent for every child must be solicited before performing the assessment.
Results
The overall study design is visualized in (Figure 1), showing the size of the eligible population, the participation rate, and the overall outcome of the OSA-screening measured as referral to CAP, and ASD diagnosis. The mean age among the participants at the study closure (when ASD information was retrieved from the CAP unit) was 7.3 years, ranging from 6.0 to 8.9 years. Out of the total study population (2571 children) screened for ASD, 106 children (4%) were referred to CAP for a further assessment due to suspected symptoms of ASD or other developmental disorders, during the period 2010- 2016. The reason for the referral could vary and was often initiated by parents themselves or by different health services. The most common referrals were made by parents and families (43%), who contacted the CAP for an ASD assessment because they were worried about developmental delay in social or communicative skills in their child. Thirty-one children (29%) were referred directly from a CHC-unit. The remaining referrals were made from social- or other health services due to suspected developmental deviations and/or concerns raised by preschool- or medical staff. In total, 26 children (1.01% of the study population) were diagnosed with ASD after a child neuropsychiatric evaluation (mean referral age 50 months, range 7-77 months). Sixty-two percent of the referrals were made from a CHC-unit. Social- or medical- services accounted for 7-11% of the referrals. No children who were diagnosed with ASD were referred directly from their parents or families.
Figure 1: Flow-chart showing the study population by screening results,
referral to Child and Adolescents Psychiatry (CAP) for ASD assessment, and
diagnosis of Autism Spectrum Disorder (ASD).
Table 1 shows the characteristics of the children who were screened with the OSA by presence or not of available M-CHAT-data. There was quite an equal distribution of boys and girls participating in the screening with a majority of children having Swedish as the native language (68%). The most common non- Nordic language was Arabic which constituted more than half of the non-European languagegroup. A majority, 89% of all children and parents, performed both the OSA and M-CHAT screening. The M-CHAT was more often available for children who had Swedish as their native language than for children speaking other languages (p<.001).
Table 2 presents the results from the OSA, including nine items and using three scores as cut-off for suspected ASD. Among the 2,571 children screened, 35 children (1.4%) reached cut-off using the OSA instrument. The items with the most frequent negative observations were; “child pointing”, “kicking ball”, “wave good-by”, “adequate response to removal of toy” and “pretend play”. All items (with exception of “name recognition”) were significantly more often scored among children who were later diagnosed with ASD than among children in the non- ASD group.
ASD-diagnosis
No ASD-diagnosis
p-value for difference*
N=26
N=2545
n
%
n
%
OSA items (n with positive scores)
1 Name recognition
0
0.0
6
0.2
.941
2 Adequate response to removal of toy
5
19.2
64
2.5
<.001
3 Interplay with parents
3
11.5
4
0.2
<.001
4 Adequate eye contact
2
7.7
5
0.2
.002
5 Following point direction
4
15.4
27
1.1
<.001
6 Pretend play
6
23.1
51
2.0
<.001
7 Child pointing
8
30.8
135
5.3
<.001
8 Kicking ball
5
19.2
81
3.2
.002
9 Waves good-bye
8
30.8
68
2.7
<.001
OSA, sum of scores
0
10
38.5
2237
87.9
<.001
1-2
11
42.3
278
10.9
<.001
3-5
2
7.7
28
1.1
.039
6-7
3
11.5
2
0.1
<.001
ASD: Autism Spectrum Disorder; OSA: Observation Scale for Autism
*p-values were obtained through Fisher-Exact tests.
Table 2: Association between ASD-diagnosis and negative OSA observations (individual OSA items, or sum of negative observations, respectively).
Table 3 shows the performance of OSA and M-CHAT, one by one and in combination, respectively, in detecting ASD. The sensitivity ranged from 0.18 (test positive M-CHAT critical scores) to 0.45 (combination of test positive OSA or M-CHAT). The false positive rates ranged between 20% (combination of test positive OSA and M-CHAT) and 93% (M-CHAT). The Negative Predictive Value (NPV) were all >99% whereas the PPV ranged from 7% (M-CHAT) to 80% (combination of test positive OSA and M-CHAT) (Table 3 around here).
ASD-diagnosis
No ASD-diagnosis
Sensitivity
False positive (1-specificity)
Positive Predictive Value
Negative Predictive Value
n
(%)
n
(%)
OSA scores
=3
5
14.3
30
85.7
0.19
0.86
0.14
<3
21
0.8
2515
99.2
0.99
M-CHAT total scores
=3
9
7.4
112
92.6
0.41
0.93
0.07
<3
13
0.6
2148
99.4
0.99
M-CHAT critical scores
=2
4
50.0
4
50.0
0.18
0.50
0.50
<2
18
0.8
2256
99.2
0.99
Combinations
OSA=3 and M-CHAT =3a
4
80.0
1
20.0
0.18
0.20
0.80
OSA<3 or M-CHAT<3b
18
0.8
2259
99.1
0.99
OSA=3 or M-CHAT =3c
10
7.9
133
92.1
0.45
0.93
0.07
OSA<3 and M-CHAT<3d
12
0.6
2127
99.4
0.99
OSA: Observation Scale for Autism; M-CHAT: Modified Checklist for Autism in Toddlers; ASD: Autism Spectrum Disorder
aCombination A: Three or more scores with OSA and three or more scores with M-CHAT
bCombination B: Less than three scores with OSA or less than three scores with M-CHAT (compliment to combination A)
cCombination C: Three or more scores with OSA or three or more scores with M-CHAT
dCombination D: Less than three scores with OSA and less than three scores with M-CHAT (compliment to combination C)
Table 3: Performance of OSA and M-CHAT in relation to ASD, regarding sensitivity, specificity, and positive and negative predictive value.
Discussion
The current study aimed to evaluate the efficiency of a new observational screening tool (the OSA) for detecting children at risk for ASD when used at the general Swedish Child Health Care program at 30 months. A secondary aim was to compare the performance of the OSA with that of another, previously validated, parent’s questionnaire for autism (the M-CHAT). Information on ASD diagnoses was retrieved from the CAP unit, independently from the screening results (a true level-1 setting). All participants had reached the age of 6 at the time of the study closure when outcome data were obtained. Thus, the estimates for sensitivity, specificity and PPV calculated in the present study are valid for children followed to the age of 6-9 years. In this naturalistic, general setting, both the OSA and the M-CHAT showed low PPV as “stand-alone-instruments” (PPV=14% and 7%, respectively) in detecting ASD in children before 6-9 years of age. The corresponding sensitivity was also low, especially for the OSA (19%).
In the light of the evidence of positive effects of early interventions programs designed for children with ASD [2], it has become of uttermost interest to develop instruments for early ASD detection to be used in clinical settings. It is an obvious challenge to find and establish a working structure for a universal screening tool with good precision which is easy to handle without being time consuming. The results of studies evaluating the performance of the M-CHAT, alone or in combination with interview, are varying [14]. Not surprisingly, studies using the M-CHAT as a stand-alone instrument [39,40] reported considerable lower PPV (6 respectively 11%) than studies using the M-CHAT in combination with a follow-up interview. The age of the child at the time of the screening procedure is another crucial factor, and could possibly account for some of the heterogeneity of the M-CHAT results reported. In 2012, the National Health Services reported that 25-30 % of the children diagnosed with ASD at 24 months lose their diagnoses by the age of four years (NHS report 2012).
A former Swedish study [23], designed as a level-1 screening, reported promising results with PPV exceeding 90% when the M-CHAT was combined with a follow up-interview. In combination with an observation tool, Joint Attention Ability –Observation –test (JA-OBS), the sensitivity was estimated to be as high as 96%). These results are in sharp contrast to those of the current study, and are also in conflict with all other published reports [15]. It shall be noted that for most children participating in the quoted study, the time interval between the screen positive test and the ASD diagnose was very short. Thus, test positive children were examined by the CAP team almost immediately after the screening, which could explain the extremely high estimates for sensitivity and specificity. It would have been interesting to review a follow-up of the screen positive as well as the screen negative children when the children participating in the former Swedish study had reached the age of at least 6 years. Another study using the M-CHAT in combination with parent interview [39] for children aged 16-30 months, showed considerably lower PPV (43%).
The performance of the OSA was evaluated in a former pilot study [28] where participants were either diagnosed with ASD, with Down’s syndrome, or were typically developing children. The study yielded promising results with a very high sensitivity (92%). Using the OSA in the primary child care program rendered considerably lower sensitivity (19%). Important for the overall ASD detection rate in any clinical setting is not only the performance of the individual diagnostic tool, but also the state of the child health care infrastructure and the effectiveness of the referral routines between the CAP unit and the CHC.
In the current study, only 10 out of the 35 children (29%) who reached the OSA-cut-off were actually referred from the CHC-unit to the CAP unit for further assessment for ASD-symptoms. Thus, 25 children (71%) were not considered at risk for ASD at the 30-month follow-up despite the fact that the OSA gave such an indication. It is not clear if the low referral rate depends on the fact that the CHCnurses did not adequately react to the OSA-results, or if the parents of the test-positive children failed to respond to the advice from the CHC-nurses. When evaluating the OSA procedure, many CHCnurses appreciated the instrument, and found it to be of crucial help when selecting children who would benefit from a referral to the CAP clinic for ASD assessment. However, several nurses experienced that some parents, especially those with little knowledge of ASD, were reluctant to accept the recommendation of further assessment, even though their child scored positive on the OSA. Similar observations have been described from other studies [15]. However, the OSA also failed regarding sensitivity, which could not be explained by the mechanisms described above. Our results revealed that only 5 out of the 26 children (19%) who met the criteria for an ASD diagnose reached the OSA cut-off. Thus, our screening instrument could not detect 21 of the children (81%) who were later diagnosed with ASD at the CAP unit. This information is not consistent with the observation that 14 out of these 21 test-negative children were later assessed and diagnosed at the CAP because of a written referral from a CHC unit. Thus, these children were observed by a CHC-nurse even though they did not reach the OSA cut-off at 30 months of age. These children were typically seen again at the CHC at the age of 50-65 months, when they demonstrated signs of suspected ASD.
Completed M-CHAT forms were available for 89% of the screened children. The performance of the M-CHAT was not better than that of the OSA assessment. The M-CHAT showed a higher sensitivity than the OSA (41% vs 19%), but had a higher false positive rate (93% compared to 86%). If the score results from the tests were combined, the combinations either yielded acceptable sensitivity and low PPV (screen positive OSA or screen positive M-CHAT), or low sensitivity with acceptable PPV (screen positive OSA and screen positive M-CHAT). Thus, when used in the current clinical setting, none of the instruments, neither used one by one nor used in combination, showed satisfying ability to identify children at risk for ASD.
Strengths and Limitations
When evaluating the results, it must be concluded that the study had some crucial limitations. Even though all participating CHCnurses were educated in how to recognize early signs of autism and how to use the OSA instrument, the study management was not in full control of how the screening procedure was arranged and performed at each CHC-unit. A more persistent surveillance and a closer cooperation between the study management and each CHC might have improved the performance of the OSA instrument. On the other hand, the current study produced a realistic estimate of the ability of the screening instrument to detect children with ASD in a truly naturalistic setting. One advantage of the present study was that because the children were followed to at least six years of age, it was possible to calculate not only the PPV, but also to produce trustworthy estimates of the sensitivity of the OSA and the M-CHAT (one by one, and in combination, respectively).
Conclusion
To the best of our knowledge, this is the first report from a true level-1 observation based screening program. The ambitions were to create a screening system independent from parental awareness regarding their children’s development. Although the evaluation of the OSA, when used in a selected group of children, showed promising potential to detect children with ASD, the performance of the test in the clinical level-1 situation showed to be less impressive. High demands are required for a screening system designed to be performed on children without any previous suspicion of any developmental abnormalities. The benefits of early ASD detection must be weighed against the risk for false positive test results causing parents unnecessary worry. Also, the time consumption of new tasks must be considered before introducing new instruments to the already strained primary health care. The results from the current study, in line with previous ones evaluating other instruments, cannot support any recommendation to use neither the OSA nor the M-CHAT as “stand-alone-instruments” in primary care.
References
- Rogers SJ, Vismara LA. Evidence-based comprehensive treatments for early autism. J Clin Child Adolesc Psychol. 2008; 37: 8-38.
- Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, et al. Randomized controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics. 2010; 125: 17-23.
- National Institute for Health and Care Excellence (NICE). Autism diagnosis in children and young people evidence update. 2013.
- Reichov B, Barton EE, Boyd BA, Hume K. Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD). Cochrane Database of Systematic Reviews. 2012; 10.
- Robins DL, Fein D, Barton ML, Green JA. The Modified Checklist for Autism in Toddlers: an initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord. 2001; 31: 131-144.
- Johnson CP, Myers SM. Identification and evaluation of children with autism spectrum disorders. Pediatrics. 2007; 120:1183-1215.
- Council on Children with Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee. Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening. Pediatrics. 2006; 118: 405-420.
- Council on Children with Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee. Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening. Pediatrics. 2006; 118: 405-420.
- Al-Qabandi M, Gorter JW, Rosenbaum P. Early autism detection: are we ready for routine screening? Pediatrics. 2011; 128.
- Campos-Outcalt D. Should all children be screened for autism spectrum disorders? No: screening is not ready for prime time. Am Fam Physician. 2011; 84: 377-378.
- Autism Spectrum Disorder in Young Children: Screening. U.S. Preventive Services Task Force. 2016.
- The UK NSC recommendation on Autism screening in children. 2012.
- Elsabbagh M, Divan G, Koh YJ, Kim YS, Kauchali S, Marcín C, et al. Global prevalence of autism and other pervasive developmental disorders. Autism Res. 2012; 5: 160-179.
- Haglund NG, Källén KB. Risk factors for autism and Asperger syndrome. Perinatal factors and migration. Autism. 2011; 15: 163-183.
- Barton ML, Dumont-Mathieu T, Fein D. Screening young children for autism spectrum disorders in primary practice. J Autism Dev Disord. 2012; 42: 1165- 1174.
- Zwaigenbaum L, Bauman ML, Fein D, Pierce K, Buie T, Davis PA, et al. Early Screening of Autism Spectrum Disorder: Recommendations for Practice and Research. Pediatrics. 2015; 136: 41-59.
- Pierce K, Carter C, Weinfeld M, Desmond J, Hazin R, Bjork R, et al. Detecting, Studying, and Treating Autism Early: The One-Year Well-Baby Check-Up Approach. J Pediatr. 2011; 153: 458-465.
- Rutter M, Bailey A, Lord C. Manual for the Social Communication Questionnaire. Los Angeles: Western Psychological Services. 2003.
- Wetherby AM, Allen L, Cleary J, Kublin K, Goldstein H. Validity and reliability of the communication and symbolic behavior scales developmental profile with very young children. J Speech Lang Hear Res. 2002; 45: 1202-1218.
- Constantino JN, Gruber CP. Social Responsiveness Scale (SRS). Los Angeles, CA: Western Psychological Services. 2005.
- Dietz C, Swinkels C, van Daalen E, van Engeland H, Buitelaar JK. Screening for autistic spectrum disorder in children aged 14-15 months. II: population screening with the early Screening of Autistic Traits Questionnaire (ESAT). Design and general findings. J Autism Dev Disord. 2006; 36: 713-722.
- Matson JL, Boisjoli JA, Hess JA, Wilkins J. Factor structure and diagnostic fidelity of the Baby and Infant Screen for Children with autism Traits-Part 1 (BISCUIT-part 1). Dev Neurorehabil. 2010; 13: 72-79.
- Gilliam JE. Gilliam Autism Rating Scale-Second Edition (GARS2). Austin, TX: Pro-Ed. 2005.
- Nygren G, Sandberg E, Gillstedt F, Ekeroth G, Arvidsson T, Gillberg C. A new screening programme for autism in a general population of Swedish toddlers. Res Dev Disabil. 2012; 33: 1200-1210.
- Stone WL, McMahon CR, Henderson LM. Use of the Screening Tool for Autism in Two-Year-Olds (STAT) for children under 24 months: an exploratory study. Autism. 2008; 12: 557-573.
- Nah YH, Young RL, Brewer N, Berlingeri G. Autism detection in early childhood (ADEC): reliability and validity data for a Level 2 screening tool for autistic disorder. Psychol Assess. 2014; 26: 215-226.
- Robins DL. Screening for autism spectrum disorders in primary care settings. Autism. 2008; 12: 537-556.
- Oosterling IJ, Swinkels SH, van der Gaag RJ, Visser JC, Dietz C, Buitelaar JK. Comparative analysis of three screening instruments for autism spectrum disorder in toddlers at high risk. J Autism Dev Disord. 2009; 39: 897-909.
- Haglund N, Dahlgren SO, Källén K, Gustafsson P, Råstam M. The Observation Scale for Autism (OSA): A New Screening Method to Detect Autism Spectrum Disorder before Age Three Years. Intellect Disabl Deign J. 2015; 3: 230-237.
- Dahlgren SO, Gillberg C. Symptoms in the first two years of life. A preliminary population study of infantile autism. Eur Arch Psychiatry Neurol Sci. 1989; 238: 169-174.
- Clifford SM, Dissanayake C. The early development of joint attention in infants with autistic disorder using home video observations and parental interview. J Autism Dev Disord. 2008; 38: 791-805.
- Zwaigenbaum L, Bryson S, Garon N. Early identification of autism spectrum disorders. Behav Brain Res. 2013; 251: 133-146.
- Statistics Sweden Statistiska Centralbyrån/ Statistics Sweden. 2016.
- Lord C, Risi S, Lambrecht L, Cook EH Jr, Leventhal BL, DiLavore PC, et al. The Autism Diagnostic Observation Schedule-Generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord. 2000; 30: 205-223.
- Lord C, Rutter M, Le Couteur A. Autism Diagnostic Interview Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994; 24: 659-685.
- Wechsler D. Wechsler Preschool and Primary Scale of Intelligence-Revised. Swedish version. 1999.
- Alin-Åkerman B, Nordberg L. Griffiths’ Developmental Scales I and II. Stockholm: Hogrefe Psykologiförlaget AB. 1980.
- Diagnostic and Statistical Manual of Mental Disorders. 4th edn. Washington (DC): American Psychiatric Association. 2000; 2.
- Diagnostic and Statistical Manual of Mental Disorders. 5th edn. Washington (DC): American Psychiatric Association. 2013.
- Pandey J, Verbalis A, Robins DL, Boorstein H, Klin AM, Babitz T, et al. Screening for autism in older and younger toddlers with the Modified Checklist for Autism in Toddlers. Autism. 2008; 12: 513-535.
- Kleinman JM, Robins DL, Ventola PE, Pandey J, Boorstein HC, Esser EL, et al. The Modified Checklist for Autism in Toddlers: a follow-up study investigating the early detection of autism spectrum disorders. J Autism Dev Disord. 2008; 38: 827-839.
Download PDF