Review Article
Austin J Psychiatry Behav Sci. 2015; 2(3): 1043.
Challenges Related to Assessing Posttraumatic Stress Symptoms in Children and Adolescents: A Generalizability Theory Study
Hukkelberg SS1,2 and Hagtvet KA²
¹Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), The Norwegian Center for Child Behavioral Development2 Oslo, Norway
²Department of Psychology, University of Oslo, Norway
*Corresponding author: Silje S Hukkelberg, Norwegian Centre for Violence and Traumatic Stress Studies and Department of Psychology, The Norwegian Center for Child Behavioral Development and University of Oslo, Postbox 7053 Majorstuen O306 Oslo, Norway
Received: September 19, 2015; Accepted: December 18, 2015; Published: December 22, 2015
Abstract
The aims of the study were: 1) To assess different sources of variation of posttraumatic stress symptoms, and 2) to estimate the generalizability of symptom scores in a multi-facet assessment situation by taking into account different sources of measurement error. The Child PTSD Symptom Scale was administered to a clinical sample of 301 children and adolescents age 10 to 18 who had experienced one or more traumatic events. The facets of observation included raters, clinics, items, and traumatic symptoms clusters. Two perspectives on the conceptualization of the PTSD symptoms were applied. Estimated G-study variance and covariance components did not support the differentiation between the three à priori defined clusters re-experiencing, avoidance, and hyper arousal. The D-study results showed acceptable generalizability and dependability coefficients.
Keywords: Posttraumatic stress symptoms; children; Generalizability theory; Multi-facet assessment; Different conceptualizations of PTSD
Introduction
The Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (American Psychiatric Association, 2000) denotes three clusters of Posttraumatic Stress Symptoms (PTSS) [1] across children and adults based on 17 symptom items: re-experiencing (e.g., through nightmares and intrusive thoughts), avoidance (e.g., situations important for the trauma), and hyper arousal (e.g., concentration problems). These categories of Posttraumatic Stress Disorder (PTSD), as well as their affiliated indicators, have gradually been selected and differentiated into the current 17 items since the diagnosis of PTSD was formally recognized in 1980. Both exploratory and confirmatory factor studies have provided support for this tripartite model in children and adolescents [2-5], although it’s superiority compared to other factor models remains an issue of debate [3,6-8].
As a consequence of the lack of consensus on how PTSS is structured in children and adolescents specifically, most measurement studies focus on the dimensionality and the stability in symptom items. However, this research does not take into account the complex measurement situation that often is associated with assessments of PTSS. A comprehensive assessment of children and adolescents with PTSS often includes collecting data from multiple sources (e.g., raters) and contexts (e.g., clinics) [9]. In order to draw conclusions about the validity of PTSS [10-12]. It is essential that the symptom items show generality across these sources and contexts. However, to our knowledge no studies have actually examined whether PTSS show generality across different contexts. The issue of generality of PTSS in children and adolescents may be especially important, since young people may be more vulnerable to contextual factors in an assessment situation compared to adults. The present study aimed to examine the generality of PTSS in a complex measurement situation. More specifically, a sample of multi-traumatized children and adolescents where assessed for PTSS, assumed to reflect the tripartite DSM-IVdefined symptom categories. The young sample was assessed in seven different clinics by two different raters. Measurement error, or error of generalization, is associated with a facet of observation which is defined as a “… a set of similar conditions of measurement” [13]. In the present study, error of measurement is associated with the facets of items, raters, and clinics, respectively. By applying Generalizability Theory (GT) it is possible to estimate the amount of variance related to each of these facets of the measurement situation which in turn will provide information for estimating generalizability of PTSS as measured in children and adolescents.
Reliability of posttraumatic stress symptoms scores
Two issues are of importance when considering the generality of a construct [14]. Most commonly, factor structure and reliability are estimated from sample data to be generalized to a population of persons. The present study deals with another type of generalization, that is, to what extent the specific score of a construct can be generalized to a universe of items beyond the items included in the specific inventory [15] that is, generalizing to a content domain of construct relevant items. Further, it may be that factors related to the assessment situation, like clinics, raters, or more idiosyncratic properties of a sample, may influence on how well PTSS scores can be generalized. This type of generalization has, to our knowledge, not yet been considered in the measurement research on PTSS.
The DSM-IV defined categorization of symptom items, as well as other constellations of items, is often found to show acceptable levels of reliability using Cronbach’s alpha [1,7,16,17]. However, is founded on classical test theory which assumes the existence of only one undifferentiated error term. When studies in fact are based on multi- faceted measurement designs, i.e., with several sources of measurement error, it is likely that Cronbach’s coefficient alpha represents an inflated estimate of generalizability because relevant sources of error variation have not been included in the estimation procedure. In light of the upcoming revision of PTSD, an investigation of the reliability of the current tripartite symptom clusters seems warranted. The present study aimed to extend knowledge on reliability of the PTSS indicators in children and adolescents by providing better arguments for the generality of the construct (Figure 1).
Figure 1: Data collection design.
The data collection design (Figure 1) is a multi-facet un balanced design. In this design two raters assessed PTSS in a different number of patients in different clinics by different numbers of items within each fixed category of symptoms. The items (i) are thus nested within (:) the three types of symptoms (f) re-experiencing, avoidance and hyper arousal, and crossed (x) with patients (p) within clinics (c) and raters (r), resulting in a (i: f) x (p: c: r) design. Figure 2 shows the eleven different sources of variation identified in this design, which in turn justifies the application of generalizability theory to address the present research questions (Figure 2).
Figure 2: Venn diagram showing the eleven different sources of variance associated with the (i:f) x (p:c:r) design.
Generalizability theory
Generalizability theory (GT) is an extension of classical test theory [13,18,19] by taking into account the multiple sources of variation that can affect item scores. Unlike classical test theory, which treats error as an undifferentiated term, GT enables to isolate different sources of measurement error that may be associated with a measurement situation. The methodology of GT is described in terms of a conceptual framework combined with statistical estimation of measurement parameters [13]. GT addresses two main successive objectives: 1) a generalizability study (G-study) where the relative impact of à priori defined facets in the universe of admissible observations is estimated. In the G-study, variance components for main and interaction effects in the universe of admissible observations are estimated. Second, based on the results from the G-study, a decision study (D-study) provides information of how a measurement design should be created to properly estimate the construct of PTSS for decision purposes. Consequently, results of a D-study in the present application will suggest the optimal combination of facet conditions (i.e., types of symptoms, items, raters, clinics) necessary to suggest a minimum level of generalizability (reliability). For generalizability studies to be conducted, the universe of generalization has to be defined. In the present design, it seems reasonable to define the facets of clinics, raters, and items as random facets since these facets could be exchanged by other conceptually equivalent instances, respectively [19]. Patients, as objects of measurement, are assumed randomly sampled to represent each clinic, respectively. On the other hand, symptom categories were defined as fixed, in accordance with the theoretically DSM-IV-defined symptom structure of PTSD [20].
Two types of generalizability coefficients can be estimated within the framework of generalizability theory. When the attention is focused on the consistency of individual differences, or comparing the relative strength of individuals’ symptoms, the relative generalizability coefficient (Eρ2), is relevant. The proper error variance related to this coefficient is the relative error variance which estimates the discrepancy between the individuals’ observed and universe (true) deviation scores. However, for other measurement purposes, the ideal score of interest is a person’s universe score independent of the universe score of other persons. Then the proper error term is the discrepancy between the persons’ observed and universe scores. This discrepancy is called the absolute error and is associated with the absolute generalizability or the dependability coefficient, In the field of measuring traumatic symptoms one may be interested in the person’s absolute score or level of traumatic experience independent of other persons’ level of trauma as well as comparing individuals’ relative levels of symptom scores. The focus may be on both comparisons to others as well as each individual’s level of performance. Thus, we believe that both relative and absolute score interpretations are of relevance in measuring PTSS.
The aims of the present study
The present study aimed to address the following questions: 1) To assess different sources of variation of posttraumatic stress symptoms, and 2) to estimate the generalizability of symptom scores in a multifacet assessment situation by taking into account different sources of measurement error.
Method
Participants and procedure
The sample of participants was part of a larger study on traumatized children and adolescents. Participants were recruited from seven different child guidance clinics in different cities in Norway. The children were referred to the clinics through standard procedures (i.e., from a general practitioner or through the child welfare system). The eligibility criteria for the study required children to be 10 to 18 year, have experienced one or more traumatic events, and speak Norwegian. Children were excluded if they suffered from severe psychosis, mental retardation, or if they presented symptoms of severe untreated conduct disorder before the traumatic experience.
The present study included 301 children and adolescents who were consecutively recruited to the study in the period April 2008 to April 2010. Age ranged from 10 to 18.
(Mean age = 14.26, SD = 2.32), and 238 (61%) were girls and 152 (39%) were boys. Most of the participants were Norwegian (n = 291, 74.6%) or had one Norwegian parent (n = 31, 7.9%). Three (0.8%) of the participants had another Scandinavian origin, whereas 36 (9.2%) were Asian, 11 (2.8%) were from Eastern Europe, 12 (3.1%) were African, 3 (0.8%) were from South or Central America, and 3 (0.8%) had other ethnicity. The number of experienced traumas ranged from 1-10 (mean = 3.0; SD = 1.67). The most frequent reported traumas were sudden death/severe illness of a close person (n = 165; 54.8%), assault/peer violence (n = 160; 53.2%), witnessed violence in the family (n = 105; 34.9%), and physical abuse by someone in the family (n = 99; 32.9%). The least reported traumas were kidnapped (n = 7; 2.3%) and war (n = 6; 2.0%).
Measures
The Child PTSD Symptom Scale (CPSS). Posttraumatic stress symptoms were assessed using the CPSS screening instrument [1]. This instrument is suitable for children and adolescents [1]. Have demonstrated satisfactory internal consistency for the three sub-scales as defined by DMS-IV. The instrument consists of two parts; one that assesses PTSS, and another part that assesses daily functioning. The present study deals with the first part only, since this part provides the most valid information on whether the child is in need of a traumaspecific intervention. The CPSS consists of 17 items that directly map onto the present DSM-described symptoms of PTSD (Criteria B, C, and D). CPSS instructions were as follows: “Circle the number that describes how often the problem has bothered you over the last 2 weeks” on a four-point scale from 0-3 (0 = Not at all; 1 = Once a week or less/once in a while; 2 = Two to 4 times a week/half the time; and 3 = Five or more times a week/almost always). Since most of the children had experienced more than one traumatic event, PTSS were assessed based on the children’s self-reported worst experience? The CPSS was administered 4 weeks or more after the trauma.
CPSS assessments were conducted at the clinic with one child at a time by either of two psychologists. Older participants responded to the CPSS items on a computer, whereas a psychologist filled out the CPSS for the younger children. All participants had the psychologist present during the session to assist if needed. One clinical psychologist (i.e., rater) was in charge of the assessment in clinic 1, 2, 3, and 4, whereas another clinical psychologist assisted children in clinic 5, 6, and 7. Both clinical psychologists were equally experienced. The CPSS instrument was translated to Norwegian and back-translated to English according to recommendations [9] and in cooperation with the developers. Reported Cronbach’s alpha for the total scale has been high also for translated versions, e.g., in Spanish [21,22] and in German [23]. The Norwegian Regional Ethical Committee approved the study.
Applied software for data analyses
Descriptive statistics were computed using SPSS for Windows, release 18.0 [24]. The G-study variance components associated with the unbalanced facet structure of the CPSS were estimated by the software urge NOVA [25]. These variance components were then inserted in the software GENOVA [26] assuming a balanced design, in order to estimate D-study statistics [25]. Estimation of variance and covariance components of the universe scores for the three categories of posttraumatic symptoms was applied by the multivariate software mGENOVA.
Results
(Table 1) Shows the estimated G-study variance components and their relative amount of variance accounted for a small percentage of the total variance (88%) was attributable to variation between raters, confirming that raters in general scored patients similarly. Further, the c: r component only explained .94% of the total variance, indicating that clinics did not diverge much in their scores within raters. As expected, substantial amount of the variance (17.13%) was accounted for by the p: c: r component. This component tells how much patients differ in terms of the general level (general component) of their PTSS scores within clinics and raters. The f and i: f component did not explain any variance at all indicating that types of symptoms (re-experiencing, avoidance and hyper arousal), as well as items within types of symptoms, were consistently assessed across patients.
Source
d.f.
VC
%
r
1
.014
.88
c:r
4
.015
.94
p:c:r
252
.273
17.13
F
2
0
0
i:f
15
0
0
rf
2
.062
3.89
ri:f
15
.554
34.76
fc:r
8
.004
.25
(i:f)(c:r)
60
.052
3.26
f (p:c:r)
504
.040
2.51
(i:f) (p:c:r)
3780
.580
36.39
Table 1: Estimated G-study variance components for PTSS.
Furthermore, the scores are also affected by different interactions between facets. The interaction component between rater and types of symptoms (rf) explained 3.89% of the total variance, indicating that the rank order of raters did not vary much from one type of symptom to another. The interaction between raters and items within symptom categories (ri: f) explained about 35% of the total variance. This component indicates relative high inconsistency of rank order of raters across items within symptom categories. The fc: r component did explain only a trivial amount of variance (25%) suggesting that the rank order of clinics was quite consistent across types of symptoms. The (i: f)(c: r) component explained 3.26% of the total variance indicating that clinics are ranked somewhat inconsistently across items within combinations of types of symptoms and rater categories. The small amount of variance accounted for by the f (p: c: r) component suggests that patients within clinics and rater categories display rather similar rank orders across symptom categories. The relative small amount of variance accounted for by the f (p:c:r)-, rf-, and fc:r components compared to the relative strong impact of the person component, p:c:r, supports the existence of a general component interpretation of the posttraumatic symptoms. As expected, the (i: f) (p: c: r) component explained the largest amount of variance (36.39%). This component is usually large, as it is confounded with both systematic and unsystematic measurement error. In sum, with the exception of the components p: c: r, ri: f and (i: f) (p: c: r), small or trivial amount of variance was accounted for by the remaining sources of variation. It should be borne in mind that the relative size of the G-study variance components are for single observations as opposed to average scores across facets which fall in the realm of D-study considerations.
As alluded to above the relative strong person component (p: c: r) compared to weak -, fc: r - and f (p: c: r) components suggests relative strong covariances/correlations among the three symptoms categories. The Univariate variance components as referred to above, do not explicitly inform about the size of the correlations between the three symptoms categories. However, the size of correlations can be estimated by using multivariate generalizability analysis. As reported in Appendix B the pattern of covariances and correlations did not provide strong support for the assumption of three differentiated symptoms categories. Rather the correlation pattern is more easily interpreted to represent a general symptom category. The estimated correlations between symptom categories were all in the interval of 85 to 90.
D-study: reliability estimates
Reports the generalizability coefficients (Eρ2), and the dependability coefficients (Φ) for different objects of measurement and numbers of items. Estimation formulas for the four scenarios combining the two objects of measurement and the two types of generalizability coefficients are reported (Table 2).
Items
5
6
7
Object of measurement: Patients within a single randomly selected clinic and rater
s2t
0.286
0.286
0.286
s2d
0.039
0.032
0.028
s2D
0.079
0.066
0.057
Er2
0.881
0.899
0.912
F
0.783
0.813
0.835
Object of measurement: Patients over groups: clinics and raters
s2t
0.3156
0.3156
0.3156
s2d
0.0791
0.0659
0.0565
s2D
0.0791
0.0659
0.0565
Er2
0.7996
0.827
0.848
F
#######
0.827
0.848
Table 2: Generalizability and dependability coefficients for different number of items and objects of measurement based on the (I:f)(p:c:r) design.
Assuming objects of measurement are patients within a single randomly selected group, the upper section of (Table 2) shows that the Eρ2- coefficient ranged from .881 to .912 when items for each symptom category increased from five to seven. The dependability coefficients displayed consistently smaller estimates than the generalizability coefficients, which is due to inclusion of more error terms associated with absolute measurement. Nevertheless, both the generalizability and dependability coefficients displayed acceptable levels indicating that CPSS scores generalize well across sets of items within fixed categories of posttraumatic stress symptoms.
Also when assuming patients over clinics and raters as objects of measurement, estimating the dependability coefficient includes more error terms than estimating the relative generalizability coefficient. Again, both coefficients reached acceptable level by varying from .799 to .848 when the number of items increased from five to seven within each symptom category. However, in this scenario both coefficients obtained identical values because the relative and absolute error variances were of equal size.
Discussion
The first aim of the present study was to assess different sources of variation related to the assessment of PTSS in a clinical sample of children and adolescents age 10 to 18. This research question is important from a measurement point of view because a) measuring PTSS may oppose basic assumptions of main stream measurement models applied to psychological constructs and b) the variance components structure can inform how well the three symptom categories of the theoretically DSM-IV-defined symptom structure are differentiated.
Commonly indicators of constructs (here: items representing traumatic events) are crossed with patients. This notion assumes that all patients are exposed to the same traumatic events. This assumption may be debatable in the present realistic sample. As noted above most children in the present sample experienced more than one traumatic event. When assessing PTSS the children were asked to have in mind their worst traumatic event. This event may not necessarily be equivalent with other children’s traumatic exposures. As with adults, symptoms deriving from children and adolescents’ traumatic experiences constitute a spectrum of traumas. Moreover, symptoms are usually related to degree of exposure [27,28] and traumatic exposure can be through direct experience, witnessing or simply hearing about an event [29]. It seems likely that different experiences, as well as different degrees of exposure, may produce different PTSS. These conceptual challenges lead to methodological challenges when measuring PTSS in the present study. It should be noted that the present analysis of traumatic symptom indicators was based on the assumption that the indicators are crossed with patients. In the present crossed design all patients have been assumingly exposed to the same set of indicators, independent of trauma experience. Prior assessments of individual differences of PTSS in children and adolescents appear to have been based on the same assumption as when estimating internal consistency reliabilities by means of a variety of instruments [30]. The assumption of crossed indicators with patients may, however, not be entirely congruent with the personal or idiosyncratic spectrum of experienced traumatic stress symptoms in different children. To accommodate such complexity of unique experiences in the present study so far, the PTSS scores were assessed by focusing on the children’s self-reported worst experience (79% of the children in this sample had experienced more than one traumatic event). This implies that the assumed “same set of indicators” may not represent equivalent indicators across the young participants. A nested design, however, may serve this purpose better. A facet is nested when two or more of the levels of the nested facet appear only at one condition of another facet [19]. Thus, the applied sets of traumatic indicators may to a large extent represent unique but conceptually equivalent indicators that all belong to the same content domain of PTSS. This re-conceptualization of the traumatic stress symptoms may represent a challenge to suggest the most appropriate model for interpreting the stress symptoms. For the purpose of assessing individual differences in posttraumatic stress symptoms it may not be feasible to capture all individual idiosyncratic experiences in a statistical model. However, this situation of uncertainty may suggest an alternative analytic approach that differs from common analytical approaches, that is, an alternative model that approximates the assumed complexity of individual differences in order to assess the generalizability of the symptom scores. To provide a comprehensive assessment of PTSS two analytic models based on different assumptions were carried out.
A characteristic pattern of variance components in the analysis based on the (i:f) (p:c:r) design reported above attracted most attention. This pattern was characterized by a relative strong p:c:r - component compared to weak rf-, fc:r and f(p:c:r) components. As noted above this pattern suggests a relative strong correlation among the three fixed symptom categories. To further interpret the correlation pattern we took advantage of the inherent relationship between the univariate mixed model and the corresponding multivariate model in the framework of generalizability theory [13]. Based on the pattern and size of the universe (true) correlations among the three fixed symptom categories, re-experiences, avoidance and hyper arousal, a clear differentiation between the three symptom categories was not supported. Rather the pattern of correlations could be interpreted as reflecting a general construct of PTSS.
The framework of generalizability theory may allow analyzing the stress symptoms being based on the alternative assumption that sets of stress symptoms are uniquely embedded within patients or in terms of generalizability theory language; symptoms are completely nested within patients. Thus the design, (i: f) (p: c: r), where items are crossed with patients (Figure 2), can be transformed into a completely nested design, that is, i: f: p: c: r where items are nested within patients (Figure 3). In generalizability terms an admissible universe of observations where indicators (items) are crossed with patients is logically congruent with an admissible universe where indictors are nested within patients. Alternatively, in the crossed design all the patients are assessed by the same set or sample of indicators, while in the nested design each participant is assessed by his/her own unique set of indicators sampled from the same universe of admissible traumatic stress symptoms. The G-study variance components for the nested design are presented. The differentiation between the three symptom categories was also addressed in the nested design. The same lack of differentiation was indicated by this analysis. A multivariate generalizability analysis based on the nested design has been described in detail. The elaborations above suggested the following conclusions for the first research aim: A characteristic pattern of variance components occurred that suggested a relative strong general component made up of the three symptom categories, re-experience, avoidance and hyper arousal. This interpretation was further supported by a multivariate generalizability analysis which explicitly estimated strong correlations between the three symptom categories. This conclusion was derived independent of the different conceptualizations of the universe of admissible observations being crossed with or nested within patients.
Figure 3: Venn diagram representing variance components for the completely nested design, i: f: p: c: r.
The second aim of the study was to examine the generalizability of the DSM-IV defined structure of PTSS taking into account the different sources of variance (facets) from the G-study. The estimation formulas for the generalizability coefficients and the corresponding results based on the completely nested design. Compared to the crossed design the results for the completely nested design displayed trivial differences with respect to estimated generalizability coefficients. However, it is important to note that quite different assumptions guided the different analytical models to assess posttraumatic stress symptoms. As expected the estimates of the r-, c: r and p: c: r – components in the nested model are the same as for the crossed model. However, both the universe (true) variance component and the error variance components in the completely nested design differed from the corresponding terms in the crossed design (I : f) (p : c : r). For further elaborations of the different nature of the universe and the error variance terms in the two models, the reader is referred.
When assuming objects of measurement to be patients within a single randomly selected group, the generalizability coefficients ranged from .881 to .912, whereas the dependability coefficients ranged from .783 to .835, when increasing the number of items from 5 to 7. This indicates that CPSS scores generalize well across items within the three categories of PTSS. When assuming patients over clinics and raters as objects of measurement, the generalizability and the dependability coefficients ranged from .799 to .848 when increasing number of items from 5 to 7 within each of the symptom categories, indicating that relative and absolute error variances are of the same size in this scenario of generalizability. Corresponding estimates were obtained from the nested design as can be seen. In conclusion, even though different error terms were taken into account, acceptable levels of generalizability over items were obtained for a) both objects of measurements being represented by a single random group or extending the objects of measurement to patients representing different clinics and raters, b), relative and absolute interpretations and c) different conceptualizations of the universe of admissible observations.
Limitations
The CPSS instrument is developed as a self-report screening instrument assessing PTSS in children and adolescents. However, in practice, many of these children and adolescents needed more or less help to score symptoms, like clarifying the content meaning of symptoms items, and thereby some involvement by a clinical psychologist (rater). A pertinent limitation of this study is that we do not know to what extent participants were actually assisted by the psychologist, and who did fill out the instrument all by themselves. However, since a clinical psychologist was present in the different clinics, all participants in this study are classified as being rated by one of the two psychologists. Secondly, since the sample included children and adolescents who had experienced multiple traumas, assessing symptoms at a fixed time frame for all persons after the traumatic incident(s) was not possible. Nevertheless, all children were assessed four or more weeks after the last traumatic incident.
Whereas many studies investigating PTSS are restricted to samples where all the children have been exposed to the same type of trauma, this study examined a realistic clinical sample of children and adolescents exposed to different types of traumas. Also, it should be noted that the present study only considered the DSM-IV-defined tripartite categories of PTSS. Research is still inconclusive as to this model’s superiority compared to other models, and some findings indicate that this particular model probably represent a suboptimal solution for children and adolescents [31,3,6-8,32].
Conclusion
The present study illuminates the complexity of assessing PTSS in children and adolescents in a realistic sample where each patient experiences multiple events of traumas. To our knowledge, this is the first study to investigate PTSS in the framework of G-theory, and illuminates its flexibility by taking into account different sources of measurement error as well as different conceptualizations of PTSS. The G-studies applied to accommodate different conceptualizations of PTSS in the present realistic sample agreed in obtaining strong correlations among the three symptom categories re-experiencing, avoidance, and hyper arousal which in turn may support a general concept of PTSS rather than three distinct symptom categories. The D-studies showed acceptable levels of dependability and generalizability coefficients when assuming objects of measurement as patients within a single randomly selected group as well as patients over groups of clinics and raters when increasing the number of items from 5 to 7. Future studies may take into account the distinction between crossed and nested designs to address different conceptualizations of posttraumatic stress symptoms in realistic samples with multiple sources of traumatic stress events.
References
- Foa EB, Johnson KM, Feeny NC, Treadwell KR. The Child PTSD Symptom Scale: A preliminary examination of its psychometric properties. Journal of Clinical Child Psychology. 2001; 30: 376-384.
- Bal A, Jensen B. Post-traumatic stress disorder symptom clusters in Turkish child and adolescent trauma survivors. European Child & Adolescent Psychiatry. 2007; 16: 449–457.
- Ford JD, Elhai JD, Ruggiero KJ, Frueh BC. Refining posttraumatic stress disorder diagnosis: Evaluation of symptom criteria with the National Survey of Adolescents. Journal of Clinical Psychiatry. 2009; 70: 748-755.
- Foy DW, Wood JL, King D, W, King LA, Resnick HS. Los Angeles Symptom Checklist: Psychometric evidence with an adolescent sample. Assessment. 1997; 4: 377-384.
- Giannopoulou I, Strouthos M, Smith P, Dikaiakou A, Galanopoulou V, Yule W. Post-traumatic stress reactions of children and adolescents exposed to the Athens 1999 earthquake. European Psychiatry. 2006; 21: 160-166.
- Hukkelberg SS, Jensen TK. The Dimensionality of Posttraumatic Stress Symptoms and Their Relationship to Depression in Children and Adolescents. Journal of Traumatic Stress. 2011; 326-333.
- Kassam-Adams N, Marsac ML, Cirilli C. Posttraumatic stress disorder symptom structure in injured children: Functional impairment and depression symptoms in a confirmatory factor analysis. Journal of the American Academy of Child & Adolescent Psychiatry. 2010; 49, 616-625.
- Saul AL, Grant KE, Carter JS. Post-traumatic reactions in adolescents: How well do the DSM–IV PTSD criteria fit the real life experience of trauma exposed youth? Journal of Abnormal Child Psychology. 2008; 36: 915–925.
- Nader K. Understanding and assessing trauma in children and adolescents: Measures, methods, and youth in context. New York: Routledge. 2008.
- Garvey AJ, Bliss RE, Hitchcock JL, Heinold JW. Predictors of smoking relapse among self-quitters: A report from the Normative Aging Study. Addictive Behaviors. 1992; 17: 367-377.
- Messick S. Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist. 1995; 50: 741-749.
- Terry DJ, O'Leary JE. The theory of planned behaviour: The effects of perceived behavioral control and self-efficacy. British Journal of Social Psychology. 1995; 34: 199-220.
- Brennan RL. mGENOVA. 2001a.
- Hagtvet KA, Man F, Sharma S. Generalizability of self-related cognitions in test anxiety. Personality and Individual Differences. 2001; 31: 1147-1171.
- Hagtvet KA. Assessment of latent constucts: A joint application of generalizability theory and covariance modeling with an emphasis on inference and structure. Scandinavian Journal of Educational Research. 1998; 44: 49 - 78.
- Garrison CZ, Bryant ES, Addy CL, Spurrier PG, Freedy JR, Kilpatrick DG. Posttraumatic stress disorder in adolescents after Hurricane Andrew. Journal of the American Academy of Child & Adolescent Psychiatry. 1995; 34: 1193-1201.
- Yule W. Post-traumatic stress disorder in children and adolescent. International Review of Psychiatry. 2001; 13: 194-200.
- Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The dependability of behavioral measurements. Theory of generalizability for scores and profiles. New York: Wiley. 1972.
- Shavelson RJ, Webb NM. Generalizability theory: A primer. Thousand Oaks, CA: Sage Publications, Inc; US. 1991.
- American Psychiatric Association. Diagnostic and Statistical Manual of mental Disorders, 4th edition, revised. Washington, DC. 2000.
- Kataoka S, Langley A, Stein B, Jaycox L, Zhang L, Sanchez N, et al. Violence exposure and PTSD: The role of English language fluency in Latino youth. Journal of Child and Family Studies. 2009; 18, 334-341.
- Kataoka S, Stein B, Jaycox L, Wong M, Escudero P, Tu W, Fink A. A school-based mental health program for traumatized Latino immigrant children. Journal of the American Academy of Child & Adolescent Psychiatry. 2003; 42, 311-318.
- Wenninger K, Ehlers A. Dysfunctional cognitions and adult psychological functioning in child sexual abuse survivors. Journal of Traumatic Stress. 1998; 11: 281-300.
- IBM SPSS Statistics. PASW Statistics for Windows version 18.0. Chicago, IL, USA: IBM Corporation. 2011.
- Brennan RL. urGENOVA. 2001c.
- Crick JE, Brennan RL. GENOVA: A general purpose analysis of variance system. 1983.
- Pynoos RS, Eth S. Witness to violence: The child interview. Annual progress in child psychiatry and child development. 1987.
- Thabet AA, Vostanis P. Post traumatic stress disorder reactions in children of war: A longitudinal study. Child Abuse & Neglect. 2000; 24: 291-298.
- Saigh PA. Verbally mediated childhood post-traumatic stress disorder. British Journal of Psychiatry. 1992; 161, 704-706.
- Ohan JL, Myers K, Collett BR. Ten-year review of rating scales. IV: Scales assessing trauma and its effects. Journal of the American Academy of Child & Adolescent Psychiatry. 2002; 41: 1401-1422.
- Anthony JL, Lonigan CJ, Hecht SA. Dimensionality of posttraumatic stress disorder symptoms in children exposed to disaster: Results from confirmatory factor analyses. Journal of Abnormal Psychology. 1999; 108: 326-336.
- Stewart AJ, Steiman M, Cauce AM, Cochran BN, Whitbeck LB, Hoyt DR. Victimization and posttraumatic stress disorder among homeless adolescents. [Journal; Peer Reviewed Journal]. Journal of the American Academy of Child & Adolescent Psychiatry. 2004; 43: 325–331.