Validity Review the articles by Fergus (2013), Kosson, et al. (2013) and Mathieu, Hare, Jones, Babiak, & Neumann (2013). Analyze the information presented in these articles on factor analysis and des
A Comparison of Three Self-Report Measures of Intolerance of Uncertainty: An Examination of Structure and Incremental Explanatory Power in a Community Sample Thomas A. FergusBaylor University Intolerance of uncertainty (IU)has been broadly defined as a dispositional fear of the unknown and appears to span across emotional disorders. Despite the fact that IU has received increased attention from clinical researchers, little systematic research has been completed to compare existing self-report measures of this construct. To help fill this gap in the extant literature, the structure and incremental explanatory power of 3 self-report measures of IU were examined in the present research using a large community sample of adults in the United States (N 624). Results from confirmatory factor analyses supported the distinctiveness of the items of the 3 measures. Nonetheless, a higher order factor accounted for the latent correlations among the 3 measures, indicating that each measure assesses the same construct. Results from structural regression models indicated that each measure of IU tended to evidence incremental explanatory power beyond one another in the concurrent prediction of variance in scores of symptom measures. These results support the notion that all 3 of the targeted measures assess IU, although each measure appears to assess a distinct aspect of this construct. Keywords:emotional disorders, intolerance of uncertainty, Intolerance of Uncertainty Index, Intolerance of Uncertainty Scale, Obsessive Beliefs Questionnaire Intolerance of uncertainty (IU) is an individual difference vari- able that has garnered increased interest from clinical researchers (Birrell, Meares, Wilkinson, & Freeston, 2011;Carleton, 2012). A seminal operational definition of IU within the clinical literature came fromFreeston, Rhéaume, Letarte, Dugas, and Ladouceur (1994), who opined that IU represented “cognitive, emotional, and behavioral reactions to uncertainty in everyday life situations” (p. 792). A number of refinements to Freeston et al.’s definition of IU have been put forth in the literature (seeBirrell et al., 2011; Carleton, 2012), withDugas and Robichaud (2007)recently de- fining IU as “a dispositional characteristic that results from a set of negative beliefs about uncertainty and its implications” (p. 24). Carleton sought to identify a common theme across extant defini- tions of IU and noted that IU fundamentally represents a disposi- tional fear of the unknown. In taxometric studies, IU has been identified as being nontaxonic (Carleton, Weeks, et al., 2012; Olatunji, Broman-Fulks, Bergman, Green, & Zlomke, 2010), in- dicating that IU is continuous and should be assessed using the full range of available scores. IU was once thought to be specific to generalized anxiety disorder (GAD), worry in particular (seeKoerner & Dugas, 2006). However, the existing literature currently supports the possibility that IU spans across emotional disorders (i.e., IU appears to be transdiagnostic;Carleton, Mulvogue, et al., 2012). To understand how IU relates to various clinical phenomena, it is of coursenecessary to have valid assessment tools. To date, the most com- monly used measure of IU has been the 27-item Intolerance of Uncertainty Scale (IUS) developed byFreeston et al. (1994). One limitation of the IUS is that a number of divergent factor structures have been found for the 27 IUS items (Berenbaum, Bredemeier, & Thompson, 2008;Buhr & Dugas, 2002;Freeston et al., 1994; Norton, 2005).Norton (2005)noted that the factor structure of the IUS items might be improved through item removal.Carleton, Norton, and Asmundson (2007)later developed a 12-item short form of the IUS, labeled the IUS–12. Carleton et al. found that scores on the IUS–12 correlated strongly with scores on the full-length IUS (r .96), and scores on both versions of the IUS evidenced statistically equivalent correlations with key criteria. Carleton et al. identified a two-factor structure for the IUS–12 items, withMcEvoy and Mahoney (2011)advocating for labeling these two factorsProspective IUandInhibitory IU. Sexton and Dugas (2009)critiquedCarleton et al.’s (2007) approach in developing the IUS–12, noting that content was not a chief consideration when selecting the short-form items and thus the IUS–12 might not capture the IU construct in a manner analogous to the full-length IUS. Addressing limitations of prior research examining the factor structure of the full-length IUS items (e.g., relatively small samples), Sexton and Dugas identified a replicable two-factor full-length IUS solution. The two factors were labeledUncertainty Has Negative Behavioral and Self- Referent ImplicationsandUncertainty Is Unfair and Spoils Every- thing. DespiteSexton and Dugas’s (2009)concern as to how the IUS–12 was developed, their full-length IUS factors strongly par- alleledCarleton et al.’s (2007)IUS–12 factors. In fact, all of the items from Carleton et al.’s Prospective IU factor loaded on Sexton and Dugas’s Uncertainty Is Unfair and Spoils Everything factor, This article was published Online First August 12, 2013. Correspondence concerning this article should be addressed to Thomas A. Fergus, Department of Psychology and Neuroscience, Baylor Univer- sity, Waco, TX 76798. E-mail:[email protected] This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Psychological Assessment© 2013 American Psychological Association 2013, Vol. 25, No. 4, 1322–13311040-3590/13/$12.00 DOI:10.1037/a0034103 1322 and all of the items from Carleton et al.’s Inhibitory IU factor loaded on Sexton and Dugas’s Uncertainty Has Negative Behav- ioral and Self-Referent Implications. Studies have been conducted to directly compareSexton and Dugas’s (2009)full-length andCarleton et al.’s (2007)short-form versions of the IUS. In one study,Khawaja and Yu (2010)found that scores on the corresponding IUS full-length and short-form scales evidenced strong correlations (rs ranging from .90 to .97), and scores on these sets of scales showed a similar pattern of correlations with theoretically relevant criteria. Despite some re- sults indicating that test scores on one of Sexton and Dugas’s full-length IUS scale evidenced more favorable discriminative validity relative to test scores on the corresponding scale of the IUS-12, Khawaja and Yu concluded that “clinicians and research- ers may choose either version without any serious limitations” (p. 105). In another study,McEvoy and Mahoney (2011)found that Carleton et al.’s IUS–12 two-factor solution, but not Sexton and Dugas’s full-length IUS two-factor solution, demonstrated an ad- equate fit to their data using confirmatory factor analyses (CFAs). In reviewing available factor analytic studies,Birrell et al. (2011) concluded that the 12 items of the IUS–12 have most consistently loaded on the same factors across studies, and Carleton et al.’s two-factor solution was supported by Birrell et al. (2011) as their proposed factor structure of the IUS items. Given its brevity and strong convergence with scores on the full-length IUS, the IUS–12 appears to be viable substitute for the full-length IUS and is the focus of the present research. Carleton (2012)stated that the “IUS–12 has been designed specifically to research the core aspects of IU across different populations and different disorders” (p. 941). At the same time, additional self-report measures that purportedly assess IU exist in the clinical literature, and interrelations among the IUS–12 and these other measures have yet to be thoroughly examined. One additional measure of IU in the clinical literature is the Obsessive Beliefs Questionnaire (OBQ), which was developed bySteketee, Frost, and the Obsessive Compulsive Cognitions Working Group (OCCWG; 2001)to assess IU and related dysfunctional beliefs. According to theOCCWG (2001), IU represents “beliefs about the necessity for being certain, that one has poor capacity to cope with unpredictable change, and that it is difficult to function adequately in ambiguous situations” (p. 1004). Within the OBQ, IU items have historically loaded on the same factor as items assessing perfectionism (Steketee, Frost, & OCCWG, 2005 [OCCWG, 2005]). This pattern of factor loadings led researchers to label this scale Perfectionism/Certainty (P/C). TheOCCWG (2005)defined P/C as reflecting “high, absolute standards of completion, rigidity, concern over mistakes, and feelings of uncertainty” (p. 1532). Moulding et al. (2011)recently developed a 20-item short form of the OBQ, labeled the OBQ–20, to provide researchers with an economical assessment of the targeted dysfunctional beliefs. Pur- suant to the present research,Moulding et al. (2011)found that the OBQ–20 –P/C items were best represented by a single factor and that scores of the OBQ–20 –P/C strongly converged with scores of a full-length version of the OBQ–P/C (rs of .95 and .96).Moulding et al. (2011)also found that scores on the OBQ–20 –P/C and full-length OBQ–P/C shared nearly identical correlations with criteria. Given its brevity and strong convergence with the full- length OBQ–P/C, the OBQ–20 –P/C was used in the present re- search. To date, researchers have found that scores on the IUS andthe OBQ–P/C moderately intercorrelate (rs of .55 and .57;Fergus & Wu, 2011). Such intercorrelations indicate that these two mea- sures are not redundant and likely assess IU in a somewhat distinct manner (Gentes & Ruscio, 2011). The Intolerance of Uncertainty Index (IUI) is a self-report measure that was recently developed byGosselin et al. (2008)to address purported limitations of the IUS. Gosselin et al. asserted that the items of the IUS tap general reactions to uncertainty rather than tolerance or acceptance of uncertainty. The IUI consists of two separate parts, but as noted by Gosselin et al., only the items of Part A of the IUI (i.e., IUI–A) assess IU in a manner consistent with how this construct has historically been defined in the liter- ature (e.g.,Dugas, Gosselin, & Ladouceur, 2001). Therefore, the IUI–A is the only part of the measure considered in this article. The content of the IUI–A items converges withGosselin et al.’s (2008)stated attempt to assess IU as a tolerance/acceptance of uncertainty (e.g.,Not knowing what will happen in advance is often unacceptable for me). Scores on the IUS and IUI–A strongly intercorrelate (r .68;Gosselin et al., 2008), although the mag- nitude of this correlation does not suggest redundancy between the two measures.Carleton, Gosselin, and Asmundson (2010)vali- dated an English version of the IUI–A, finding that the IUI–A items were best represented by a single factor.Carleton et al. (2010)called for future research to compare relations between the IUI–A and other measures of IU in an attempt to explicate the potential differential utility of these available options for assessing IU. Indeed, despite IU receiving increased attention from research- ers, little systematic research has been completed to directly com- pare existing self-report measures of IU. The lack of data speaking to interrelations among available measures of IU represents a gap in the existing literature, as it is currently unknown whether the measures all can be conceptualized as assessing the same con- struct. The different definitions used by researchers when creating the items of the IUS-12, OBQ–P/C, and IUI–A, coupled with the relatively modest interrelations among scores of these measures, suggests that it is possible that these measures assess distinct aspects of IU. The structure of the IUS–12, OBQ–P/C, and IUI–A items was examined in the present research to study this possibil- ity. The first aim of the present research was to examine whether items of these IU measures were factorially distinct. If the items of the IUS–12, OBQ–P/C, and IUI–A were factorially distinct, the next aim of the present research was to investigate whether the distinct content assessed by these items was nonetheless represen- tative of the same higher order construct (i.e., IU). Such a pattern of findings would support the notion that each of the three targeted measures assesses a distinct aspect of IU. A final aim of the present research was to further investigate the distinctiveness of the IUS–12, OBQ–P/C, and IUI–A by examin- ing unique relations between the three targeted measures and symptom measures of emotional disorders. Although typically used to support the validity of test scores, the concept of incre- mental validity can also have important conceptual implications (Hunsley, 2003). In the context of the present research, finding a unique relation between each IU measure and symptom measure after accounting for variance shared across IU measures would suggest thatCarleton’s (2012)notion that the IUS–12 assessthe core aspects of IU across different disorders might be broadened to consider the possibility that the IUS–12, OBQ–P/C, and IUI–A This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1323 INTOLERANCE OF UNCERTAINTY each assess core aspects of IU that span across different disorders. Finding relatively robust relations between scores on certain IU measures and symptom measures might also shed light onto op- erational definitions of IU that are particularly relevant to symp- toms of specific emotional disorders. Depression, generalized anx- iety, and obsessive– compulsive symptoms were targeted in the present research, as these three symptom types have garnered particular interest from IU researchers (Gentes & Ruscio, 2011). A higher order CFA approach, followingBrown (2006), was used in the present research to examine the structure of the items of the IUS–12, OBQ–P/C, and IUI–A. This data analytic approach allowed for an examination of the initial two study aims outlined earlier. As recommended byBrown (2006), an adequate first-order model was initially fit to the data before examination for the presence of a higher order factor was performed. When testing the adequacy of first-order models, it was predicted that a four-factor model—in which the items of IUS–12–Prospective, IUS–12– Inhibitory, OBQ–P/C, and IUI–A loaded on separate, yet intercor- related, factors—would provide an adequate fit to the data and a better fit to the data relative to alternative first-order factor struc- tures. This prediction was based on the reviewed factor analytic studies of the targeted measures, the differences in operational definitions used by researchers when creating the items of each measure, and the relatively modest magnitude of interrelations among the scores of the measures. It was next predicted that a higher order factor would account for the intercorrelations among the four first-order factors. Despite the expectation that items of the IUS–12–Prospective, IUS–12–Inhibitory, OBQ–P/C, and IUI–A would be distinct, the presence of a higher order factor would support the notion that these measures are all representative of the same overarching construct (i.e., IU). A higher order factor was predicted to account for the latent factor correlations, as each of the targeted measures was developed to assess IU. Structural equation modeling (SEM) was then used to examine the incremental explanatory power of each measure in the concur- rent prediction of depression, generalized anxiety, and obsessive– compulsive symptoms. Using SEM provides advantages compared with using scaled scores, including the ability to account for scale unreliability (Brown, 2006). Because of such advantages, using SEM in the present research was expected to provide clearer estimates of interrelations among the IU measures and symptom types than what would be expected using scaled scores. These tests of incremental explanatory power were considered exploratory. Method Participants The sample consisted of 624 adults recruited through the Inter- net. The mean age of the sample was 33.1 years (SD 11.4; range from 18 to 71). Respondents primarily self-identified as female (56.1%), having received at least a 2-year college degree (55.9%), working at least part-time (68.4%), and being currently unmarried (66.9%). In terms of racial/ethnic identification, 79.2% self- identified as White, 6.6% self-identified as African American, 6.4% self-identified as Asian, 3.7% self-identified as Hispanic, 3.5% self-identified as bi- or multiracial, and 0.6% self-identified as “other” race/ethnicity. Measures Intolerance of Uncertainty Scale–12-item version (IUS–12; Carleton et al., 2007).As introduced, the IUS–12 is a 12-item short form of the full-length 27-item IUS (Freeston et al., 1994; English translation:Buhr & Dugas, 2002). The IUS–12 consists of seven items that assess Prospective IU (e.g.,Unforeseen events upset me greatly) and five items that assess Inhibitory IU (e.g., When I am uncertain, I can’t function very well). The IUS–12 items are rated on a 5-point scale (ranging from 1 to 5). In previous research, the test scores on the Prospective IU and Inhibitory IU scales demonstrated adequate internal consistency (Cronbach’s s of .85 and .88;Carleton et al., 2007;McEvoy & Mahoney, 2011). Further,Khawaja and Yu (2010)found that test scores on the IUS–12 evidenced satisfactory test–retest reliability (2-weekr .77). Test scores on the IUS–12 scales (total:M 35.22,SD 8.83; Prospective IU:M 22.41,SD 5.06; Inhibitory IU: M 12.81,SD 4.56) demonstrated good internal consistency in the present research (Cronbach’s s ranging from .86 to .91). The average interitemrfor test scores on the Prospective IU scale was .46 (ranged from .35 to .59), and the average interitemrfor test scores on the Inhibitory IU scale was .62 (ranged from .54 to .75) in the present research. Obsessive Beliefs Questionnaire–20-item version (OBQ–20; Moulding et al., 2011).As introduced, the OBQ-20 is a 20-item short form of the 44-item version of the OBQ (Steketee, Frost, & OCCWG, 2005). The OBQ–20 –PC—the OBQ–20 scale of inter- est in the present research— consists of five items that assess for perfectionism and certainty (e.g.,For me, things are not right if they are not perfect). The OBQ–20 items are rated on a scale ranging from 1 to 7. In previous research, test scores on the OBQ–20 –P/C have demonstrated adequate internal consistency given the brevity of the scale (Cronbach’s s of .78 and .81; Moulding et al., 2011). Test scores on full-length versions of this scale also have evidenced satisfactory test–retest reliability (2- month to 3-monthrs ranging from .66 to .77;Steketee, Frost, & OCCWG, 2003). Test scores on the OBQ–20 –P/C (M 18.50, SD 7.16) evidenced adequate internal consistency in the present research ( .83). The average interitemrfor test scores on the OBQ–20 –P/C was .49 (ranged from .35 to .68) in the present research. Intolerance of Uncertainty Index–Part A (IUI–A;Gosselin et al., 2008; English translation:Carleton et al, 2010).As introduced, the IUI–A is a 15-item measure that assesses difficul- ties tolerating or accepting uncertainty (e.g.,I have difficulty tolerating life’s uncertainties). IUI–A items are rated using a scale ranging from 1 to 5. Previous research has shown that test scores on the IUI–A demonstrated good internal consistency (Cronbach’s s of .94 and .96;Carleton et al., 2010;Gosselin et al., 2008), and Gosselin et al. (2008) found that test scores on the IUI–A evi- denced satisfactory test–retest reliability (5-weekr .76). Test scores on the IUI–A (M 40.99,SD 14.10) evidenced good internal consistency in the present research ( .96). The average interitemrfor test scores on the IUI–A was .63 (ranged from .40 to .82) in the present research. Depression, Anxiety, and Stress Scales–21-item version (DASS–21;Lovibond & Lovibond, 1995).The DASS–21 is a 21-item short-form version of the original 42-item version of the DASS (Lovibond & Lovibond, 1995) that assesses depression, This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1324 FERGUS anxiety, and stress symptoms the respondent has experienced over the previous week. The Depression scale of the DASS-21—the DASS-21 scale of interest in the present research— consists of seven items (e.g.,I couldn’t seem to experience any positive feeling at all) that are rated using a scale ranging from 0 to 3. Antony, Bieling, Cox, Enns, and Swinson (1998)found that the depression items of the DASS–21 significantly loaded on a single factor that wasdistinct from the anxiety and stress factors of the measure. Antony et al. (1998) further found that test scores on the Depres- sion scale of the DASS–21 demonstrated good internal consistency ( .94) and correlated strongly with scores on other symptom measures of depression (r .79). Prior research has shown that test scores on the full-length Depression scale evidenced satisfac- tory test–retest reliability (2-weekr .71;Brown, Chorpita, Korotitsch, & Barlow, 1997). Test scores on the Depression scale of the DASS–21 (M 5.93,SD 5.59) evidenced good internal consistency in the present research ( .93). The average interi- temrfor test scores on the Depression scale of the DASS–21 was .64 (ranged from .51 to .74) in the present research. Generalized Anxiety Disorder–7 (GAD–7;Spitzer, Kroenke, Williams, & Lowe, 2006).The GAD–7 is seven-item measure that assesses generalized anxiety symptoms the respondent has experienced over the previous 2 weeks (e.g.,Worrying too much about different things). Spitzer et al. (2006) found that the GAD–7 items all significantly loaded on a single factor and that test scores on the GAD–7 evidenced good internal consistency ( .92) and satisfactory test–retest reliability (1-weekr .83). Scores on the GAD–7 correlate strongly (r .64) with scores on another mea- sure of generalized anxiety symptoms (Kertz, Bigda-Peyton, & Bjorgvinsson, in press). Test scores on the GAD–7 (M 6.94, SD 5.46) evidenced good internal consistency in the present research ( .91). The average interitemrfor test scores on the GAD–7 was .59 (ranged from .45 to .84) in the present research. Dimensional Obsessive–Compulsive Scale (DOCS;Abra- mowitz et al., 2010).The DOCS is a 20-item measure that assesses the severity of obsessive– compulsive symptoms using a scale ranging from 0 to 4. The four symptom dimensions assessed by the DOCS are contamination, responsibility for harm, unac- ceptable thoughts, and symmetry. Each DOCS scale assesses for the time spent, avoidance, distress, interference, and attempts of control surrounding the respective symptom dimension. Abramowitz et al. (2010)found that a four-factor solution best represented the items of the DOCS and that test scores on the DOCS scales evidenced good internal consistency ( s ranging from .83 to .96) and satisfactory test–retest reliability (5-weekrs ranging from .55 to .66).Abramowitz et al. (2010)also found that scores on the DOCS scales correlated moderately to strongly (rs ranging from .39 to .88) with scores of other symptom measures that assess corresponding symptom dimensions. Test scores on each DOCS scale (Contamination:M 2.79,SD 3.34; Respon- sibility:M 3.40,SD 3.89; Unacceptable Thoughts:M 3.51, SD 4.11; Symmetry:M 3.14,SD 3.95) demonstrated good internal consistency in the present research ( s ranging from .86 to .93). The average interitemrfor test scores on each of the DOCS scales in the present research was as follows: Contamination .57 (ranged from .54 to .61), Responsibility .69 (ranged from .63 to .77), Unacceptable Thoughts .68 (ranged from .60 to .75), and Symmetry .73 (ranged from .68 to .76). Procedure Participants were recruited using Amazon’s Mechanical Turk (MTurk), an Internet-based platform that allows respondents to complete jobs (e.g., survey completion) for monetary compensa- tion. Respondents completing surveys through MTurk have been found to produce high-quality data (Buhrmester, Kwang, & Gos- ling, 2011). The use of an Internet sample was supported by the equivalence of IU measures across paper-and-pencil versus Inter- net administration (Coles, Cook, & Blake, 2007). The present research was approved by the local institutional review board. Recruitment was limited to MTurk workers over 18 years old and located in the United States. Participants were required to provide electronic consent, and there was no penalty for withdrawing from the study. Upon completion of the study, participants were de- briefed and paid in full. Compensation was $1, an amount consis- tent with the compensation given to MTurk workers completing prior studies of similar length (Buhrmester et al., 2011). Results Structure As described earlier, a higher order CFA approach was used to examine the structure among the items of the IUS–12, OBQ–P/C, and the IUI–A. 1Tests for multivariate skewness and kurtosis were significant for some of the item scores of these measures, suggest- ing the presence of multivariate nonnormality. Multivariate non- normality can negatively impact results obtained when maximum likelihood (ML) estimation is used. Robust ML estimation (Satorra & Bentler, 1994) was therefore used for all reported analyses, as this estimation procedure provides parameter estimates with stan- dard errors that are robust to nonnormality (Brown, 2006). All models were tested by inputting covariance and asymptotic cova- riance matrices into LISREL Version 8.80 software (Jöreskog & Sörbom, 2007). Four commonly recommended (Brown, 2006;Hu & Bentler, 1999;Kline, 2011) goodness-of-fit statistics were used to evaluate all tested models. These fit statistics were the compar- ative fit index (CFI), nonnormed fit index (NNFI), root-mean- square error of approximation (RMSEA), and standard root-mean- square residual (SRMR).Hu and Bentler’s (1999)guidelines were used to evaluate fit: CFI and NNFI should be close to .95, RMSEA should be close to .06, and SRMR should be close to .08. Further, the upper limit of the 90% RMSEA confidence interval (CI) should not exceed .10 (Kline, 2011). Nested models were com- 1An exploratory factor analysis (EFA) yielded an identical pattern of results as the results obtained when using the CFA approach described in this article. Specifically, a four-factor solution was obtained when using an EFA, which consisted of principal axis factoring and oblique (oblimin) rotation. A parallel analysis for both mean and 95th percentile eigenvalues (usingO’Connor’s, 2000, syntax) indicated the appropriateness of the four-factor solution. The first five eigenvalues were 15.91, 1.82, 1.68, 1.44, and 0.96. Each item loaded saliently (i.e., factor loading .40) on the expected factor (i.e., IUS–12-Prospective, IUS–12–Inhibitory, OBQ–P/C, or IUI–A), and there were no salient cross-loadings within the four-factor EFA solution. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1325 INTOLERANCE OF UNCERTAINTY pared using the Satorra–Bentler scaled difference chi-square test (i.e., SDCS test; followingBryant & Satorra, 2012). 2 As recommended byBrown (2006), the adequacy of a first- order measurement model was initially examined. The metric of latent factors was set by fixing one of the unstandardized item factor loadings to 1.0 (Brown, 2006). Fit statistics for all of the tested measurement models are presented inTable 1. A correlated four-factor model was tested first. This model consisted of the items of the IUS–12–Prospective, IUS–12–Inhibitory, OBQ–P/C, and IUI–A loading on one of the four latent factors, with no secondary loadings. Based on the specified guidelines, the corre- lated four-factor model provided an adequate fit to the data. Despite the adequacy of the correlated four-factor first-order model, alternative first-order models were considered. For exam- ple, it is possible that the IUS–12 items are not best represented by distinct factors in the context of the items of the OBQ–P/C and IUI–A. To examine this possibility, a correlated three-factor model was tested next. This model was identical to the correlated four- factor model, except all of the IUS–12 items were collapsed onto a single latent factor. With the exception of the RMSEA, this three-factor model provided an adequate fit to the data. However, the fit statistics were more favorable for the four-factor model. Further, the SDCS test indicated that the three-factor model pro- vided a significant decrement in model fit relative to the four- factor model: D2(3) 329.69,p .01. For a more parsimonious model, a one-factor first-order model was tested next. Within this model, the items of the IUS–12, OBQ–P/C, and IUI–A were all collapsed onto a single latent construct. With the exception of the RMSEA and 90% RMSEA CI, the one-factor model provided an adequate fit to the data. However, all of the fit statistics were more favorable for the correlated four-factor model. Further, the SDCS test indicated that the one-factor model provided a significantly poorer fit to the data than did the four-factor model: D2(6) 668.66,p .01. Despite the decrement in model fit surrounding the one factor, the pattern of factor loadings on this factor was used to guide the consider- ation of any remaining tenable first-order models. The completely standardized factor loadings from the one-factor model are pre- sented inTable 2. Generally speaking, the IUS–12 items and the IUI–A items loaded strongly on the factor, whereas the OBQ–P/C items evidenced more modest factor loadings. Therefore, it is possible that a correlated two-factor model, in which the IUS–12 items and the IUI–A items load on one factor and the OBQ–P/C items load a second factor, might provide an adequate fit to the data. The adequacy of this model was tested next. With the exception of the RMSEA and 90% RMSEA CI, the correlated two-factor model provided an adequate fit to the data. However, the fit statistics were all more favorable for the correlated four- factor model. Further, the SDCS test indicated that the two-factor model provided a significantly poorer fit to the data than did the four-factor model: D2(5) 571.83,p .01. Based on the specified guidelines for model comparisons, the correlated four-factor model provided the best model fit to these data. All factor loadings within the four-factor model were significant (p .01), and the completely standardized loadings are presented inTable 2. All of the correlations among the latent factors were significant (p .01) and are presented inTable 3. Given the superiority of the four-factor first-order model, a second-order CFA model was tested next. FollowingBrown (2006), this second-order model removed thefirst-order latent factor intercorrelations and added direct effects from a higher order factor to each of the first-order factors (IUS– Prospective, IUS–Inhibitory, OBQ–P/C, and IUI–A). The items of each of these scales were retained as indicators of each first-order factor in the second-order model. Evidence of alackof significant decrement in model fit between the first-order model and second- order model is indicative of the presence of a higher order factor (Brown, 2006). As presented inTable 1, the second-order model provided an adequate fit to the data. All of the fit statistics met or exceeded the specified guidelines and were comparable to the corre- lated four-factor first-order model. Further, the SDCS test indicated that the second-order model did not provide a significant decrement in model fit relative to the four-factor first-order model: D2(2) 0.56, ns. 3The second-order factor loadings of all of the first-order factors were significant (p .01), and the completely standardized loadings on the second-order factor were as follows: IUS–12–Prospective .90; IUS–12–Inhibitory .89; OBQ–P/C .68; and IUI–A .92. Incremental Explanatory Power As described previously, unique relations between each IU measure and the symptom measures were examined using SEM. More specifically, three latent structural regression models were fit to the data. The three models only differed based on the endoge- nous construct in each model. Each model consisted of four exogenous constructs being allowed to predict one endogenous construct. The exogenous constructs were allowed to co-vary in each model. The exogenous constructs were each of the IU mea- sures, and the endogenous construct was one of the symptom measures. The indicators for the exogenous constructs were the items of the respective IU measure. The indicators for the depres- sion and generalized anxiety constructs were the items of the respective symptom measures, whereas the indicators for the obsessive– compulsive construct were the separate four scales of the respective symptom measure. 4The metric of each latent factor was set by fixing one of the unstandardized factor loadings to 1.0 2Bryant and Satorra (2012)outlined a modified method for LISREL users to compute the SDCS. This modified method allows LISREL users to compute a SDCS in a manner that is analogous to the method used by users of other SEM software packages. Bryant and Satorra’s method involves computing a scaling correction factor (c) for each model by dividing the normal theory weighted least squares chi-square value by the Satorra–Bentler chi-square value. All SDCS values reported in this article correspond to Bryant and Satorra’s method. 3Brown (2006)stated that “when the higher order model is overidenti- fied, the nested 2test can be used to determine whether the specification produces a significant degradation in fit relative to the first-order solution” (p. 332). Following Brown, the SDCS was used to compare the second- order model with the correlated four-factor first-order model in the present research. 4The rationale for using item indicators instead of item parcels for these latent constructs was threefold. First, the measures used in the present research were generally too brief to form a sufficient number of item parcels. Second, it is generally unknown whether the underlying structure of the items in a parcel is unidimensional, whereas prior research has supported the items indicators used for each construct as being unidimen- sional. Third, research has shown that models with parcels do not outper- form models based on individual items (seeBrown, 2006). Because Abramowitz et al. (2010)found that a one-factor solution of the DOCS items provided a significantly poorer fit to the data relative to a four-factor solution, the four DOCS scales (as scaled scores) were used as indicators rather than using each DOCS item as a separate indicator. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1326 FERGUS (Brown, 2006). Partial path coefficients were used to investigate unique relations between the IU and symptom factors. Latent correlations among the constructs in the structural regres- sion models are presented inTable 3. 5Results from the structural regression models indicated the presence of a statistical suppres- sion effect, such that the IUS–12-Prospective construct tended to evidence significantnegativestandardized partial path coefficients predicting the symptom constructs— depression .34,p .01; generalized anxiety .14,ns; obsessive– compulsive .42, p .01— despite IUS–12–Prospective sharing a significantposi- tivelatent correlation with each symptom construct. Supplemental analyses were completed to examine this suppression effect. Struc- tural regression models in which IUS–12–Prospective and IUS– 12–Inhibitory were the only two exogenous constructs in the model indicated that the overlap among these two constructs appeared to be affecting the partial path coefficients between the constructs and criteria. Specifically, in the models with either depression or obsessive– compulsive symptoms as the endogenous constructs, the IUS–12–Prospective construct shared nonsignifi- cant unique relations with these symptom constructs after the variance shared with the IUS–12–Inhibitory construct was con- trolled. Moreover, the IUS–12–Inhibitory construct shared positive unique relations that werelargerin magnitude than the latent correlations with these symptom constructs after the variance shared with the IUS–12–Prospective construct was controlled. The standardized partial path coefficients from these supplemental structural regression models, in which IUS–12–Prospective and IUS–12–Inhibitory were the only two exogenous constructs pre- dicting the respective symptom construct, were as follows: depres- sion (Prospective –.11,ns; Inhibitory .72,p .01); gener- alized anxiety (Prospective .16,p .05; Inhibitory .52,p .01); and obsessive– compulsive (Prospective .10,ns; Inhibi- tory .65,p .01). Because of the impact of each IUS–12 construct on one another in these models, a composite IUS–12 construct, which consisted of all 12 IUS–12 items loading a single latent construct, was used for the results reported from the struc- tural regression analyses. Fit statistics from the structural regression models, in which the correlated IUS–12, OBQ–P/C, and IUI–A constructs were allowed to predict the respective symptom construct, are presented inTable1. With the exception of slightly elevated RMSEA statistics, all of the fit statistics met or exceeded the specified guidelines. Partial path coefficients from the structural regression models are pre- sented inTable 4. As shown, the OBQ–P/C and IUI–A constructs both shared unique relations with each symptom construct. The IUS–12 construct, however, only shared a unique relation with the depression construct. Collectively, the IU constructs explained a large amount of variance in the symptom constructs. Discussion IU has received increased attention from clinical researchers in recent years, which highlights the need for valid assessment tools of this construct. Whereas in the existing literatureFreeston et al.’s (1994)IUS has been the measure most often used to assess IU, additional self-report measures exist for researchers and clinicians to choose from when assessing this construct. Unfortunately, little systematic research has been completed to compare extant self- report measures of IU. The present research helped fill this gap in the literature by examining the structure and incremental explan- atory power of three self-report measures that were developed to assess IU—the IUS–12 (Carleton et al., 2007;Freeston et al., 1994), OBQ–P/C (Moulding et al., 2011;Steketee, Frost, & OC- CWG, 2005), and IUI–A (Gosselin et al., 2008). The present results indicated that all three of the measures can be conceptual- ized as assessing IU, although each measure appears to assess a distinct aspect of this construct. 5Kline (2011)advocated using a two-step process when examining structural regression models, with the first step consisting of fitting mea- surement models to the data before proceeding to testing structural regres- sion models. The benefit of using this two-step process is that it is easier to locate the source of any poorly fitting structural regression models. This two-step process was adhered to when testing the reported structural regression models. However, given the adequate fit of the structural re- gression models, only the fit statistics from the structural regression models are reported for ease of interpretation of the results from the structural regression models. The measurement models consisted of removing path coefficients from the exogenous to the endogenous constructs and freeing latent factor intercorrelations. The latent factor correlations reported in Table 3were obtained from the initial measurement models. Table 1 Goodness-of-Fit Statistics for Tested Models Model 2 NTWLS 2 SB 2 dfRMSEA RMSEA 90% CI CFI NNFI SRMR Structure First-order models Correlated four-factor 1794.64 2131.30 1769.67 458 .068 [.065, .071] .982 .980 .051 Correlated three-factor 2069.63 2795.17 2325.55 461 .081 [.077, .084] .974 .972 .053 One-factor 3195.60 4404.13 3622.21 464 .105 [.101, .108] .956 .953 .067 Correlated two-factor 2641.29 3692.48 3058.38 463 .095 [.092, .098] .964 .962 .061 Second-order model Higher order factor 1795.24 2127.96 1767.77 460 .068 [.064, .071] .982 .981 .052 Incremental explanatory power Structural regression models Depression 2655.98 3539.60 3016.19 696 .073 [.071, .076] .976 .974 .057 Generalized anxiety 2626.35 3448.80 2931.97 696 .072 [.069, .075] .977 .976 .052 Obsessive–compulsive 2356.56 3157.60 2676.90 588 .076 [.073, .078] .974 .972 .054 Note. NTWLS normal theory weighted least-squares; SB Satorra–Bentler; RMSEA root-mean-square error of approximation; CI confidence interval; CFI comparative fit index; NNFI nonnormed fit index; SRMR standard root-mean-square residual. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1327 INTOLERANCE OF UNCERTAINTY The distinctiveness of the three measures of IU is not surprising, given the different operational definitions of IU used by research- ers when creating the items of these measures.Gosselin et al. (2008)asserted that IU fundamentally consists of two defining features: (a) the tendency for an individual to find uncertaintyintolerable or unacceptable and (b) reactions to uncertainty that may result from this tendency. As noted earlier, when developing the IUS items,Freeston et al. (1994)used a definition of IU that only pertained to reactions to uncertainty. Because the IUS–12 is a short-form of the IUS, the items of the IUS–12, by extension, are also representative ofFreeston et al.’s (1994)definition of IU. As Table 2 Completely Standardized Factor Loadings From Select Confirmatory Factor Analyses Scale/item One-factor modelFour-factor model I II III IV Intolerance of Uncertainty Scale–12-item version IUS–12 Item 1 .71 .78 IUS–12 Item 2 .60 .68 IUS–12 Item 3 .70 .78 IUS–12 Item 4 .40 .53 IUS–12 Item 5 .59 .68 IUS–12 Item 6 .66 .81 IUS–12 Item 7 .68 .82 IUS–12 Item 8 .57 .66 IUS–12 Item 9 .67 .75 IUS–12 Item 10 .70 .82 IUS–12 Item 11 .50 .60 IUS–12 Item 12 .70 .74 Obsessive Beliefs Questionnaire–Perfectionism/Certainty OBQ–P/C Item 1 .50 .78 OBQ–P/C Item 2 .49 .63 OBQ P/C Item 3 .53 .84 OBQ–P/C Item 4 .42 .64 OBQ–P/C Item 5 .54 .64 Intolerance of Uncertainty Index–Part A IUI–A Item 1 .79 .80 IUI–A Item 2 .82 .83 IUI–A Item 3 .80 .81 IUI–A Item 4 .79 .80 IUI–A Item 5 .81 .82 IUI–A Item 6 .68 .68 IUI–A Item 7 .85 .85 IUI–A Item 8 .84 .85 IUI–A Item 9 .85 .86 IUI–A Item 10 .78 .79 IUI–A Item 11 .89 .90 IUI–A Item 12 .78 .79 IUI–A Item 13 .58 .59 IUI–A Item 14 .87 .87 IUI–A Item 15 .67 .66 Note.N 624. All factor loadings significant atp .01 (two-tailed). Table 3 Latent Factor Correlations Factor12345 1. IUS–12-Total — 2. IUS–12-Prospective — 3. IUS–12-Inhibitory .80 — 4. OBQ–Perfectionism/Certainty .64 .61 .60 — 5. IUI–Part A .87 .82 .81 .63 — 6. Depression .60 .47 .64 .55 .60 7. Generalized Anxiety .66 .58 .66 .56 .70 8. Obsessive–Compulsive .53 .41 .57 .55 .58 Note.N 624. All latent factor correlations significant atp .01 (two-tailed). IUS–12 Intolerance of Uncertainty Scale–12-item version; OBQ Obsessive Beliefs Questionnaire; IUI Intolerance of Uncertainty Index. Table 4 Standardized Partial Path Coefficients From Structural Regression Models Endogenous constructExogenous construct IUS–12 OBQ–P/C IUI–AR 2 Depression .24 .25 .22 .42 Generalized anxiety .14 .17 .47 .52 Obsessive–compulsive .01 .31 .37 .39 Note.N 624. IUS–12 Intolerance of Uncertainty Scale–12-item version; OBQ–P/C Obsessive Beliefs Questionnaire–Perfectionism/ Certainty; IUI–A Intolerance of Uncertainty Index–Part A. Partial path coefficient significant atp .01 (two-tailed). This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1328 FERGUS noted byCarleton (2012), the IUS–12 items can be viewed as assessing cognitive (e.g.,Unforeseen events upset me greatly) and behavioral (e.g.,The smallest doubt can stop me from acting) reactions to uncertainty. The items assessing these two reactions to uncertainty were factorially distinct in the present research, as represented by Prospective IU (i.e., cognitive reactions) and In- hibitory IU (i.e., behavioral reactions). However, the present re- sults failed to replicate preliminary results that the separate scales of the IUS–12 share differential unique relations with criterion after the variance shared by the other IUS–12 scales has been accounted for (McEvoy & Mahoney, 2011). Moreover, the present results suggest that after other aspects of IU have been accounted for, the reactions to uncertainty assessed by the IUS–12 are par- ticularly relevant to depression symptoms. The lack of a unique relation between the IUS–12 and generalized anxiety symptoms in the present research was particularly surprising, given the robust relations observed between worry and the IUS in prior research (Koerner & Dugas, 2006). However,Gentes and Ruscio (2011) noted that the bulk of existing research has examined IU in relation to worry, rather than to generalized anxiety symptoms more broadly.Spitzer et al.’s (2006)GAD–7 provides a broad assess- ment of generalized anxiety symptoms, as the items of this mea- sure are not solely focused on worry. The use of this measure could help account for the lack of a unique relation between the IUS–12 and generalized anxiety symptoms found in the present research. Whereas the items of the OBQ–P/C (e.g.,I must keep working until it’s done exactly right) appear less conceptually linked to definitions of IU frequently cited in the existing literature relative to the items of either the IUS–12 or IUI–A, the OBQ–P/C was nevertheless found to belong to the same higher order factor as both of these other two measures. One tenable reason for this finding is that, similar to the items of the IUS–12, the items of the OBQ–P/C could be viewed as assessing reactions to uncertainty. For example,Birrell et al. (2011)stated that “individuals who are intolerant of uncertainty respond to situations in ways that reduce the level of uncertainty. This is often achieved through obtaining sufficient information for the individual to judge the situation as predictable (and, therefore, safe)” (p. 1205). Perfectionism has been conceptualized as a reaction to uncertainty that functions in this manner. Specifically, some theorists have viewed perfection- ism within obsessive– compulsive disorder as an attempt to avoid feelings of uncertainty via striving to achieve a perfect state in perceptions and behavior (Frost, Novara, & Rhéaume, 2002). Feelings of uncertainty (e.g., uncertainty about risk of harm) might be reduced, at least temporarily, through perfection-seeking be- havior that makes feared situations become more predictable. At the same time, Frost et al. (2002) noted that perfection-seeking behavior might also increase uncertainty, thereby engendering even greater distress for individuals with obsessive– compulsive disorder. Because IU and perfectionism both appear to have trans- diagnostic importance (Carleton, Mulvogue, et al., 2012;Egan, Wade, & Shafran, 2011), perfectionism might be viewed as a reaction to uncertainty that spans across emotional disorders. Consistent with assertions made byGosselin et al. (2008), the IUI–A (e.g.,I have difficulty tolerating the possibility that a negative event may happen to me) would appear to be the only one of the three measures targeted in the present research that assesses the tendency for an individual to find uncertainty intolerable or unacceptable. This feature of IU is important, asDugas et al.(2001)opined that “intolerance of uncertainty may be defined more specifically as the excessive tendency of an individual to consider it unacceptable that a negative event may occur, however small the probability of its occurrence” (p. 552). The present results (i.e., that the IUI–A shares unique relations with symptom measures while accounting for the variance shared with measures that seem to assess the other defining feature of IU—reactions to uncertainty) further support the importance of assessing the ten- dency for an individual to find uncertainty intolerable or unaccept- able. Taken with the noted content differences across measures, the present results stand in contrast toCarleton’s (2012)assertion that the IUS–12 assessesthecore aspects of IU that span across psychological disorders, but rather indicate that the IUS–12, OBQ–P/C, and IUI–A each assess a separate core aspect of IU that is relevant to symptoms of psychological disorders. The future directions of research into the assessment of IU are plentiful and will help speak to the relative merits of the IUS–12, OBQ–P/C, and IUI–A. For example, it is important for researchers to examine whether these three measures demonstrate differential utility in clinical practice, as well as to examine the incremental explanatory power of these measures within prospective longitu- dinal studies. In addition, it might be informative to extend the present results by investigating whether the full-length IUS items (usingSexton & Dugas’s 2009factor structure) function similarly as the IUS–12 items in relation to the items of the OBQ–P/C and IUI–A. Given the interest in examining the assessment of IU among racial/ethnic minorities (Norton, 2005), replication of these findings among respondents with greater racial/ethnic diversity is needed. Moreover, whereas the bulk of the extant literature con- cerning the assessment of IU has been completed in nonclinical samples (Berenbaum et al., 2008;Buhr & Dugas, 2002;Carleton et al., 2007,2010;Freeston et al., 1994;Gosselin et al., 2008; Moulding et al., 2011;Norton, 2005;Sexton & Dugas, 2009), extending the present findings to samples that consistently score highly on both IU and symptom measures is warranted. Finally, depression, generalized anxiety, and obsessive– compulsive symp- toms were specifically chosen because these symptom types have been most often targeted in available IU literature (Gentes & Ruscio, 2011). Nonetheless, it is important to broaden the symp- tom assessment in future research to include other symptom types of potential importance (e.g., panic, social anxiety;Carleton, Mul- vogue, et al., 2012). Although the present results suggest that the IUS–12, OBQ– P/C, and IUI–A each assess IU, the distinct content assessed by each measure warrants attention. Specifically, researchers and clinicians should carefully attend to the differences in the item content across these measures when selecting one of the measures to assess IU. Future research efforts examining the relative merits of the IUS–12, OBQ–P/C, and IUI–A will be important in eluci- dating whether one of these measures might be considered a preferred measure of IU. Through such efforts, the assessment of IU can be refined, and the transdiagnostic importance of IU better understood. References Abramowitz, J. S., Deacon, B. J., Olatunji, B. O., Wheaton, M. G., Berman, N. C., Losardo, D.,…Hale, L. R. (2010). Assessment of obsessive-– compulsive symptom dimensions: Development and evaluation of the This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1329 INTOLERANCE OF UNCERTAINTY Dimensional Obsessive–Compulsive Scale.Psychological Assessment, 22,180 –198.doi:10.1037/a0018260 Antony, M. M., Bieling, P. J., Cox, B. J., Enns, M. W., & Swinson, R. P. (1998). Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample.Psychological Assessment, 10,176 –181.doi:10.1037/1040- 3590.10.2.176 Berenbaum, H., Bredemeier, K., & Thompson, R. J. (2008). Intolerance of uncertainty: Exploring its dimensionality and associations with need for cognitive closure, psychopathology, and personality.Journal of Anxiety Disorders, 22,117–125.doi:10.1016/j.janxdis.2007.01.004 Birrell, J., Meares, K., Wilkinson, A., & Freeston, M. (2011). Toward a definition of intolerance of uncertainty: A review of factor analytical studies of the Intolerance of Uncertainty Scale.Clinical Psychology Review, 31,1198 –1208.doi:10.1016/j.cpr.2011.07.009 Brown, T. A. (2006).Confirmatory factor analysis for applied research. New York, Y: Guilford Press. Brown, T. A., Chorpita, B. F., Korotitsch, W., & Barlow, D. H. (1997). Psychometric properties of the Depression Anxiety Stress Scales (DASS) in clinical samples.Behaviour Research and Therapy, 35, 79 – 89.doi:10.1016/S0005-7967(96)00068-X Bryant, F. B., & Satorra, A. (2012). Principles and practice of scaled difference chi-square testing.Structural Equation Modeling, 19,372– 398.doi:10.1080/10705511.2012.687671 Buhr, K., & Dugas, M. J. (2002). The Intolerance of Uncertainty Scale: Psychometric properties of the English version.Behaviour Research and Therapy, 40,931–945.doi:10.1016/S0005-7967(01)00092-4 Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechan- ical Turk: A new source of inexpensive, yet high-quality, data?Per- spectives on Psychological Science, 6,3–5.doi:10.1177/ 1745691610393980 Carleton, R. N. (2012). The intolerance of uncertainty construct in the context of anxiety disorders: Theoretical and practical perspectives. Expert Review of Neurotherapeutics, 12,937–947.doi:10.1586/ern .12.82 Carleton, R. N., Gosselin, P., & Asmundson, G. J. G. (2010). The Intol- erance of Uncertainty Index: Replication and extension with an English sample.Psychological Assessment, 22,396 – 406.doi:10.1037/a0019230 Carleton, R. N., Mulvogue, M. K., Thibodeau, M. A., McCabe, R. E., Antony, M. M., & Asmundson, G. J. G. (2012). Increasingly certain about uncertainty: Intolerance of uncertainty across anxiety and depres- sion.Journal of Anxiety Disorders, 26,468 – 479.doi:10.1016/j.janxdis .2012.01.011 Carleton, R. N., Norton, P. J., & Asmundson, G. J. G. (2007). Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. Journal of Anxiety Disorders, 21,105–117.doi:10.1016/j.janxdis.2006 .03.014 Carleton, R. N., Weeks, J. W., Howell, A. N., Asmundson, G. J. G., Antony, M. M., & McCabe, R. E. (2012). Assessing the latent structure of the intolerance of uncertainty construct: An initial taxometric analy- sis.Journal of Anxiety Disorders, 26,150 –157.doi:10.1016/j.janxdis .2011.10.006 Coles, M. E., Cook, L. M., & Blake, T. R. (2007). Assessing obsessive compulsive symptoms and cognitions on the Internet: Evidence for the comparability of paper and Internet administration.Behaviour Research and Therapy, 45,2232–2240.doi:10.1016/j.brat.2006.12.009 Dugas, M. J., Gosselin, P., & Ladouceur, R. (2001). Intolerance of uncer- tainty and worry: Investigating specificity in a nonclinical sample. Cognitive Therapy and Research, 25,551–558.doi:10.1023/A: 1005553414688 Dugas, M. J., & Robichaud, M. (2007).Cognitive– behavioral treatment for generalized anxiety disorder: From science to practice. New York, NY: Routledge.Egan, S. J., Wade, T. D., & Shafran, R. (2011). Perfectionism as a transdiagnostic process: A clinical review.Clinical Psychology Review, 31,203–212.doi:10.1016/j.cpr.2010.04.009 Fergus, T. A., & Wu, K. D. (2011). Searching for specificity between cognitive vulnerabilities and mood and anxiety symptoms.Journal of Psychopathology and Behavioral Assessment, 33,446 – 458.doi: 10.1007/s10862-011-9245-6 Freeston, M. H., Rhéaume, J., Letarte, H., Dugas, M. J., & Ladouceur, R. (1994). Why do people worry?Personality and Individual Differences, 17,791– 802.doi:10.1016/0191-8869(94)90048-5 Frost, R. O., Novara, C., & Rhéaume, J. (2002). Perfectionism in obsessive– compulsive disorder. In R. O. Frost & G. Steketee (Eds.), Cognitive approaches to obsessions and compulsions: Theory, assess- ment, and treatment(pp. 91–105). Oxford, United Kingdom: Pergamon. doi:10.1016/B978-008043410-0/50007-6 Gentes, E. L., & Ruscio, A. M. (2011). A meta-analysis of the relation of intolerance of uncertainty to symptoms of generalized anxiety disorder, major depressive disorder, and obsessive– compulsive disorder.Clinical Psychology Review, 31,923–933.doi:10.1016/j.cpr.2011.05.001 Gosselin, P., Ladouceur, R., Evers, A., Laverdiere, A., Routhier, S., & Tremblay-Picard, M. (2008). Evaluation of intolerance of uncertainty: Development and validation of a new self-report measure.Journal of Anxiety Disorders, 22,1427–1439.doi:10.1016/j.janxdis.2008.02.005 Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alterna- tives.Structural Equation Modeling, 6,1–55.doi:10.1080/ 10705519909540118 Hunsley, J. (2003). Introduction to the special section on incremental validity and utility in clinical assessment.Psychological Assessment, 15, 443– 445.doi:10.1037/1040-3590.15.4.443 Jöreskog, K. G., & Sörbom, D. (2007). LISREL (Version 8.80) [Computer software]. Chicago, IL: Scientific Software. Kertz, S., Bigda-Peyton, J., & Bjorgvinsson, T. (in press). Validity of the Generalized Anxiety Disorder–7 Scale in an acute psychiatric sample. Clinical Psychology and Psychotherapy.doi:10.1002/cpp.1802 Khawaja, N. G., & Yu, L. N. H. (2010). A comparison of the 27-item and 12-item Intolerance of Uncertainty Scales.Clinical Psychologist, 14, 97–106.doi:10.1080/13284207.2010.502542 Kline, R. B. (2011).Principles and practice of structural equation mod- eling(3rd ed.). New York, NY: Guilford Press. Koerner, N., & Dugas, M. J. (2006). A cognitive model of generalized anxiety disorder: The role of intolerance of uncertainty. In G. C. L. Davey & A. Wells (Eds.),Worry and its psychological disorders: Theory, assessment and treatment(pp. 201–216). Sussex, United King- dom: Wiley.doi:10.1002/9780470713143.ch12 Lovibond, S. H., & Lovibond, P. E. (1995).Manual for the depression anxiety stress scales. Sydney: Psychology Foundation of Australia. McEvoy, P. M., & Mahoney, A. E. J. (2011). Achieving certainty about the structure of intolerance of uncertainty in a treatment-seeking sample with anxiety and depression.Journal of Anxiety Disorders, 25,112–122. doi:10.1016/j.janxdis.2010.08.010 Moulding, R., Anglim, J., Nedeljkovic, M., Doron, G., Kyrios, M., & Ayalon, A. (2011). The Obsessive Beliefs Questionnaire (OBQ): Exam- ination in nonclinical samples and development of a short version. Assessment, 18,357–374.doi:10.1177/1073191110376490 Norton, P. J. (2005). A psychometric analysis of the Intolerance of Un- certainty Scale among four racial groups.Journal of Anxiety Disorders, 19,699 –707.doi:10.1016/j.janxdis.2004.08.002 O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments & Computers, 32,396 – 402. doi:10.3758/BF03200807 Olatunji, B. O., Broman-Fulks, J. J., Bergman, S. M., Green, B. A., & Zlomke, K. R. (2010). A taxometric investigation of the latent structure This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1330 FERGUS of worry: Dimensionality and associations with depression, anxiety, and stress.Behavior Therapy, 41,212–228.doi:10.1016/j.beth.2009.03.001 Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.),Latent variable analysis: Applications for development research(pp. 399 – 419). Thousand Oaks, CA: Sage. Sexton, K., & Dugas, M. (2009). Defining distinct negative beliefs about uncertainty: Validating the factor structure of the Intolerance of Uncer- tainty Scale.Psychological Assessment, 21,176 –186.doi:10.1037/ a0015827 Steketee, G., Frost, R., and Obsessive Compulsive Cognitions Working Group. (2001). Development and initial validation of the Obsessive Beliefs Questionnaire and the Interpretation of Intrusions Inventory. Behaviour Research and Therapy, 39,987–1006.doi:10.1016/S0005- 7967(00)00085-1 Steketee, G., Frost, R., and Obsessive Compulsive Cognitions Working Group. (2003). Psychometric validation of the Obsessive Beliefs Ques-tionnaire and Interpretation of Intrusions Inventory–Part 1.Behaviour Research and Therapy, 41,863– 878.doi:10.1016/S0005- 7967(02)00099-2 Steketee, G., Frost, R., and Obsessive Compulsive Cognitions Working Group. (2005). Psychometric validation of the Obsessive Belief Ques- tionnaire and Interpretation of Intrusions Inventory–Part 2: Factor anal- yses and testing of a brief version.Behaviour Research and Therapy, 43, 1527–1542.doi:10.1016/j.brat.2004.07.010 Spitzer, R. L., Kroenke, K., Williams, J. B., & Lowe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD–7.Ar- chives of Internal Medicine, 166,1092–1097.doi:10.1001/archinte.166 .10.1092 Received December 5, 2012 Revision received April 9, 2013 Accepted July 11, 2013 This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 1331 INTOLERANCE OF UNCERTAINTY
Validity Review the articles by Fergus (2013), Kosson, et al. (2013) and Mathieu, Hare, Jones, Babiak, & Neumann (2013). Analyze the information presented in these articles on factor analysis and des
BRIEF REPORT Factor Structure of the B-Scan 360: A Measure of Corporate Psychopathy Cynthia Mathieu Universite´ du Que´ bec a` Trois-Rivie` res Robert D. Hare and Daniel N. Jones University of British Columbia Paul Babiak Anubis-Research, Hopewell Junction, New York Craig S. NeumannUniversity of North Texas Psychopathy is a clinical construct defined by a cluster of personality traits and behaviors, including grandiosity, egocentricity, deceptiveness, shallow emotions, lack of empathy or remorse, irresponsibility, impulsivity, and a tendency to ignore or violate social norms. The majority of empirical research on psychopathy involves forensic populations most commonly assessed with the Psychopathy Checklist– Revised (PCL-R), a 20-item rating scale that measures 4 related factors or dimensions (Interpersonal, Affective, Lifestyle, and Antisocial) that underpin the superordinate construct of psychopathy. Recently, researchers have turned their attention to the nature and implications of psychopathic features in the workplace. This research has been hampered by the lack of an assessment tool geared to the corporate/ organizational world. Here we describe the B-Scan 360, an instrument that uses ratings of others to measure psychopathic features in workplace settings. In this study, large samples of participants used an online survey system to rate their supervisors on the B-Scan 360. Exploratory and confirmatory factor analyses supported a reliable 20-item, 4-factor model that is consistent with the PCL-R 4-factor model of psychopathy. Although more research is needed before the B-Scan 360 can be used in organizational settings, we believe that these results represent an important step forward in the study of corporate psychopathy. Keywords:psychopathy, corporate psychopathy, B-Scan, corporate settings Scholars studying deviant behavior in the workplace have shown strong interest in narcissism and Machiavellianism, but only recently have researchers turned their attention to another “dark personality,” corporate psychopathy (Babiak & Hare, 2006). The public commonly associates psychopathy with individuals who murder, rape, assault, rob, or commit other serious crimes.Offenders high on psychopathy generally are at greater risk for committing such crimes than are other offenders, but this should not obscure the facts that at the measurement level psychopathy is dimensional (Guay, Ruscio, Knight, & Hare, 2007), that most offenders are not psychopaths, and that many high psychopathy individuals manage to avoid being brought into formal contact with the criminal justice system (e.g., Babiak, Neumann, & Hare, 2010). An international standard for the assessment of psychopathy in forensic populations is thePsychopathy Checklist–Revised (PCL-R; Hare, 2003), a clinical construct rating scale administered by qualified clinicians from interview and collateral information. A derivative of the PCL-R, thePsychopathy Checklist: Screening Version(PCL: SV; Hart, Cox, & Hare, 1995) is widely used for assessing psychopathy in civil psychiatric and community popu- lations, and is closely related to the PCL-R, both conceptually and empirically (Hare, Neumann, & Widiger, in press). The psycho- metric properties and correlates of the PCL-R and the PCL: SV are well known, and evidence for their reliability and validity as measures of psychopathy is extensive (Hare & Neumann, 2008; Hare et al., in press). Confirmatory factor analyses (CFAs) of very large data sets indicate that both instruments can be modeled in terms of four strongly correlated unidimensional factors (with well-delineated item-to-factor relations) that are accounted for by a single superordinate factor (e.g., Neumann & Hare, 2008; Neu- mann, Hare, & Johansson, in press; Neumann, Hare, & Newman, This article was published Online First July 9, 2012. Cynthia Mathieu, Business Department, Universite´ du Que´ bec a` Trois- Rivie` res, Que´ bec, Canada; Robert D. Hare and Daniel N. Jones, Depart- ment of Psychology, University of British Columbia, Vancouver, British Columbia, Canada; Paul Babiak, Anubis-Research, Hopewell Junction, New York; Craig S. Neumann, Department of Psychology, University of North Texas. Daniel N. Jones is now at the Department of Psychology, University of Texas, El Paso. Paul Babiak and Robert D. Hare do not receive any financial compen- sation directly from the B-Scan; however, it is being developed for future commercial purposes. Robert D. Hare receives royalties from the sale of the PCL-R and its derivatives. This research was supported by grants from the Donner Foundation to Cynthia Mathieu, Robert D. Hare, and Craig S. Neumann. We would like to note that all participating coauthors have contributed equally to the elaboration of the present article. We thank Kylie Neufeld for her assistance in preparing this article. Correspondence concerning this article should be addressed to Cynthia Mathieu, Business Department, Universite´ du Que´ bec a` Trois-Rivie` res, P.O. 500, Trois-Rivie` res, Quebec, G9A 5H7, Canada. E-mail: [email protected] Psychological Assessment© 2012 American Psychological Association 2013, Vol. 25, No. 1, 288 –2931040-3590/13/$12.00 DOI: 10.1037/a0029262 288 2007). The four factors (and for illustrative purposes, the PCL: SV items that comprise them) are as follows:Interpersonal(Superfi- cial, Grandiose, Deceitful),Affective(Lacks remorse, Lacks em- pathy, Doesn’t accept responsibility for actions),Lifestyle(Impul- sive, Lacks realistic goals, Irresponsible), andAntisocial(Poor behavioral controls, Adolescent antisocial behavior, Adult antiso- cial behavior). Cooke and Michie (2001) have argued that antiso- cial features (the Antisocial factor) should not be included in the assessment of psychopathy. However, traditional clinical concep- tions of psychopathy are replete with antisociality, and it is diffi- cult to understand how the defining traits of the construct could be measured without reference to antisocial behaviors (Hare, 2003; Hare & Neumann, 2008). As Lynam and Miller (in press) put it, Antisocial behavior [ASB] plays a clear and prominent role in psy- chopathy. . . . In fact, if there is an essential behavioral feature in common across the conceptualizations [of psychopathy], it is the presence of ASB. Any description of psychopathy is incomplete without ASB. We included antisocial features in the current study of corporate psychopathy. Given the defining features of psychopathy (that is, personality traits that make it easy to defraud, bilk, scam, dominate, and control), along with a context that includes loosely regulated financial environments, plenty of opportunities, lax regulatory oversight, huge rewards and trivial penalties, it is not difficult to suspect that psychopathy should be closely connected to corporate misbehavior and white-collar crime (Perri, 2011). Corporate Psychopathy In a recent study of 203 upper-level managers, Babiak, Neu- mann, and Hare (2010) found that the PCL-R—particularly its interpersonal component—was positively associated with in-house ratings of Charisma/Presentation style (creativity, strategic think- ing and communication skills) and negatively associated with ratings of Responsibility/Performance (being a team player, lead- ership and management skills, and overall accomplishments). The authors concluded that the ability to charm, manipulate, and de- ceive others allowed psychopathic leaders to achieve apparent success in their careers despite negative performance ratings and behaviors potentially harmful to the corporation and its personnel. Others have found psychopathy to be positively related to uneth- ical decision making (Stevens, Deuling, & Armenakis, 2012), a recurring theme in the business world during the past few years. At first glance, individuals with psychopathic tendencies may seem attractive during recruitment and may succeed in the short-term, but we argue that similar to individuals with narcissistic tenden- cies, the destructive aspects of the personality will appear in the long-run (Campbell, Hoffman, Campbell, & Marchisio (2011). A major impediment to advancing our understanding of corpo- rate psychopathy is the unavailability of a suitable instrument for assessing the construct in business settings. Rather than relying on formal clinical assessments or extant self-reports to assess psy- chopathy, Babiak and Hare began working several years ago on the development of an instrument for rating psychopathic features in corporate and organizational settings. The original item set was based on a multitude of behaviors, attitudes, and judgments con- sidered problematic (by human resources personnel and industrial/organizational psychologists) in corporate succession plans, not all of which were possible indicators of psychopathy. The result was the 113-item Business-Scan 360, referred to here as the B-Scan, designed as a rating scale in which various members of an orga- nization rate others, including supervisors, peers, and subordinates (Babiak & Hare, 2012). The goal of the present study was to determine the factor structure of the B-Scan as a measure of psychopathy when used by subordinates to rate their supervisors in corporate settings. It has proven difficult to enlist the participation of organizations to validate the B-Scan. Fortunately, the recent introduction of online internet surveys has made it possible for researchers to collect large amounts of data that are reasonably representative of populations of interest. Amazon’s Mechanical Turk (MTurk; www.mturk.com) has proven to be particularly useful in this regard (e.g., Buhrmester, Kwang, & Gosling, 2011). MTurk is an online marketplace connecting requesters offering payment for completion of human intelligence tasks (HITs) and workers will- ing to complete such tasks. In terms of generalizability, many studies report that samples obtained through MTurk are more representative than studies using student participants (e.g., Beh- rend, Sharek, Meade, & Wiebe, 2011; Buhrmester et al., 2011). Mturk is a useful option for employee-focused research (Barger, Behrend, Sharek, & Sinar, 2011). Current Study Early versions of the B-Scan were designed to capture the four-factor model of psychopathy, but the process was largely rational, and the amount of empirical data available was too small to conduct satisfactory statistical analyses. The 113-item set also was rather large for routine use and might have contained items that were not directly related to the psychopathy construct. In this study, we used MTurk to collect two large independent samples of data from business personnel who rated their supervisors on the original B-Scan items and on several relevant external variables. In Sample 1, we conducted exploratory analyses to delineate the factor structure of the B-Scan. In Sample 2, we conducted a CFA of the items derived from the analyses in Sample 1. Sample 1: Exploratory Analyses Participants in Sample 1 were 340 working adults recruited on Amazon’s MTurk website (57% women; mean age 33.64 years, SD 11.78; 74% European heritage, 7% African heritage, 7% East Asian, 4% South Asian, 8% other mixed ethnicities). On MTurk, all participants preselect tasks they wish to complete, for which they receive a nominal fee. The task in this case was to “rate your boss’s personality.” Of those who took the survey, 2% reported being a CEO or senior manager, 12% were middle man- agement, 13% were line management, and 73% did not hold a managerial position. Salaries ranged from less than $25K to over $200K per year. The demographics of participants’ immediate supervisors or bosses were also collected: 40% were women, 75% were of European heritage, and mean age was 45.8 years (SD 10.0). The MTurk instructions for the B-Scan ratings were as follows: “Please answer the following questions with respect to your cur- rent (or most recent) boss or supervisor. If you have (or had) more 289 B-SCAN than one, answer with respect to the most relevant to your career.” Participants responded to each item on a 5-point Likert-like scale (1–5) fromStrongly DisagreetoStrongly Agree. Sample items are as follows: Comes across as smooth, polished and charming; Shows no regret for making decisions that harm the company, shareholders, or employees. In this sample, we used several statistical procedures to explore the underlying dimensions of the initial pool of B-Scan items. The first and primary procedure permitted examination of the Parallel Analysis (PA; Horn, 1965) and Velicer’s minimal average partial (MAP; Zwick & Velicer, 1986)criteria for factor extraction. Given that psychopathy scores usually have nonnormal distributions and that the items were answered on an ordinal scale, we conducted a PA and MAP on both the polychoric and Pearson correlation matrixes (see Cho, Li, & Bandalos, 2009). We used the statistical package R (http://www.r-project.org/), which is capable of han- dling a polychoric and Pearson matrix when calculating such statistics. To supplement this approach, we examined eigenvalues and scree plot graphs—all of which agreed with the PA and MAP criteria. The second step was to conduct a series of exploratory factor analyses (EFAs) in order to isolate factors that fit the Hare Four- Factor model of psychopathy. Here we discuss only B-Scan factors directly related to psychopathy. These EFA procedures were car- ried out with Mplus because of its ability to model nonlinear (i.e., ordinal) data (Muthen & Muthen, 2010). In all cases, we used the mean and variance adjusted weighted least squares (WLSMV) estimation procedure. Given the ordinal nature of the items, the items were treated as polytomous and analyzed using polychoric correlations via WLSMV, a preferred method for this analysis (Muthen & Muthen, 2010). We note that use of maximum likeli- hood resulted in a similar pattern of findings. Results The results of the PA identified six factors. This solution was obtained for both polychoric and Pearson solutions. We then entered the data into an EFA procedure, extracting six factors. Two factors were not directly relevant to psychopathy but were consid- ered important in evaluating corporate potential and performance. We tentatively described them as an “ability” dimension (e.g., has the knowledge to perform his/her job well) and a “disruptive behavior” dimension (e.g., enjoys being disruptive at times). They were similar to the performance and presentation style ratings identified in the corporate psychopathy study by Babiak and col- leagues (2010). That is, although not directly part of the psychop- athy construct, they appear to be related to corporate performance and perhaps leadership style. Because the present study’s goal was to test the viability of a four-factor structure of psychopathy, similar to other well-established derivatives of the PCL-R, we decided to remove items that loaded on these two factors and items that did not sufficiently load on one of the remaining four factors. These two factors will be addressed in future research testing the B-Scan in organizational settings. This left us with a pool of 38 items. We then conducted an EFA on these items. The first four eigenvalues were greater than 1.5, suggesting a four-factor solu- tion. The fourth factor, which consisted of aggression-related items, had only five items with loadings greater than .40, as did manipulation-related items. Two other factors had more than fiveitems with a loading of .40 or greater. In order to create a balanced scale, we selected the best five from each factor (i.e., items that were not redundant with another item, had the highest loading, and/or had the lowest loading with other factors). The result was a set of 20 items (five per factor) to represent the reduced B-Scan scale. Finally, these items were subjected to a new technique available in Mplus referred to as exploratory structural equations modeling (ESEM). The advantage of ESEM is that items can freely cross load (EFA), but model fit (SEM) is also estimated. In this sense, it is a more open test of a model than conventional CFA, which usually involves specific item-to-factor relations. The ESEM identified four factors with an eigenvalue greater than 1.25. With the exception of one item on the Callous/Insensitive factor (“threatens co-workers”), all items loaded sufficiently on their respective factor (i.e., .40; e.g., Factor 1 mean loading .63, range: .48 to .89; Factor 2 mean loading .74, range: .65 to .86; Factor 3 mean loading .74, range: .67 to .82; Factor 4 mean loading .42, range: .31 to .51). The overall fit of the four-factor ESEM was acceptable (TLI .97; SRMR .03). All of these values exceed conventional cut-off criteria (Marsh, Hau, & Wen, 2004). Furthermore, each item clearly mapped onto the four well- knownPCL-R-based psychopathy factors (i.e., Interpersonal, Affective, Lifestyle, and Antisocial). However, given that the B-Scan (latent) factors are meant to have utility in a corporate environment, welabeled them as follows:Manipulative/ Unethical,Callous/Insensitive,Unreliable/Unfocused, andIntimi- dating/Aggressive. Coefficient alpha ( ) for the total scale score was .90. Table 1 lists the (manifest or observed) means, standard deviations, mean interitem correlations, factor intercorrelations, and for each factor. Sample 2: Confirmatory Factor Analysis In Sample 1 we identified a preliminary 20-item B-Scan scale consistent with the four PCL-based factors of psychopathy. The next step was to confirm this factor structure and its reliability. Participants in Sample 2 were 806 working adults recruited on Amazon’s MTurk website. The demographics of participants and supervisors were similar to those in Sample 1 (59% women; mean age 30.3,SD 10.3; 68% European heritage, 12% East Asian, 5% Latino, 6% African heritage, 4% other mixed ethnicities; 58% of the supervisors were men). In addition, participants reported that they had known their supervisors for an average of 4.5 years. After filling out demographics, participants then rated their super- visors online with the same 113 B-Scan items used in Sample 1, as part of a larger study on personalities in business. They received a nominal fee for their participation. We conducted a CFA on the 20-item B-Scan model identified in Sample 1. We again used the WLSMV estimation procedure as recommended when analyzing ordinal (i.e., Likert-like) data (Mu- then & Muthen, 2010). We used the Tucker-Lewis Index (TLI) and the standardized root mean square residual (SRMR) as our primary tests of model fit. Our current sample (e.g., 500) was sufficient for testing a model consisting of less than 70 parameters (i.e., 20 items in a 4-factor model). Specifically, the 20-item model esti- mates 46 parameters, which is well within the 10:1 subjects-to- parameters ratio recommended by Bentler (1995). 290 MATHIEU, HARE, JONES, BABIAK, AND NEUMANN Results The CFA results for the 20-item model (four correlated factors, five items per factor) selected in Sample 1 had acceptable fit to the Sample 2 data, 2(75) 692.46,p .001, TLI .93, SRMR .07. As shown in Figure 1, the resulting model replicated the four psychopathy factors found in Sample 1. In addition, the total score and each factor score were about as reliable as those in Sample 1 (see Table 1 for means, standard deviations, alphas, mean interitem correlations, and factor intercorrelations). General Discussion This study provides support for a four-factor structure of the B-Scan 360, an instrument designed for managers, subordinates,and peers to assess corporate psychopathy in others. In Sample 1, participants rated their supervisors on the original set of B-Scan items. Exploratory analyses of the items yielded a 20-item, four-factor model. In Sample 2, confirmatory factor analyses replicated this model. These results provide initial empirical evidence for a reliable structural model of the B-Scan, one that is conceptually similar to the four-factor structure of the PCL-R and its derivatives. Within an organizational setting, psychopathic traits are likely to find expression in behaviors that are self-serving, damaging to the organization and its members, or covertly unethical or illegal, such as manipulation, deception, intimidation, threats, coercion, bully- ing, fraud, and corruption. The features reflected in the four factors of the B-Scan seem to be related to workplace deviant behaviors previously described in business literature, such as organizational Table 1 B-Scan 360 Factors and Total Score: Means, Standard Deviations, Reliabilities, and Factor Intercorrelations B-Scan 36012345M(SD) MIC Sample 1 (n 340) 1. Manipulative/Unethical (.76) 3.45 (.76) .36 2. Callous/Insensitive .51 (.99) 2.61 (.99) .54 3. Unreliable/Unfocused .47 .51 (.87) 2.27 (.87) .50 4. Intimidating/Aggressive .46 .64 .44 (.92) 2.91 (.92) .40 5. Total Score .49 .73 .52 .61 (.90) 2.69 (.70) .30 Sample 2 (n 806) 1. Manipulative/Unethical (.70) 2.94 (.79) .31 2. Callous/Insensitive .48 (.82) 2.54 (.92) .49 3. Unreliable/Unfocused .43 .50 (.82) 2.21 (.81) .48 4. Intimidating/Aggressive .40 .59 .38 (.70) 2.78 (.84) .32 5. Total Score .46 .67 .49 .55 (.88) 2.62 (.65) .27 Note.For all factor intercorrelations,p .001. Alpha reliability is on the diagonal. Factor scores were calculated using summed item grouping scores. MIC mean interitem correlations. .48 .60 .38 .74 .77 .86.83 .83 .40 .81 .60 .62 .79 .77 .94 .63.86 .56 .54 .58 .55 .58 .66.68 .55 .85 Ingra ates him/herself Glib Uses charm Claims exper se Ra onalizesInsensi ve Not loyal No planning Unfocused Not pa ent UnreliableDrama c Threatens coworkersAsks harsh ques onsAngry In mida ng No empathy RemorselessCold inside Rarely shows emo ons Manipulative / Unethical Intimidating / Aggressive Unreliable / UnfocusedCallous / Insensitive Figure 1.Four-factor B-Scan model of psychopathy. 291 B-SCAN retaliatory behavior (Skarlicki, Folger, & Tesluk, 1999), work- place bullying (Mathisen, Einarsen, & Mykletun, 2011), and in- terpersonal deviance (Bolton, Becker, & Barber, 2010). Further- more, we believe that employees high on psychopathic traits will exhibit few behaviors that facilitate organizational functioning and many behaviors that harm the organization and its members. Such a pattern of behaviors, along with other factors, has been associ- ated to job performance (Rotundo & Sackett, 2002), an important variable in organizational settings. Implications of These Findings We believe that our findings provide important information about the underlying personality structure of counterproductive work behavior. Thus far, the business literature has focused more on identifying these toxic behaviors than on understanding their origins. We propose that psychopathic features in employees (as measured by the B-Scan) may help to explain these counterpro- ductive behaviors. The originality of the B-Scan lies in the fact that it is a 360 degree tool for the evaluation of psychopathic features in business settings. The present study focused on the validation of the B-Scan when used to evaluate psychopathic traits in others (data on the validation of the B-Scan Self, its sister version, will be presented in another manuscript). Given the self-serving and deceptive na- ture of the psychopathic personality, corroboration of self-report scores by others is vital, especially in business settings. Even trained industrial/organizational (I/O) psychologists, if not made aware of problematic evaluations or profiles, may think certain individuals are good fits for the organization when, in fact, they are potentially toxic. Babiak and colleagues (2010) observed that some executives tend to rely on their “gut feeling” to judge candidates and that “unfortunately, once decision-makers believe that an individual has future leader potential, even bad performance re- views or evaluations from subordinates and peers do not seem to be able to shake their belief” (Babiak et al., 2010, p. 190). The addition of data from a standardized assessment instrument, based on the observations of others who may work closely with such persons, could counterbalance inaccurate perceptions. Limitations and Future Research The results are based on a sample from MTurk and may not generalize to other means of collecting B-Scan data from organi- zations or corporate environments. We chose MTurk because it provided us with data from a wide array of individuals with different occupations and supervisors, and different work environ- ments and settings. Given such diversity of participants, it is unlikely that the results would be limited to a particular work environment. Nevertheless, MTurk samples have limitations that could potentially affect external validity, such as the facts that employees are not all from the same organization and that orga- nizational context and other variables cannot be controlled. We believe that future research using corporate samples is needed in order to establish generalizability as well as the predictive validity of B-Scan scores and organizational variables. In sum, given the destructive nature of individuals high on psychopathic features, the development of a sound and reliable business-friendly measure is badly needed. Our results suggest thatour instrument may represent such a measure and although more research is needed before it can be used in organizational settings, we think that this first validation study of the B-Scan is an important step forward in the study of corporate psychopathy. References Babiak, P., & Hare, R. D. (2006).Snakes in suits: When psychopaths go to work.New York, NY: Harper Collins. Babiak, P., & Hare, R. D. (2012).The B-Scan 360 Manual.Manuscript in preparation. Babiak, P., Neumann, C. S., & Hare, R. D. (2010). Corporate psychopathy: Talking the walk.Behavioral Sciences & the Law, 28,174 –193. Barger, T., Behrend, T. S., Sharek, D. J., & Sinar, E. F. (2011). I-O and the crowd: Frequently asked questions about using Mechanical Turk for research.The Industrial-Organizational Psychologist, 49,11–18. Behrend, T. S., Sharek, D. S., Meade, A. W., & Wiebe, E. N. (2011). The viability of crowdsourcing for survey research.Behavior Research Methods, 43,800 – 813. doi:10.3758/s13428-011-0081-0 Bentler, P. M. (1995).EQS structural equations program manual.Encino, CA: Multivariate Software. Bolton, L. R., Becker, L. K., & Barber, L. K. (2010). Big five trait predictors of differential counterproductive work behavior dimensions. Personality and Individual Differences, 49,537–541. doi:10.1016/ j.paid.2010.03.047 Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechan- ical Turk: A new source of cheap, yet high-quality, data?Perspectives on Psychological Science, 6,3–5. doi:10.1177/1745691610393980 Campbell, W. K., Hoffman, B. J., Campbell, S. M., & Marchisio, G. (2011). Narcissism in organizational contexts.Human Resource Man- agement Review, 21,268 –284. Cho, S. J., Li, F., & Bandalos, D. (2009). Accuracy of the parallel analysis procedure with polychoric correlations.Educational and Psychological Measurement, 69,748 –759. doi:10.1177/0013164409332229 Cooke, D. J., & Michie, C. (2001). Refining the construct of psychopathy: Towards a hierarchical model.Psychological Assessment, 13,171–188. doi:10.1037/1040-3590.13.2.171 Guay, J. P., Ruscio, J., Knight, R. A., & Hare, R. D. (2007). A taxometric analysis of the latent structure of psychopathy: Evidence for dimension- ality.Journal of Abnormal Psychology, 116,701–716. doi:10.1037/ 0021-843X.116.4.701 Hare, R. D. (2003).Manual for the Hare Revised Psychopathy Checklist (2nd ed.). Toronto, Ontario, Canada: Multi-Health Systems. Hare, R. D., & Neumann, C. S. (2008). Psychopathy as a clinical and empirical construct.Annual Review of Clinical Psychology, 4,217–246. doi:10.1146/annurev.clinpsy.3.022806.091452 Hare, R. D., Neumann, C. S., & Widiger, T. A. (in press). Psychopathy. In T. A. Widiger (Ed.),The Oxford handbook of personality disorders, New York, NY: Oxford University Press. Hart, S. D., Cox, D. N., & Hare, R. D. (1995).Manual for the Psychopathy Checklist: Screening Version (PCL: SV).Toronto, Ontario, Canada: Multi-Health Systems. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis.Psychometrika, 30,179 –185. doi:10.1007/BF02289447 Lynam, D. R., & Miller, J. D. (in press). Fearless dominance and psychop- athy: Response to Lilienfeld et al.Personality Disorders: Theory, Re- search, and Treatment. Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999). findings.Structural Equation Modeling, 11,320 –341. doi:10.1207/ s15328007sem1103_2 Mathisen, G. E., Einarsen, S., & Mykletun, R. (2011). The relationship between supervisor personality, supervisors’ perceived stress and work- 292 MATHIEU, HARE, JONES, BABIAK, AND NEUMANN place bullying.Journal of Business Ethics, 99,637– 651. doi:10.1007/ s10551-010-0674-z Muthen, L. K., & Muthen, B. O. (2010).Mplus: The comprehensive modeling program for applied researchers: User’s guide.Los Angeles, CA: Author. Neumann, C. S., & Hare, R. D. (2008). Psychopathic traits in a large community sample: Links to violence, alcohol use, and intelligence. Journal of Consulting and Clinical Psychology, 76,893– 899. doi: 10.1037/0022-006X.76.5.893 Neumann, C. S., Hare, R. D., & Johansson, P. (in press). The Psychopathy Checklist-Revised (PCL-R), low anxiety, and fearlessness.Personality Disorders: Theory, Research, and Treatment. Neumann, C. S., Hare, R. D., & Newman, J. P. (2007). The super-ordinate nature of the Psychopathy Checklist-Revised.Journal of Personality Disorders, 21,102–117. doi:10.1521/pedi.2007.21.2.102 Perri, F. S. (2011). White-collar criminals: The “kinder, gentler” offender? Journal of Investigative Psychology and Offender Profiling, 8,217–241. doi:10.1002/jip.140Rotundo, M., & Sackett, P. R. (2002). The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach.Journal of Applied Psychol- ogy, 87,66 – 80. doi:10.1037/0021-9010.87.1.66 Skarlicki, D., Folger, R., & Tesluk, P. (1999). Personality as a moderator in the relationship between fairness and retaliation.Academy of Man- agement Journal, 42,100 –108. doi:10.2307/256877 Stevens, G. W., Deuling, J. K., & Armenakis, A. A. (2012). Successful psychopaths: Are they unethical decision-makers and why?Journal of Business Ethics, 105,139 –149. doi:10.1007/s10551-011-0963-1 Zwick, W. R., &Velicer, W. F. (1986). Factor influencing five rules for determining the number of components to retain.Psychological Bulletin, 99,432– 442. doi:10.1037/0033-2909.99.3.432 Received October 1, 2011 Revision received May 5, 2012 Accepted May 9, 2012 293 B-SCAN
Validity Review the articles by Fergus (2013), Kosson, et al. (2013) and Mathieu, Hare, Jones, Babiak, & Neumann (2013). Analyze the information presented in these articles on factor analysis and des
Factor Structure of the Hare Psychopathy Checklist: Youth Version (PCL:YV) in Adolescent Females David S. Kosson Rosalind Franklin University of Medicine and Science Craig S. NeumannUniversity of North Texas Adelle E. ForthCarleton University Randall T. SalekinUniversity of Alabama Robert D. Hare University of British Columbia Maya K. Krischer and Kathrin Sevecke University of Cologne Despite substantial evidence for the fit of the 3- and 4-factor models of Psychopathy Checklist-based ratings of psychopathy in adult males and adolescents, evidence is less consistent in adolescent females. However, prior studies used samples much smaller than recommended for examining model fit. To address this issue, we conducted a confirmatory factor analysis of 646 adolescent females to test the fit of the 3- and 4-factor models. We also investigated the fit of these models in more homogeneous subsets of the full sample to examine whether fit was invariant across geographical region and setting. Analyses indicated adequate fit for both models in the full sample and was generally acceptable for both models in North American and European subsamples and for participants in less restrictive (probation/detention/ clinic) settings. However, in the incarcerated subsample, the 4-factor model achieved acceptable fit on only two of four indices. Although model fit was not invariant across continent or setting, invariance could be achieved in most cases by simply allowing factor loadings on a single Psychopathy Checklist: Youth Version (Forth, Kosson, & Hare, 2003) item to vary across groups. In summary, in contrast to prior studies with small samples, current findings show that both the 3- and 4-factor models fit adequately in a large sample of adolescent females, and the factor loadings are largely similar for North American and European samples and for long-term incarcerated and shorter-term incarcerated/probation/clinic samples. Keywords:adolescents, psychopathy, confirmatory factor analysis, antisocial behavior, sex differences Models of the factor structure of psychological tests play a critical role in understanding and validating the constructs they are designed to assess. Scores on subsets of items for a measure that cohere similarly in diverse and independent samples pro- vide evidence for the generalizability of the construct being measured. Evidence that a pattern of covariances is consistent with theoretical expectations makes an important contributionto construct validation (Strauss & Smith, 2009). Factor models that generalize across different kinds of samples provide a foundation for subsequent scientific studies that examine whether these dimensions are characterized by similar nomo- logical networks across samples. Such studies, in turn, can be used to test hypotheses about the mechanisms underlying the components of a syndrome. This article was published Online First June 25, 2012. David S. Kosson, Department of Psychology, Rosalind Franklin Uni- versity of Medicine and Science; Craig S. Neumann, Department of Psy- chology, University of North Texas; Adelle E. Forth, Department of Psychology, Carleton University; Randall T. Salekin, Department of Psy- chology, University of Alabama; Robert D. Hare, Department of Psychol- ogy, University of British Columbia, Vancouver, British Columbia, Can- ada; Maya K. Krischer and Kathrin Sevecke, Department of Child and Adolescent Psychiatry, University of Cologne, Cologne, Germany. Adelle E. Forth, David S. Kosson, and Robert D. Hare are coauthors of the Psychopathy Checklist: Youth Version (PCL:YV), published by Multi- Health Systems, 3770 Victoria Park Avenue, Toronto, Ontario, Canada M2H 3M6 and receive royalties from its sale. We thank Henrik Andershed, Jacqueline Das, Heather Gretton, Derek Indoe, Malin Hemphälä, Sheilagh Hodgins, Kathleen Lewis, RoyO’Shaughnessy, R. Rowe, Corine de Ruiter, Fred Schmidt, Anders Teng- ström, Anita Thapar, and Todd Willoughby for providing much of the data analyzed in these studies. Several of the samples examined here are described in greater detail in the PCL:YV manual (Forth, Kosson, & Hare, 2003). The collection of some of the data examined in this study was supported by National Institute of Mental Health Grant MH49111 to David S. Kosson; by funding from the William H. Donner Foundation to Craig S. Neumann; by grants from the Centers for Disease Control, the Department of Justice, and the Department of Youth Services to Randall T. Salekin; and by a grant from the Alexander Humboldt Foundation to Kathrin Sevecke. Correspondence concerning this article should be addressed to David S. Kosson, Department of Psychology, Rosalind Franklin University of Med- icine and Science, 3333 Green Bay Road, North Chicago, IL 60064. E-mail: [email protected] Psychological Assessment© 2012 American Psychological Association 2013, Vol. 25, No. 1, 71– 831040-3590/13/$12.00 DOI: 10.1037/a0028986 71 Psychopathy is a severe syndrome of personality pathology that is widely associated with callous and manipulative interpersonal behavior as well as impulsive and irresponsible antisocial behav- ior. The standard clinical measures of the psychopathy construct are the Hare Psychopathy Checklist (PCL) scales that ask raters to make inferences about underlying dispositions by integrating in- formation from interviews, behavioral observations, and file or other collateral material (Hare & Neumann, 2009). These scales include the Psychopathy Checklist—Revised (PCL-R; Hare, 2003), the Psychopathy Checklist: Screening Version (PCL:SV; Hart, Cox, & Hare, 1995), and the Psychopathy Checklist: Youth Version (PCL:YV; Forth, Kosson, & Hare, 2003). Several factor models of the PCL scales have been proposed. The four-factor model suggests that individual differences in the disposi- tions that comprise psychopathy are underlain by differences in one or more of four correlated dimensions that reflect specific interpersonal, affective, lifestyle, and antisocial features. Evidence corroborating this model comes from confirmatory factor analyses (CFAs) of PCL scores in a variety of forensic, clinical, and community populations (e.g., Babiak, Neumann, & Hare, 2010; Hare & Neumann, 2008; Neumann & Hare, 2008). This pattern of strong correlations has been explained by a second-order general factor (Neumann, Hare, & New- man, 2007; Neumann, Kosson, Forth, & Hare, 2006) said to reflect the superordinate syndrome of psychopathic personality. This inter- pretation is consistent with behavior genetic research that has shown that four psychopathy factors similar to the PCL factors can all be accounted for by a common genetic trait (Larsson, Andershed, & Lichtenstein, 2006). The three-factor model (Cooke & Michie, 2001) is identical with respect to the first three dimensions of the four-factor model but omits the antisocial features dimension (and the five items that load on that component). 1Because tests of both the three- and four-factor PCL models of psychopathy often yield acceptable fit in adult and adolescent males (Cauffman, Kimonis, Dmitrevia, & Monahan, 2009; Neumann, Hare, & Newman, 2007; Neumann et al., 2006; Salekin, Brannen, Zalot, Leistico, & Neumann, 2006), these models are currently the dominant models for the internal structure of psychopathy based on clinical measures. 2 In contrast, the fit of these factor models in female samples is more controversial. Among adult women, Warren et al. (2003) reported good fit for the two- and three-factor models. Similarly, Neuman, Hare, and Newman (2007) reported good fit for the four-factor model in both male and female adult inmates, whereas Salekin, Rogers, and Sewell (1997) suggested the factor structure is somewhat different in women than men. Bolt, Hare, Vitale, and Newman (2004) conducted item analyses in large samples of adult male and female offenders and reported that scalar equivalence may hold, at least approximately, for male and female offenders, in spite of some evidence for differential test functioning and for differential item functioning on some lifestyle and antisocial di- mension items. Vitale and Newman (2001b) noted that most prior factor ana- lytic studies have involved small samples that may have provided inadequate power for examining factor structure. They emphasized the need for researchers to conduct studies with larger samples of females. Examining adolescents, Forth et al. (2003) reported ac- ceptable fit for the three-factor model in a sample of female adolescents, whereas the four-factor model achieved acceptable fit only on the absolute fit indices examined. However, their sample(based on six different subsamples) included only 147 girls. Con- sequently, analyses were likely underpowered for evaluating both models. 3In addition, Forth et al. did not subdivide the sample to examine fit separately for incarcerated versus probation samples of girls or for samples from different parts of the world. Subsequent studies have also yielded conflicting findings. Jones, Cauffman, Miller, and Mulvey (2006) reported reasonable fit for the three- and four-factor models in girls but only after making minor changes to the factor structures that have not been evaluated in other studies. In contrast, Sevecke, Pukrop, Kosson, and Krischer (2009) reported that neither the three-factor nor the four-factor model yielded generally acceptable fit among incarcerated German adolescent fe- males. In an item response theory analysis, Schrum and Salekin (2006) reported that some of the same items that are most discrimi- nating in male samples were most discriminating in a sample of female youths. However, they noted that some items were more or less discriminating in girls than in boys. Despite a few recent studies the relative dearth of research in this area is of concern because of the potential differences in measurement structure and in the correlates of constructs that can occur across sex. Although prior studies provide some information about the factor structure of PCL:YV psychopathy in specific settings and locations, the small size of these samples is 1Some researchers have argued that the antisocial dimension of the four-factor model should be excluded based on conceptual grounds and have specifically argued that antisociality is not central enough to psy- chopathy to justify its inclusion. This argument is beyond the scope of the current study, and we encourage interested readers to see Skeem and Cooke (2010) and Hare and Neumann (2008, 2010) for recent discussions of relevant issues. We note here only that some of these discussions do not make clear that the five items comprising the dimension commonly re- ferred to as the antisocial dimension are not scored on the basis of participation in antisocial behavior per se. Rather, these items are designed to assess early, persistent, and versatile expressions of antisocial behavior that distinguish some individuals who commit antisocial behavior from other individuals who commit antisocial behavior. 2The internal structure of self-reports and observer ratings of psycho- pathic features depends upon the instruments used. For example, analyses of mother and teacher ratings of psychopathic traits in pre-adolescents, as measured by the Antisocial Process Screening Device (Frick & Hare, 2001), suggest a slightly different three-factor structure than has been identified using PCL-based measures. Factor structures resulting from analyses of self-report scores are variable across instruments, with studies reporting evidence for three-factor and four-factor structures similar to those seen in the PCL measures for scores on the Self-Report Psychopathy Scale and the Youth Psychopathy Inventory (Larsson et al., 2006; Mahmut, Menictas, Stevenson, & Homewood, 2011; Williams et al., 2007) but reporting very different factor structures for some other self-report mea- sures (e.g., Brinkley, Diamond, Magaletta, & Heigel, 2008). In some cases, different studies using the same instrument suggest different internal struc- tures. For example, different factor analytic studies of scores on the Psychopathic Personality Inventory suggest disparate solutions involving two versus three dimensions (Benning, Patrick, Hicks, Blonigen, & Krueger, 2003; Neumann, Malterer, & Neumann, 2008; Uzieblo, Ver- schuere, Van den Bussche, & Crombez, 2010). 3Forth et al. (2003) also did not use an optimal model estimation strategy for conducting their analyses. EQS (Version 5.6) and LISREL (Version 8.30) were not designed for use with ordinal item indicators. Mplus has advantages in analyses of ordinal indicators. When one of us reconducted the CFAs on the 147 females examined by Forth et al. using Mplus, results indicated acceptable fit for both the three- and four-factor models. 72 KOSSON ET AL. likely to work against obtaining good fit for both the three- and four-factor models. The possibility of a different factor structure for girls than for boys is especially interesting in light of evidence that some of the correlates of scores (on clinical measures of psychopathy) in males do not consistently generalize to female samples. For example, associations between psychopathic traits and response modulation deficits (Vitale & Newman, 2001a; cf. Vitale, Brinkley, Hiatt, & Newman, 2007) and affective modulation of startle reflexes (Sutton, Vitale, & Newman, 2002) appear less consistent in adult female than male samples. Although some studies have yielded relatively similar patterns of correlations for psychopathy ratings in females and in males (Ken- nealy, Hicks, & Patrick, 2007; Stockdale, Olver, & Wong, 2010) or patterns of correlations in females similar to those previously reported for males (Bauer, Whitman, & Kosson, 2011; Penney & Moretti, 2007), other studies have cast doubt on the construct validity of PCL:YV scores among female adolescents (Odgers, Reppucci, & Moretti, 2005; Vincent, Odgers, McCormick, & Corrado, 2008). It is important to keep in mind that all of the findings on the factor structure of psychopathy reviewed above are based on PCL measures of psychopathy. Factor analytic studies can only provide evidence on the structure of a construct as assessed using a specific measure. Even so, evidence that the factor structure differs for girls and boys when psychopathy is assessed with the PCL:YV would suggest the possi- bility that some of the differences in behavioral and physiological correlates of psychopathy ratings may reflect differences in the nature of the psychopathy construct in girls. In brief, evidence that the symptoms of psychopathy (as assessed by clinical measures) cohere differently for girls than boys would suggest that different features may be critical to the expression of psychopathy in girls and would increase the plausibility of the perspective that different mechanisms may account for the appearance of these symptom dimensions. In contrast, evidence for a similar underlying factor structure would suggest that a psychopathy measure is performing similarly in boys and girls. To the extent that the symptoms examined cohere in similar ways across sex, it becomes more likely that a pattern of similar correlations between psychopathy ratings and external criteria reflects similar underlying mechanisms. Although evidence for similarity in internal structure does not invalidate the differences reported in correlational studies, it would be consistent with the possibility that similar mechanisms may account for those relationships between psychopathy and external criteria that are similar in males and females. 4 As noted above, one of the chief limitations of prior factor analytic investigations in females has been the use of small sam- ples. Small samples can lead to poor fit, even though the fit might be quite good when examined within large enough samples. An- other limitation of prior studies is that none of the above men- tioned studies compared factor models across youth in different countries or continents, and no prior studies have compared the fit of the different models in different kinds of settings. The Current Study The primary goal of the present study was to test the factor structure underlying PCL:YV-based psychopathy in a large sample of adolescent females. We assembled data from a large number of prior published studies that used the PCL:YV with adolescent girls. This provided us with a relatively large data set of 776adolescent females (646 with no missing values). In examining this large sample, we hoped to provide greater clarity on the factor structure of psychopathy in adolescent females. A secondary goal of this study was to evaluate the fit of the best fitting models in more homogeneous subgroups of participants and to assess whether the models demonstrated invariance for sub- samples of participants assessed in different continents and partic- ipants assessed in different settings. We conducted both separate confirmatory factor analyses (CFAs) for North American and European subsamples as well as multiple-group CFAs to evaluate whether the models provided good explanations for the pattern of item-to-factor relationships in samples in different continents; we conducted a parallel set of separate CFAs and multiple-group CFAs to address the same issues for incarcerated adolescent fe- males versus those in less restrictive settings, including probation and short-term detention/evaluation centers. We realized that there are important cultural differences be- tween the different countries within North America and within Europe. However, an analysis of factor loadings in North Ameri- can versus European samples provides an initial examination of whether the factor models are characterized by structural invari- ance across geographic region. Similarly, because youths who commit more frequent and more serious crimes and perform poorly under conditional release are likely to be sent to more restrictive settings, it is likely such settings will include a higher proportion of youths with many psychopathic features. Conse- quently, an analysis of the fit of the models for incarcerated adolescents versus adolescents in less restrictive (short-term de- tention, community probation, and clinic) settings provides a pre- liminary assessment of invariance across setting. Method Participants Data on 776 adolescent females were made available to the authors. These participants had participated in 14 different studies with independent samples; findings for a combination of the data from five of these samples were previously reported in the PCL:YV technical manual (Forth et al., 2003). Basic de- scriptive information about the participants from each sample is listed in Table 1. Measure Psychopathy Checklist: Youth Version (PCL:YV).The PCL:YV is a multi-item rating scale that assesses interpersonal and affective characteristics as well as overt behaviors associated with psychopathy. The measure is designed to be completed by trained observers who rate the presence of each trait disposition on the basis of a semistructured interview and a review of case history information or other collateral source(s). Ratings based on both 4Of course, even if a psychopathy measure functions similarly in boys and girls, it remains possible that the phenotypic similarities reflect differ- ent underlying mechanisms. Conversely, it could be argued that the same mechanisms operate in females and males but that these mechanisms are associated with different constellations of features in girls and boys; however, this latter perspective does not appear very parsimonious. 73 FACTOR STRUCTURE IN GIRLS interview and collateral data are described as obtained using the standard assessment method. The PCL:YV manual also permits the use of files only to complete the instrument but suggests caution in interpreting file-only scores, as the file-only method commonly provides substantially less information for scoring sev- eral of the interpersonal and affective items. Even so, prior factor analytic studies indicate acceptable fit for both PCL-R scores and PCL:YV scores completed solely on the basis of institutional files (Bolt et al., 2004; Forth et al., 2003). Scores of 0 (consistently absent),1(inconsistent), or 2 (consistently present) for each item of the PCL:YV reflect inferences about the consistency of the specific tendency or disposition across different situations and sources of information. Scores on the PCL:YV have demonstrated internal consistency, with alpha coefficients ranging from .79 to .94 for total scores and mean interitem correlations ranging from .44 to .63 (Forth et al., 2003; Vitacco, Neumann, & Caldwell, 2010). Alphas for factor scores have ranged from .68 to .77 in the validation sample (Forth et al., 2003) and from .50 to .82 in smaller samples (Andershed, Hodgins, & Tengström, 2007; Vitacco et al., 2010; Vitacco, Neu- mann, Caldwell, Leistico, & Van Rybroek, 2006) with one excep- tion (Skeem & Cauffman, 2003). Lower alphas for factor scores are expected in light of the number of items that contribute to each factor. Researchers have obtained good to excellent interrater reliability for total scores (ICCs range from .82 to .98; see Ander- shed et al., 2007; Cauffman et al., 2009; Das, de Ruiter, Dorelei- jers, & Hillege, 2009; Forth et al., 2003). The interrater reliability for factor scores is more variable, ranging from .43 to .86 (Forth et al., 2003; Skeem & Cauffman, 2003). PCL:YV scores cor- relate moderately with indices of externalizing psychopathol- ogy, instrumental violence, criminal activity, and antisocial behavior and predict recidivism in male adolescents (Flight & Forth, 2007; Kosson, Cyterski, Steuerwald, Neumann, &Walker-Matthews, 2002; Kubak & Salekin, 2009; Murrie, Cor- nell, Kaplan, McConville, & Levy-Elkon, 2004; Salekin, 2008; Salekin, Neumann, Leistico, DiCicco, & Duros, 2004; Schmidt, McKinnon, Chattha, & Brownlee, 2006; Vitacco et al., 2006, 2010). Data Analysis Confirmatory factor analyses test the fit of specific models for the latent structure underlying variation on observed indicators. Such analyses require that investigators first specify the number of latent factors, the relationships between indicators and factors, and the factor variances and covariances within a model and then statistically test the adequacy of their model in terms of standard model fit criteria. To test alternative latent structures of the PCL: YV, we carried out CFAs with Mplus (Version 5; Muthe´n & Muthe´ n, 1998 –2007), using the robust (mean- and variance- adjusted) weighted least squares (WLSMV) estimator recom- mended for use with ordinal data such as PCL:YV items (Flora & Curran, 2004; Neumann, Kosson, & Salekin, 2007). Because each fit index has limitations, and there are no agreed upon methods for definitively determining quality of fit (Kline, 1998), adequacy of fit for each model was estimated using several measures. Because the chi-square is usually significant with large samples, investigators typically rely on other fit indices to assess the adequacy of a model. We calculated two widely validated relative fit indices: the comparative fit index (CFI) and the Tucker–Lewis index (TLI). The TLI and the CFI are incremental fit measures comparing the estimated model with a null or inde- pendence model; the TLI tends to be more adversely affected by the estimation of additional parameters that do not improve model fit and is less sensitive to sample size than many other relative fit indices (Marsh, Balla, & McDonald, 1988). For these indices, Table 1 Participants SampleNSetting Country Method Samples in the PCL:YV manual 1. Lewis and O’Shaughnessy (1998) 37 arrested/inpatient (I) Canada (NA) S 2. Gretton and Hare (2002) 43 arrested/inpatient (I) Canada (NA) F 3. Rowe (2002) 54 high risk probation (D) Canada (NA) F 4. Bauer, Whitman, and Kosson (in press) 80 incarcerated (I) United States (NA) S 5. Indoe (2002) 28 incarcerated (I) United Kingdom (E) S Additional samples 6. Kosson et al. (2012) 21 detention center (D) United States (NA) S 7. Salekin, Leistico, Trobst, Schrum, & Lochman (2005) 45 detention center (D) United States (NA) S 8. Salekin, Neumann, Leistico, & Zalot (2004) 38 detention center (D) United States (NA) S 9. Salekin, Neumann, Leistico, DiCicco, & Duros (2004) 37 court evaluation (D) United States (NA) S 10. Schmidt, McKinnon, Chattha, and Brownlee (2006) 49 court evaluation (D) Canada (NA) F 11. Krischer and Sevecke (2008) 171 incarcerated (I) Germany (E) 12. Fowler et al. (2009) 11 psychiatry/pediatric clinic (D) United Kingdom (E) 13. Andershed, Hodgins, & Tengström (2007) 99 substance misuse clinic (D) Sweden (E) S 14. Das, de Ruiter, & Doreleijers (2008) 67 secure treatment facility (I) Netherlands (E) S Incarcerated 369 Samples 1, 2, 4, 5, 11, 14 Probation 277 Samples 3, 6, 7, 8, 9, 10, 12, 13 North American 285 Samples 1, 2, 3, 4, 6, 7, 8, 9, 10 European 361 Samples 5, 11, 12, 13, 14 Note.PCL:YV Psychopathy Checklist: Youth Version (Forth, Kosson, & Hare, 2003); I incarcerated; D detention center/probation/clinic; NA North America; E Europe; S standard method (interview collateral); F file only method. All of the samples listed were collected independently. 74 KOSSON ET AL. larger values indicate better fit of the hypothesized model (a conventional standard is 0.9 or above for acceptable fit, 0.95 or above for excellent fit; Hu & Bentler, 1995; Kline, 1998). We also examined two absolute fit indices: the root-mean- square error of approximation (RMSEA) and the standardized root-mean-square residual (SRMR). Absolute indices gauge how well the model-generated covariance matrix reproduces the sample covariance matrix. Smaller absolute fit indices, and thus smaller residual error values, indicate better fit. The RMSEA is an index that also rewards model parsimony (T. A. Brown, 2006), whereas the SRMR appears to be an especially sensitive indicator of poor fit. Moreover, because the SRMR and RMSEA provide relatively different approaches to estimating absolute fit, they provide some- what more independent assessments of fit than some indices (Hu & Bentler, 1995). For these indices, Hu and Bentler (1995) suggested that scores below .05 indicate good fit, whereas scores between .05 and .08 indicate acceptable fit, and values above 1.0 indicate poor fit (Browne & Cudeck, 1993; Hoyle, 1995; Marsh, Hau, & Wen, 2004; MacCallum, Browne, & Sugawara, 1996). An important caveat to the use of multiple fit indices is that, as model complexity increases, so does the size of the sample needed to test the model and the difficulty of achieving conventional levels of model fit (Marsh et al., 2004). Put another way, if a more complex model displays approximately similar fit to a less- complex model, then the former is said to have survived a riskier test (Vitacco, Neumann, & Jackson, 2005; Vitacco, Rogers, Neu- mann, Harrison, & Vincent, 2005). However, given the adequate size of our full sample for all models examined, models were considered to fit adequately only if they indicated at least fair to acceptable fit on all four primary fit indices examined. Because the subsamples were only about half as large as the full sample from which they were drawn, we required at least fair to acceptable fit on three or more fit indices to indicate acceptable fit for subsample analyses. As in most recent tests of the three-factor model, we did not test the original three-factor model that uses testlets, because prior studies conducted in several samples have shown that the model with testlets results in untenable solutions with impossible values (i.e., negative variance; Kosson et al., 2002; Salekin et al., 2006). For this reason, we examined a three-factor model without testlets (i.e., allowing the latent variables to load directly onto the PCL:YV items). It is this three-factor model that has achieved good fit in prior CFAs of adolescents (e.g., Neumann et al., 2006; Sevecke et al., 2009). Because the three-factor model is based on a different set of items and therefore a different covariance matrix than is the four-factor model, direct statistical comparison of these models is not possible (T. A. Brown, 2006; Kline, 1998). From a mathematical modeling perspective, the three-factor model is less parsimonious than the four-factor model because it requires estimating 29 parameters to model 91 data points (df 91–29 62), whereas the four-factor model uses only 42 free parameters to explain 171 data points (df 171– 42 129). However, this study was not designed to examine the relative merits of these two factor models but to examine their fit in a large sample of adolescent females and to examine whether there was evidence for invariance of these two models across different set- tings and for samples from Europe versus North America. More- over, because the three-factor model is contained within the four- factor model, the two models are quite similar in most respects.The chief difference between the models concerns the nature of the antisocial dimension of the psychopathy construct (and the five items that load on this latent factor; this issue is addressed else- where, e.g., Cooke, Michie, Hart, & Clark, 2004; Hare & Neu- mann, 2008, 2010). Results Preliminary Analyses Principal analyses were based on participants with complete data. To ensure that the same participants were included in three- factor and four-factor analyses, only participants with complete data for the 18 items needed for the four-factor analyses were included in these analyses. Complete data for the 18 items were available for 646 adolescent females (369 incarcerated females and 277 females drawn from less restrictive settings, i.e., probation, detention centers, and clinics). Supplementary analyses were also conducted including cases containing missing values. These analyses included 776 adolescent females (423 incarcerated and 353 probation/detention/clinic ado- lescents). To test the assumption of full information CFAs that missing data were missing at random (i.e., that there are no systematic reasons why some items were not scored for some participants), we first conducted analyses to ascertain whether missing data covaried with differences on demographic variables. Chi-square analyses revealed that missing values were more prev- alent in North American than in European data sets, 2(1) 94.35, p .001, and were more prevalent among incarcerated youth than among youth in less restrictive settings, 2(1) 10.60,p .001. Missingness was also much more likely for file-only than for standard (interview plus file) data, 2(1) 37.56,p .001. Although ethnicity was only available for 342 cases, missing values were also more prevalent among Caucasian and Native North American adolescents than among African or Latina ado- lescents, 2(3) 87.22,p .001. Based on these analyses, we focus on analyses for samples including no missing values. 5 Confirmatory Factor Analyses of the PCL:YV in the Full Sample of Female Adolescents We first examined the fit of the various factor models in the full sample. The one-factor and two-factor models were examined using the same 18 items as in the four-factor model. The CFA for 5Results for full-sample analyses including missing values indicated gen- erally acceptable fit for the three-factor and four-factor models; however, the CFI slipped to .90 and .89 for the three- and four-factor models. Both models were also generally acceptable for the North American and European sub- samples, although the CFI slipped to .89 for the four-factor model in both subsamples. For analyses limited to incarcerated girls, the CFI was low for the three-factor model (CFI 0.89), and, as in principal analyses, both relative fit indices were unacceptable for the four-factor model (CFI 0.82, TLI 0.87). For analyses of girls on probation, under detention, or at clinics, both models yielded low CFIs (0.89 and 0.88 for the three- and four-factor models, respectively) and borderline-unacceptable RMSEAs (0.099 and 0.097). Addi- tional multigroup analyses demonstrated that fit was also poorer when both loadings and thresholds were constrained to be equal in the two groups. Results of these analyses are available upon request. 75 FACTOR STRUCTURE IN GIRLS the one-factor model indicated unacceptable fit for two of the four primary fit indices examined, 2(78) 699.37,p .001, CFI .80, TLI .90, RMSEA .111, SRMR .087. The CFA for the two-factor model also indicated unacceptable fit for the CFI and only fair to acceptable fit for the RMSEA, with adequate fit on the other two indices examined, 2(79) 498.36,p .001, CFI 0.86, TLI 0.93, RMSEA 0.091, SRMR 0.074. In contrast, both the three-factor and four-factor model yielded adequate fit for both relative fit indices and for both absolute fit indices. Figure 1 displays the standardized parameters for these two models. The CFA results for the three- and four-factor models for the full sample and each of the subsamples are shown in Table 2. The CFAs also showed that the factors were correlated as expected. The latent correlations between the interpersonal and the affective, lifestyle, and antisocial dimensions were .68, .63, and .56; the correlations between the affective and lifestyle and anti- social dimensions were .73 and .66; and the correlation between the lifestyle and antisocial dimensions was .83. Confirmatory Factor Analyses of the PCL:YV in Subsamples of Female Adolescents Both models also yielded consistently adequate fit in the North American subsample. In fact, model fit was in the good to excel- lent range for both of the relative fit indices and in the reasonable range for both absolute fit indices for both models, despite a substantial drop in sample size from 646 to 285 cases (which makes the size of the subsamples suboptimal for assessing the fit of the four-factor model). In contrast, only the three-factor model yielded consistently acceptable fit in the European subsample. The fit of the four-factor model was in the fair to acceptable range forthree of four indices examined but slipped barely below acceptable levels for the CFI. Both models also yielded generally acceptable fit for the sub- sample of youth in less restrictive (i.e., probation, detention, and clinic) settings. In brief, the fit was acceptable to good for both models for both measures of relative fit and for the SRMR. However, the RMSEA yielded only fair fit for both models in this subsample. In contrast, only the three-factor model yielded gener- ally acceptable fit for the subsample of incarcerated girls. Only the CFI slipped below acceptable levels for this model. For the four- factor model, both relative fit indices were unacceptably low, and both indices of absolute fit indicated only fair fit. In summary, both the three- and four-factor models had good fit to the data (i.e., were able to reproduce the observed item covari- ance structure with adequate precision) in the full sample, and both models provided reasonable fit among North American girls. Even among European girls, both models generally provided at least fair to reasonable fit. Similarly, with the exception of the RMSEA, both models provided adequate levels of fit among girls on pro- bation and in detention. The only subsample in which the pattern of findings suggested fit below conventional levels of acceptability was the incarcerated subsample, for which the four-factor model yielded unacceptable fit on two indices and only fair to acceptable fit on the other two indices examined. Tests of Model Invariance Between North American and European Samples To test whether the three-factor model fit equally well in North American and European adolescent females, we con- ducted two multiple-group CFAs (MGCFAs) and compared the Figure 1.Factor loadings, factor covariances, and fit indices for the four-factor model, full sample analysis in adolescent females (N 646). behav. behavior; CFI comparative fit index; TLI Tucker–Lewis index; RMSEA root-mean-square error of approximation; SRMR standardized root-mean-square residual. 76 KOSSON ET AL. fit for the two models using a chi-square difference test. First, we allowed Mplus to freely estimate all model parameters separately by sample (i.e., factor loadings and item thresholds), fixing scale factor values at 1.0, factor means at 0, and factor variances at 1, as is the default in Mplus in multigroup analyses when using the default delta parameterization. Under these conditions, the model yielded evidence of acceptable fit on all three indices other than the chi-square, 2(80,N 646) 216.89,p .001, CFI 0.938, TLI 0.964, RMSEA 0.073. Next, we repeated the analysis constraining the loadings (but not the thresholds) to be equal in the two groups. This model yielded similar evidence of good fit across the two subsamples on relative fit indices and acceptable fit on an absolute fit index, 2(78,N 646) 289.82,p .001, CFI 0.941, TLI 0.963, RMSEA 0.074. However, the model yielded poorer fit than the less constrained model, as evidenced by a significant chi-square difference test, 2(9) 26.65,p .002. To examine whether the lack of invariance could be attributable to a single item loading differently, we reconducted the MGCFA allowing different loadings on PCL:YV Item 9, Parasitic Orientation, the item for which the multigroup CFA estimating the loadings separately had suggested the most disparity in item loadings (see Table 3). This analysis yielded a nonsignificant chi-square difference test, 2(8) 13.16,p .11. In sum, the generally reasonable fit for the three-factor model with constraints on the loadings in the two samples provides evidence of structural invariance. Moreover, it was possible to obtain evidence of structural invariance as indicated by good fit for a model requiring equal loadings on 12 of the 13 item indicators across the North American and European subsamples. We also examined whether the four-factor model fit equally well in the North American and European samples. Again, the unconstrained model yielded evidence of acceptable fit on all indices, 2(132,N 646) 411.24,p .001, CFI 0.908, TLI 0.950, RMSEA 0.081. Once again, the model con- strained to have equal loadings (but not thresholds) also yielded acceptable fit on both relative and absolute fit indices, 2(117) 362.15,p .001, CFI 0.919, TLI 0.951,RMSEA 0.081. However, as for the three-factor model, the chi-square difference test indicated poorer fit for the con- strained model, 2(13) 39.16,p .001. As above, we examined whether we could achieve structural invariance on all but one item by allowing the loadings to vary for one indicator. Once again, the MGCFA suggested the most disparate loadings were for PCL:YV Item 9. In this case, the MGCFA allowing separate thresholds in the two samples but equal loadings (except on Item 9) yielded a chi-square difference test that was still significant, 2(12) 23.19,p .03. Similarly, allowing different loadings on two items (Items 9 and 20) also did not eliminate the lack of invariance, 2(11) 21.59,p .03. In short, allowing for different loadings on one or two of 18 items did not result in a nonsignificant chi-square for the four-factor model. In order to examine whether estimated factor means differed in the European versus North American subsamples, we also conducted multigroup analyses using mean structures by allow- ing Mplus to hold the item indicator thresholds equal across groups and, by setting mean levels of each factor to 0 for the North American subsample, estimating mean levels of the latent factors separately in the European subsample. Similar to the analyses summarized above, these analyses indicated a lack of invariance across the samples, reflecting in part the fact that the items are discriminating at different levels of the underlying factors in each set of subsamples examined (e.g., European vs. North American). As expected, these analyses also indicated higher latent mean levels of several PCL factors in the North American sample (means by default set to zero) relative to the European subsample: European subsampleMs 0.36 and 0.43 for the affective and antisocial factors, respectively (zs 4.01, 4.98,ps .001), with a similar difference approach- ing significance for the lifestyle factor, European subsample latentM 0.17 (z 1.79,p .07). Interestingly, the European latent mean for the interpersonal factor was nonsig- nificantly higher than that for the North American subsample (M 0.13,z 1.26,ns). Table 2 Confirmatory Factor Analysis Model Fit Results Model/fit indexFull sample (n 646)Restrictive setting (n 369)Less restrictive setting (n 277)North American adolescents (n 285)European adolescents (n 361) Three-factor 2(df)219.87 (45) 133.573 (42) 117.612 (37) 86.264 (40) 128.401 (40) CFI0.9220.8970.922 0.956 0.925 TLI0.955 0.921 0.956 0.978 0.947 RMSEA0.078 0.077 0.089 0.064 0.078 SRMR0.060 0.072 0.073 0.058 0.069 Four-factor 2(df)367.64 (79) 270.719 (68) 196.994 (61) 146.633 (70) 254.718 (63) CFI0.9060.8250.909 0.9420.890 TLI0.9520.8740.949 0.974 0.930 RMSEA0.075 0.090 0.090 0.062 0.092 SRMR0.063 0.087 0.080 0.063 0.083 Note.CFI comparative fit index ( .90); TLI Tucker–Lewis Index ( .90); RMSEA root-mean-square error of approximation ( .10); SRMR standardized root-mean-square residual ( .10). Values considered at least fair to acceptable are shown in boldface type; cutoffs for values indicating at least fair to acceptable fit are listed in parentheses in this note. In this table, restrictive setting refers to youth facilities providing long-term incarceration, whereas less restrictive settings include probation, short-term detention, and clinic settings. 77 FACTOR STRUCTURE IN GIRLS Tests of Model Invariance Between Incarcerated and Detention/Probation/Clinic Samples A similar set of analyses was conducted to assess measurement invariance as a function of setting, although, as noted above, the sample size for these comparisons was suboptimal for assessing the four-factor model. Once again, the MGCFA for the three-factor model allowing separate estimation of parameters across the two settings demonstrated acceptable fit on both relative and absolute fit indices, 2(79) 251.20,p .001, CFI 0.912, TLI 0.943, RMSEA 0.082. Once again, the analysis that required equal loadings also provided a generally reasonable fit to the data, 2(76) 226.60,p .001, CFI 0.923, TLI 0.949, RMSEA 0.078, although its slightly poorer fit was confirmed by a significant chi-square difference test, 2(9) 19.03,p .02. Allowing the two subsamples to differ in the loading for one item, Item 5, was sufficient to yield a nonsignificant chi-square differ- ence test, 2(8) 10.21,p .25. In summary, these analysesindicated not only configural invariance but a moderate degree of metric invariance for the three-factor model across setting. Results were similar for the four-factor model. The uncon- strained model suggested generally acceptable fit across the two settings, 2(129) 469.53,p .001, CFI 0.874, TLI 0.922, RMSEA 0.090. However, it is noteworthy that this was the only unconstrained model in which an absolute or relative fit index (in this case, the CFI) fell below conventional levels of acceptability. In this case, the model requiring equal loadings (but not thresh- olds) yielded generally fair to acceptable fit, 2(121) 449.46, p .001, CFI 0.878, TLI 0.920, RMSEA 0.092, again with the exception of the CFI. In addition, the chi-square differ- ence test demonstrated that the fit was poorer when the loadings were required to be equal across the two settings, 2(13) 53.71, p .001. Table 3 shows that the indicator loadings for the two subsamples had appeared relatively discrepant across items on several factors when the loadings and thresholds were estimated Table 3 Factor Loadings and Thresholds for the Four-Factor Model in Multigroup Analyses in Which Item Loadings Are Estimated Separately in Different Subsamples Factor ItemRestrictive setting (n 369) Less restrictive setting (n 277) Loadings b1 b2 Loadings b1 b2 Interpersonal Item 1 .59 0.49 0.69 .67 0.39 0.08 Item 2 .68 0.31 0.86 .70 0.30 1.26 Item 4 .71 0.72 0.78 .58 0.41 1.14 Item 5 .61 0.98 0.38 .86 0.14 1.08 Affective Item 6 .79 0.85 0.23 .81 0.58 0.59 Item 7 .60 0.08 0.90 .58 0.20 1.46 Item 8 .91 0.90 0.33 .92 0.14 1.06 Item 16 .52 0.96 0.42 .73 0.33 0.68 Lifestyle Item 3 .57 1.51 0.02 .62 0.69 0.80 Item 9 .55 0.43 1.00 .56 0.20 1.49 Item 13 .37 0.56 0.53 .66 0.39 0.47 Item 14 .59 1.38 0.03 .71 1.43 0.42 Item 15 .64 1.45 0.05 .83 0.69 0.53 Antisocial Item 10 .78 1.11 0.17 .58 0.93 0.45 Item 12 .33 0.11 0.55 .61 0.53 1.28 Item 18 .68 1.11 0.06 .83 0.59 0.82 Item 19 .46 0.51 0.45 .63 0.30 1.11 Item 20 .79 0.41 0.40 .79 0.04 0.91 Interpersonal Item 1 .76 0.00 1.02 .65 0.20 0.87 Item 2 .77 0.05 0.85 .62 0.05 1.15 Item 4 .72 0.61 0.70 .50 0.56 1.14 Item 5 .85 0.30 0.81 .82 0.82 0.52 Affective Item 6 .78 0.91 0.24 .76 0.60 0.50 Item 7 .63 0.14 0.92 .53 0.18 1.23 Item 8 .91 0.67 0.53 .99 0.44 0.66 Item 16 .58 0.87 0.31 .67 0.51 0.71 Lifestyle Item 3 .75 0.82 0.35 .68 1.33 0.27 Item 9 .74 0.10 1.14 .48 0.20 1.19 Item 13 .43 0.75 0.36 .46 0.30 0.63 Item 14 .62 1.56 0.09 .63 1.30 0.21 Item 15 .67 1.27 0.00 .83 0.91 0.34 Antisocial Item 10 .57 1.11 0.05 .76 0.97 0.21 Item 12 .55 0.26 1.00 .57 0.07 0.68 Item 18 .68 1.65 0.18 .84 0.50 0.37 Item 19 .60 0.57 0.23 .57 0.15 1.21 Item 20 .66 0.47 0.60 .86 0.02 0.60 Note.b1 Threshold 1; b2 Threshold 2. In this table, restrictive setting refers to youth facilities providing long-term incarceration, whereas less restrictive settings include probation, short-term detention, and clinic settings. 78 KOSSON ET AL. separately in the two samples. In this case, allowing separate loadings on one or two items was not sufficient to produce invari- ance, suggesting that the differences in the loadings between the two subsamples are more numerous than for the other models and subsamples examined, albeit within the context that the samples were somewhat smaller than is recommended for conducting such analyses. An additional MGCFA using mean structures was conducted to examine whether estimated factor means differed in the incarcer- ated versus the clinic/detention subsamples. In this analysis, mean levels of each factor were set to 0 for the incarcerated subsample. This analysis yielded estimated factor means in the clinic/ detention/probation sample as follows: for the interpersonal factor, 0.86,z 8.07; for the affective factor, 0.62,z 6.84; for the lifestyle factor, 0.70,z 7.48; for the antisocial factor, 0.86, z 9.01 (allps .001). Discussion The principal aim of this study was to examine the fit of the three- and four-factor models of psychopathy as assessed with the PCL:YV in a large sample of adolescent females. Because no large-scale analyses have previously been reported, this study was designed to provide greater clarity on the factor structure of the PCL:YV in adolescent females. Analyses revealed consistently acceptable fit for both models in this large sample. More specifi- cally, the TLI and SRMR suggest good to excellent fit for both models, and the CFI and RMSEA indicate acceptable, although not excellent, fit for both models. Given that no prior studies had included as many participants as is recommended given the num- ber of parameters to be estimated, these findings demonstrate that, when sample size and power are adequate, these models provide a good explanation for the pattern of intercorrelations among PCL:YV item scores. Integrating these findings with large-scale tests of factor struc- ture using PCL-based measures in other kinds of samples, there is now evidence that both the three- and four-factor models achieve acceptable fit in adult males, in most studies of adult females, in adolescent males, and in adolescent females (Babiak et al., 2010; Cauffman et al., 2009; Hare & Neumann, 2008; Jackson, Neu- mann, & Vitacco, 2007; Neumann & Hare, 2008; Neumann et al., 2006; Salekin et al., 2006; Vitacco, Neumann, & Jackson, 2005). Taken together, these findings indicate that the factor structure of psychopathy as assessed by PCL measures is relatively robust. These results are important because some have argued, based on small sample studies, that the four-factor model does not fit. Moreover, it is important to keep in mind that, in contrast to studies that pit theoretical predictions against a null hypothesis, increases in statistical power in model-fitting analyses do not substantially increase the likelihood of obtaining results that cor- roborate a theory (e.g., see Rodgers, 2010). In other words, good fit is not a simple function of sample size. Rather, adequate sample size simply ensures a powerful test of the adequacy of a model. As noted earlier, factor analytic studies cannot provide direct tests of the superiority of one of these factor models to the other (T. A. Brown, 2006). Moreover, the indirect evidence regarding model fit indicates that, at this level, both models provide adequate fit. Thus, the relative value of one versus the other model must be decided based on other criteria (cf. Neumann, Hare, & Newman,2007; Neumann, Vitacco, Hare, & Wupperman, 2005; Vitacco, Neumann, & Jackson, 2005, 2006, 2010). Hare and Neumann (2010) and Vitacco, Neumann, & Jackson (2005) have raised questions about the methods reported by Cooke and Michie (2001) for selecting the 13 items in the three-factor model and the criteria for excluding “antisocial” items. Given that the four-factor model subsumes the three-factor model, these findings demonstrate sub- stantial evidence that it provides a powerful conceptual architec- ture for understanding the correlates and mechanisms underlying psychopathy as well as for testing the predictive validity of psy- chopathy scores. However, research should also examine the CFA results of subsamples to determine whether there are differences across subsamples of psychopathic youth. We discuss these issues below. The Internal Structure of PCL:YV Psychopathy in Subsamples The subsample analyses suggest that both the three-factor and four-factor models also provide a reasonable representation of PCL:YV item score intercorrelations in most of the smaller sub- samples examined. It must be emphasized that although these subsamples were larger than those in prior factor analytic studies of the PCL:YV in girls, they were smaller than is recommended for testing the four-factor model. Because the three- and four-factor models estimate 29 and 42 free parameters, tests of these models should include at least approximately 300 and 420 subjects, re- spectively (using a 10:1 ratio of subjects-to-free parameters; Bentler, 1980). Therefore, our subsamples of approximately 300 (Ns 277 to 369) should have been adequate to evaluate the three-factor model but may have been somewhat underpowered with respect to the four-factor model. That the models yielded evidence of adequate fit in the less restrictive (probation/detention/clinic) sample and in the North American sample and generally adequate fit in the European sample provides evidence for the robustness of these models. There was also some evidence for invariance of the three-factor model (i.e., allowing only one item loading to freely vary), but this same situation was not evident for the four-factor model. It is important to keep in mind that these analyses nevertheless allowed the groups to differ in their thresholds. Whereas differences in the pattern of indicator-to-factor loadings are commonly interpreted as indicating differences in factor structure, differences in thresholds refer to distinctions in the levels of the underlying latent constructs at which the items are maximally discriminating. Overall, the findings here showed that levels of psychopathy tend to be higher in prisons (i.e., settings involving long-term incarceration) than in community and detention (short-term incarceration) settings (Forth et al., 2003). Similarly, levels of psychopathic traits appear to be higher in North American than in European settings (Sullivan & Kosson, 2006; Verona, Sadeh, & Javdani, 2010). In other words, higher levels of psychopathic traits must be present, on average, in individuals from North American and prisons samples before the items provide information (discrimination) on those with (vs. without) psychopathic personality features. In spite of the generally acceptable fit for both models for subsamples in different continents and across settings, the multi- group analyses also demonstrated that the fit was better when the different subsamples were allowed to differ in some item loadings. 79 FACTOR STRUCTURE IN GIRLS Even allowing the indicator-to-factor loadings to differ for one or two items was sufficient to render the chi-square difference test (regarding constrained vs. unconstrained models) nonsignificant for the three-factor model. Consequently, some of the item load- ings are not identical across geographical region and setting. Yet most of the differences in item-to-factor loadings that we observed are small enough that they do not result in significant differences in overall model fit. However, in each case, there was at least one PCL:YV item for which the difference in loadings was substantial enough to produce a significant chi-square difference test, unless the loadings on this item were permitted to vary across groups. In summary, in most cases there were apparent differences in some loadings (as shown in Table 3), but these differences were not sufficient to produce a lack of invariance. In contrast, even allowing the loadings on one or two items to vary across groups was not sufficient to obtain evidence of struc- tural invariance for the four-factor model. In this case, the multi- group analysis continued to demonstrate a lack of invariance, even when loadings were estimated separately for several indicators. For example, as shown in Table 3, there appear to be important differences in the loadings of PCL:YV items associated with several different psychopathy dimensions in settings involving long-term incarceration versus those involving short-term incar- ceration. In spite of the suboptimal size of the subsamples for assessing the fit of the four-factor model, the relatively weaker fit for the four-factor model in the incarcerated subsample than in the other subsamples examined (i.e., acceptable on absolute fit indices but not relative fit indices) and the lack of invariance for the four- factor model across continents and across more restrictive and less restrictive settings merit discussion. The lack of invariance in the multigroup analysis appears to reflect the fact that there were several items with different factor loadings in the long-term versus shorter-term incarceration and community samples. It is possible that both these findings reflect the relatively poor fit of the four- factor model in a single but relatively large sample (i.e., the incarcerated sample of Sevecke et al., 2009). Alternatively, it is possible that the four-factor model does not fit as well among incarcerated girls as among probation and detention and clinic girls or among samples that collapse across settings. However, this latter possibility appears to us less parsimonious given the general consistency of large sample analyses discussed above. Alterna- tively, the lack of invariance may reflect the possibility of funda- mental differences in how well some antisocial and lifestyle items discriminate in different groups of individuals (for more informa- tion, see Mokros et al., 2010). Given that the subsamples were smaller than recommended for evaluating the fit of the four-factor model, the only way to resolve these issues would be to conduct an analysis of a large sample of incarcerated female adolescents (i.e., including 420 or more incarcerated girls). Limitations As discussed in the Introduction, factor analytic studies can make a valuable contribution to the construct validation enterprise (Strauss & Smith, 2009). Evidence that a structural model accounts for the pattern of covariances among item scores in a new popu- lation and, thus, can be generalized to the pattern observed in other populations provides powerful evidence that the larger constructthat a given measure assesses is similar across the two populations. Thus, our results highlighting that scores on PCL factor indicators covaried in similar ways in adolescent girls versus what has been found with other samples suggests that the PCL-based conceptu- alization of psychopathy is likely similar in adolescent girls, com- pared with other diverse samples. At the same time, factor analytic studies have notable limitations with respect to construct validity research. Evidence for patterns of similar coherence among item scores and similar covariance among factor scores does not ensure that the underlying construct is the same. Consequently, it remains possible that the nomological network surrounding psychopathy and the four (or three) components of psychopathy is different in critical ways in adolescent females than in adolescent males. Other types of studies and research designs are necessary to evaluate relationships between psychopathy (and psychopathy components) and the quasi-criteria linked to psychopathy in adults and in adolescent males. Even so, the existence of similar internal structure in adolescent females assessed with the PCL:YV pro- vides a foundation that permits clearer interpretation of the pattern of correlations with theoretically informed criteria in studies of adolescent females. To the extent that item scores on the different PCL:YV items and composite scores of the factors themselves cohere in similar ways in adolescent girls and in other samples, it becomes unlikely that the construct of psychopathy is wholly different in adolescent girls than in other groups, and it becomes more likely that the similarities that are observed in the patterns of correlations between PCL psychopathy (total and composite facet) scores and scores for external criteria reflect similar mechanisms. Conversely, in the context of similar underlying internal struc- ture, differences in the pattern of observed correlations are likely to reflect true differential relationships between PCL-based psy- chopathy and other constructs (cf. Odgers et al., 2005). Conse- quently, in light of current findings, future studies examining the convergent validity and discriminant validity of scores on the four dimensions of psychopathy in adolescent girls are especially im- portant. In this context, studies examining construct validity that compute correlations between scores on latent variables (instead of manifest indicators) and external criteria have the advantage that they permit modeling of variance in these indicators separately from unique and error variance (Bentler, 1980). At least one additional important limitation of our use of factor analysis is noteworthy. As noted in the introduction, this study did not address the internal structure of other kinds of measures of psychopathy. It remains possible that studies employing self-report measures and informant (i.e., parent and teacher) measures of psychopathic traits will ultimately yield evidence of a different internal structure in adolescent females than in adolescent males and adults. As discussed by Strauss and Smith (2009), the nomo- thetic span of a construct refers to the extent to which different measures of a construct provide evidence for similar patterns of relations with external criteria. As we mentioned earlier (see Footnote 2), findings obtained using different kinds of psychopa- thy measures often do not converge with respect to the nature of the internal structure of psychopathy. Absent evidence that psy- chopathy is underlain by similar components across different kinds of measures (but see Williams, Paulhus, & Hare, 2007, for one exception), we could not expect to see similar patterns of relation- ships across distinct measures. Yet attempts to understand and 80 KOSSON ET AL. overcome these methodological limitations are important to over- come a monomethod bias in psychopathy research. In general, there has been little research examining the possi- bility of differences in item loadings on underlying psychopathy dimensions in different samples. The evidence for differences across subsamples in the loadings of one or two items on a latent factor suggests that there may be differences in rater behavior across settings, or, more substantively, differences in the way psychopathic traits are manifested across diverse groups of indi- viduals. Only additional research can address the replicability of these differences and the reasons why items reflecting certain features of psychopathy appear to function differently across set- ting and in subsamples of youth in different continents. One possibility discussed by Hare and Neumann (2006) is that items reflecting early, persistent, and versatile antisociality become in- creasingly important when examining nonincarcerated and com- munity samples. However, this possibility cannot explain the cur- rent pattern of disparate loadings for items on all four dimensions in long-term incarcerated versus shorter-term detention and com- munity samples. Finally, one additional direction for future research is the ex- amination of sex differences in the latent structure of psychopathy as assessed with the PCL:YV. In light of the finding that the three-factor and four-factor models yield acceptable fit in a large samples of adolescent females, it becomes possible to ask if there is invariance in the latent structure of PCL:YV psychopathy across sex. Only a multigroup analysis with large samples of males and females can address this issue. References Andershed, H., Hodgins, S., & Tengström, A. (2007). Convergent validity of the Youth Psychopathic Traits Inventory (YPI): Association with the Psychopathy Checklist: Youth Version (PCL:YV).Assessment, 14, 144 –154. doi:10.1177/1073191106298286 Babiak, P., Neumann, C. S., & Hare, R. D. (2010). Corporate psychopathy: Talking the walk.Behavioral Sciences & the Law, 28,174 –193. Bauer, D. L. A., Whitman, L., & Kosson, D. S. (2011). Reliability and construct validity of Psychopathy Checklist: Youth Version scores in incarcerated adolescent girls.Criminal Justice and Behavior, 38,965– 987. doi:10.1177/0093854811418048 Benning, S. D., Patrick, C. J., Hicks, B. M., Blonigen, D. M., & Krueger, R. (2003). Factor structure of the Psychopathic Personality Inventory: Validity and implications for clinical assessment.Psychological Assess- ment, 15,340 –350. doi:10.1037/1040-3590.15.3.340 Bentler, P. M. (1980). Multivariate analysis with latent variables: Causal modeling.Annual Review of Psychology, 31,419 – 456. doi:10.1146/ annurev.ps.31.020180.002223 Bolt, D., Hare, R. D., Vitale, J., & Newman, J. P. (2004). A multigroup item response theory analysis of the Hare Psychopathy Checklist- Revised.Psychological Assessment, 16,155–168. Brinkley, C. A., Diamond, P. M., Magaletta, P. R., & Heigel, C. P. (2008). Cross-validation of Levenson’s Psychopathy Scale in a sample of federal female inmates.Assessment, 15,464 – 482. doi:10.1177/1073191108319043 Brown, T. A. (2006).Confirmatory factor analysis for applied research. New York, NY: Guilford Press. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.),Testing structural equation models(pp. 136 –162). Newbury Park, CA: Sage. Cauffman, E., Kimonis, E. R., Dmitrieva, J., & Monahan, K. C. (2009). A multimethod assessment of juvenile psychopathy: Comparing the pre-dictive utility of the PCL:YV, YPI, and NEO PRI.Psychological As- sessment, 21,528 –542. doi:10.1037/a0017367 Cooke, D. J., & Michie, C. (2001). Refining the construct of psychopath: Towards a hierarchical model.Psychological Assessment, 13,171–188. doi:10.1037/1040-3590.13.2.171 Cooke, D. J., Michie, C., Hart, S. D., & Clark, D. A. (2004). Reconstruct- ing psychopathy: Clarifying the significance of antisocial and socially deviant behavior in the diagnosis of psychopathic personality disorder. Journal of Personality Disorders, 18,337–357. Das, J., de Ruiter, C., Doreleijers, T., & Hillege, S. (2009). Reliability and construct validity of the Dutch Psychopathy Checklist: Youth Version: Findings from a sample of male adolescents in a juvenile justice treatment institution.Assess- ment, 16,88 –102. doi:10.1177/1073191108321999 Flight, J. I., & Forth, A. E. (2007). Instrumentally violent youths: The roles of psychopathic traits, empathy and attachment.Criminal Justice and Behavior, 34,739 –751. doi:10.1177/0093854807299462 Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9,466 – 491. doi:10.1037/1082-989X.9.4.466 Forth, A. E., Kosson, D., & Hare, R. (2003).The Hare Psychopathy Checklist: Youth Version.New York, NY: Multi-Health Systems. Fowler, T., Langley, K., Rice, F., van den Bree, M. B., Ross, K., Wilkin- son, L.,…Thapar, A. (2009). Psychopathy trait scores in adolescents with childhood ADHD: The contribution of genotypes affecting MAOA, 5HTT and COMT activity.Psychiatric Genetics, 19,312–319. Frick, P. J., & Hare, R. D. (2001).The Antisocial Process Screening Device.Toronto, Ontario, Canada: Multi-Health Systems. Gretton, H., & Hare, R. D. (2002). [Hare Psychopathy Checklist: Youth Version scores]. Unpublished raw data. Hare, R. D. (2003).Manual for the Hare Psychopathy Checklist—Revised (2nd ed.). Toronto, Ontario, Canada: Multi-Health Systems. Hare, R. D., & Neumann, C. S. (2006). The PCL-R assessment of psy- chopathy: Development, structural properties, and new directions. In C. Patrick (Ed.),Handbook of psychopathy(pp. 58 – 88).New York, NY: Guilford Press. Hare, R. D., & Neumann, C. S. (2008). Psychopathy as a clinical and empirical construct.Annual Review of Clinical Psychology, 4,217–246. doi:10.1146/annurev.clinpsy.3.022806.091452 Hare, R. D., & Neumann, C. S. (2009). Psychopathy and its measurement. In P. J. Corr & G. Matthews (Eds.),Cambridge handbook of personality psychology(pp. 660 – 686). Cambridge, England: Cambridge University Press. Hare, R. D., & Neumann, C. S. (2010). The role of antisociality in the psychopathy construct: Comment on Skeem and Cooke (2010).Psycho- logical Assessment, 22,446 – 454. doi:10.1037/a0013635 Hart, S. D., Cox, D. N., & Hare, R. D. (1995).Manual for the Psychopathy Checklist: Screening Version (PCL:SV).Toronto, Ontario, Canada: Multi-Health Systems. Hoyle, R. H. (1995).Structural equation modeling: Concepts, issues and applications.Thousand Oaks, CA: Sage. Hu, L., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Issues, concepts, and applications(pp. 76 –99). Newbury Park, CA: Sage. Indoe, D. (2002). [Hare Psychopathy Checklist: Youth Version scores]. Unpublished raw data. Jackson, R. L., Neumann, C. S., & Vitacco, M. L. (2007). Impulsivity, anger, and psychopathy: The moderating effect of ethnicity.Journal of Personality Disorders, 21,289 –304. doi:10.1521/pedi.2007.21.3.289 Jones, S., Cauffman, E., Miller, J. D., & Mulvey, E. (2006). Investigating different factor structures of the Psychopathy Checklist: Youth Version: Confirmatory factor analytic findings.Psychological Assessment, 18, 33– 48. doi:10.1037/1040-3590.18.1.33 Kennealy, P. J., Hicks, B. M., & Patrick, C. J. (2007). Validity of factors of the Psychopathy Checklist—Revised in female prisoners: Discrimi- 81 FACTOR STRUCTURE IN GIRLS nant relations with antisocial behavior, substance abuse, and personality. Assessment, 14,323–340. doi:10.1177/1073191107305882 Kline, R. B. (1998).Principles and practice of structural equation mod- eling. New York, NY: Guilford Press. Kosson, D. S., Cyterski, T. D., Steuerwald, B. L., Neumann, C., & Walker-Matthews, S. (2002). The reliability and validity of the Psychop- athy Checklist: Youth Version in non-incarcerated adolescent males. Psychological Assessment, 14,97–109. doi:10.1037/1040-3590.14.1.97 Kosson, D. S., McBride, C. K., Whitman, L. A., Riser, R. E., Greco, J., & Tercek, R. (2012).Preliminary evidence for negative affectivity and maladaptive emotion regulation strategies in youth with psychopathic traits.Manuscript submitted for publication. Krischer, M. K., & Sevecke, K. (2008). Early traumatization and psychop- athy in female and male juvenile offenders.International Journal of Law and Psychiatry, 31,253–262. doi:10.1016/j.ijlp.2008.04.008 Kubak, F. A., & Salekin, R. T. (2009). Psychopathy and anxiety in children and adolescents: New insights on developmental pathways to offending. Journal of Psychopathology and Behavioral Assessment, 31,271–284. doi:10.1007/s10862-009-9144-2 Larsson, H., Andershed, H., & Lichtenstein, P. (2006). A genetic factor explains most of the variation in the psychopathic personality.Journal of Abnormal Psychology, 115,221–230. doi:10.1037/0021-843X.115 .2.221 Lewis, K., & O’Shaughnessy, R. (1998, June).Predictors of violent recid- ivism in juvenile offenders.Poster presented at the annual convention of the Canadian Psychological Association, Edmonton, Alberta, Canada. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure mod- eling.Psychological Methods, 1,130 –149. doi:10.1037/1082- 989X.1.2.130 Mahmut, M. K., Menictas, C., Stevenson, R. J., & Homewood, J. (2011, April 25). Validating the factor structure of the Self-Report Psychopathy Scale in a community sample.Psychological Assessment.Advance online publication. doi:10.1037/a0023090 Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103,391– 410. doi:10.1037/0033-2909.103 .3.391 Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indices and dangers in overgeneralizing Hu and Bentler’s (1999) findings.Structural Equation Modeling, 11,320 –341. doi:10.1207/ s15328007sem1103_2 Mokros, A., Neumann, C. S., Stadtland, C., Osterheider, M., Nedopil, N., & Hare, R. D. (2010). Assessing Measurement Invariance of PCL-R Assessments from file reviews for North-American and German offend- ers.International Journal of Psychiatry & Law, 34,56–63. Murrie, D. C., Cornell, D. G., Kaplan, S., McConville, D., & Levy-Elkon, A. (2004). Psychopathy scores and violence among juvenile offenders: A multi-measure study.Behavioral Sciences & the Law, 22,49 – 67. doi:10.1002/bsl.573 Muthe´ n, L. K., & Muthe´ n, B. O. (1998 –2007).Mplus user’s guide(5th ed.). Los Angeles, CA: Muthe´ n & Muthe´n. Neumann, C. S., & Hare, R. D. (2008). Psychopathic traits in a large community sample: Links to violence, alcohol use, and intelligence. Journal of Consulting and Clinical Psychology, 76,893– 899. doi: 10.1037/0022-006X.76.5.893 Neumann, C. S., Hare, R. D., & Newman, J. P. (2007). The super-ordinate nature of the Psychopathy Checklist—Revised.Journal of Personality Disorders, 21,102–117. doi:10.1521/pedi.2007.21.2.102 Neumann, C. S., Kosson, D. S., Forth, A. E., & Hare, R. D. (2006). Factor structure of the Hare Psychopathy Checklist: Youth Version (PCL:YV) in incarcerated adolescents.Psychological Assessment, 18,142–154. doi:10.1037/1040-3590.18.2.142Neumann, C. S., Kosson, D. S., & Salekin, R. T. (2007). Exploratory and Confirmatory factor analysis of the psychopathy construct: Methodolog- ical and conceptual issues. In H. Herve´ & J. C. Yuille (Eds.),The psychopath: Theory, research, and practice(pp. 79 –104). Mahwah, NJ: Erlbaum. Neumann, C. S., Malterer, M. B., & Newman, J. P. (2008). Factor structure of the Psychopathic Personality Inventory (PPI): Findings from a large incarcerated sample.Psychological Assessment, 20,169 –174. doi: 10.1037/1040-3590.20.2.169 Neumann, C. S., Vitacco, M. J., Hare, R. D., & Wupperman, P. (2005). Reconstruing the “reconstruction” of psychopathy: A comment on Cooke, Michie, Hart, and Clark.Journal of Personality Disorders, 19, 624 – 640. doi:10.1521/pedi.2005.19.6.624 Odgers, C. L., Reppucci, N. D., & Moretti, M. M. (2005). Nipping psychopathy in the bud: An examination of the convergent, predictive, and theoretical utility of the PCL:YV among adolescent girls.Behav- ioral Sciences & the Law, 23,743–763. doi:10.1002/bsl.664 Penney, S. R., & Moretti, M. M. (2007). The relation of psychopathy to concurrent aggression and antisocial behavior in high-risk adolescent girls and boys.Behavioral Sciences and the Law, 25,21– 41. Rodgers, J. L. (2010). The epistemology of mathematical and statistical modeling: A quiet methodological revolution.American Psychologist, 65,1–12. Rowe, R. (2002).Predictors of criminal offending: Evaluating measures of risk/needs, psychopathy, and disruptive behavior disorders(Unpub- lished doctoral dissertation). Carleton University, Ottawa, Ontario, Can- ada. Salekin, R. T. (2008). Psychopathy and recidivism from mid-adolescence to young adulthood: Cumulating legal problems and limiting life oppor- tunities.Journal of Abnormal Psychology, 117,386 –395. Salekin, R. T., Brannen, D. N., Zalot, A. A., Leistico, A. R., & Neumann, C. S. (2006). Factor structure of psychopathy in youth: Testing the applicability of the new four factor model.Criminal Justice and Behav- ior, 33,135–157. Salekin, R. T., Leistico, A. R., Trobst, K. K., Schrum, C. L., & Lochman, J. E. (2005). Adolescent psychopathy and personality theory—the inter- personal circumplex: Expanding evidence of a nomological net.Journal of Abnormal Child Psychology, 33,445– 460. Salekin, R. T., Neumann, C. S., Leistico, A. R., DiCicco, T. M., & Duros, R. L. (2004). Psychopathy and comorbidity in a young offender sample: Taking a closer look at psychopathy’s potential importance over the disruptive behavior disorders.Journal of Abnormal Psychology, 113, 416 – 427. Salekin, R. T., Neumann, C. S., Leistico, A. R., & Zalot, A. A. (2004). Psychopathy in youth and intelligence: An investigation of Cleckley’s hypothesis.Journal of Clinical Child and Adolescent Psychology, 33, 731–742. Salekin, R. T., Rogers, R., & Sewell, K. W. (1997). Construct validity of psychopathy in a female offender sample: A multitrait–multimethod evaluation.Journal of Abnormal Psychology, 106,576 –585. Schmidt, F., McKinnon, L., Chattha, H. K., & Brownlee, K. (2006). Concurrent and predictive validity of the Psychopathy Checklist: Youth Version across gender and ethnicity.Psychological Assessment, 18, 393– 401. Schrum, C. L., & Salekin, R. T. (2006). Psychopathy in adolescent female offenders: An item response theory analysis of the Psychopathy Check- list: Youth Version.Behavioral Sciences and the Law, 24,39 – 63. Sevecke, K., Pukrop, R., Kosson, D. S., & Krischer, M. K. (2009). Factor structure of the Hare Psychopathy Checklist: Youth Version in German female and male detainees and community adolescents.Psychological Assessment, 21,45–56. Skeem, J. L., & Cauffman, E. (2003). Views of the downward extension: Comparing the Youth Version of the Psychopathy Checklist with the 82 KOSSON ET AL. Youth Psychopathic Traits Inventory.Behavioral Sciences & the Law, 21,737–770. Skeem, J. L., & Cooke, D. J. (2010). Is criminal behavior a central component of psychopathy? Conceptual directions for resolving the debate.Psychological Assessment, 22,433– 445. Stockdale, K. C., Olver, M. E., & Wong, S. C. P. (2010). The Psychopathy Checklist: Youth Version and adolescent and adult recidivism: Consid- erations with respect to gender, ethnicity, and age.Psychological As- sessment, 22,768 –781. Strauss, M. E., & Smith, G. T. (2009). Construct validity: Advances in theory and methodology.Annual Review of Clinical Psychology, 5, 1–25. Sullivan, E. A., & Kosson, D. S. (2006). Ethnic and cultural variations in psychopathy. In C. J. Patrick (Ed.),Handbook of psychopathy(pp. 437– 458). New York, NY: Guilford Press. Sutton, S. K., Vitale, J. E., & Newman, J. P. (2002). Emotion among women with psychopathy during picture perception.Journal of Abnor- mal Psychology, 111,610 – 619. Uzieblo, K., Verschuere, B., Van den Bussche, E., & Crombez, G. (2010). The validity of the Psychopathic Personality Inventory—Revised in a community sample.Assessment, 17,334 –346. Verona, E., Sadeh, N., & Javdani, S. (2010). The influences of gender and culture on child and adolescent psychopathy. In R. T. Salekin & D. R. Lynam (Eds.),Handbook of child and adolescent psychopathy(pp. 317–342). New York, NY: Guilford Press. Vincent, G. M., Odgers, C. L., McCormick, A. V., & Corrado, R. R. (2008). The PCL:YV and recidivism in male and female juveniles: A follow-up into young adulthood.International Journal of Law and Psychiatry, 31,287–296. Vitacco, M. J., Neumann, C. S., & Caldwell, M. F. (2010). Predicting antisocial behavior in high-risk male adolescents: Contributions of psy- chopathy and instrumental violence.Criminal Justice and Behavior, 37, 833– 846.Vitacco, M. J., Neumann, C. S., Caldwell, M. F., Leistico, A., & Van Rybroek, G. J. (2006). Testing factor models of the Psychopathy Check- list: Youth Version and their association with instrumental aggression. Journal of Personality Assessment, 87,74 – 83. Vitacco, M. J., Neumann, C. S., & Jackson, R. L. (2005). Testing a four-factor model of psychopathy and its association with ethnicity, gender, intelligence, and violence.Journal of Consulting and Clinical Psychology, 73,466 – 476. Vitacco, M. J., Rogers, R., Neumann, C. S., Harrison, K. S., & Vincent, G. (2005). A comparison of factor models on the PCL-R with mentally disordered offenders: The development of a four-factor model.Criminal Justice and Behavior, 32,526 –545. Vitale, J. E., Brinkley, C. A., Hiatt, K. D., & Newman, J. P. (2007). Abnormal selective attention in psychopathic female offenders.Neuro- psychology, 21,301–312. Vitale, J. E., & Newman, J. P. (2001a). Response perseveration in psycho- pathic women.Journal of Abnormal Psychology, 110,644 – 647. Vitale, J. E., & Newman, J. P. (2001b). Using the Psychopathy Checklist— Revised with female samples: Reliability, validity, and implications for clinical utility.Clinical Psychology: Science and Practice, 8,117–132. Warren, J. I., Burnette, M. L., South, S. C., Chauhan, P., Bale, R., Friend, R., & Van Patten, I. (2003). Psychopathy in women: Structural modeling and comorbidity.International Journal of Law and Psychiatry, 26, 223–242. Williams, K. M., Paulhus, D. L., & Hare, R. D. (2007). Capturing the four-factor structure of psychopathy in college students via self-report. Journal of Personality Assessment, 88,205–219. Received October 11, 2010 Revision received March 11, 2012 Accepted March 13, 2012 E-Mail Notification of Your Latest Issue Online! Would you like to know when the next issue of your favorite APA journal will be available online? This service is now available to you. Sign up at http://notify.apa.org/ and you will be notified by e-mail when issues of interest to you become available! 83 FACTOR STRUCTURE IN GIRLS