31561278

8/13/2019 31561278

1/10

Journal of Personality Assessment , 90(1), 93101, 2008Copyright C Taylor & Francis Group, LLCISSN: 0022-3891 print / 1532-7752 onlineDOI: 10.1080/00223890701693819

Differentiating Psychotic Patients From Nonpsychotic PatientsWith the MMPI2 and Rorschach

TAM K. DAO, FRANCES PREVATT , AND HEATHER LEVETA HORNE

Department of Educational Psychology and Learning Systems, Florida State University

The goal of this study was to examine the incremental validity and the clinical utility of the Minnesota Multiphasic Personality Inventory2(MMPI2; (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) and Rorschach (Rorschach, 1942) with regard to differential diagnosisin a sample of adult inpatients with a primary psychotic disorder or a primary mood disorder without psychotic features. Diagnostic efciencystatistics have suggested that the Rorschach PerceptualThinking Index (PTI;Exner, 2000a, 2000b) was better thanMMPI2 scales in discriminatingpsychotic patients from nonpsychotic patients. We compared the 84% overall correct classication rate (OCC) for the PTI to an OCC of 70% for the MMPI2 scales. Adding the MMPI2 scales to the PTI resulted in a decrease in OCC of 1%, whereas adding the PTI to the MMPI2 resultedin an increase in OCC of 14%. Sensitivity, specicity, positive predictive power, negative predictive power, and kappa were equal or higher withonly the PTI in the model.

The popularity of the Minnesota Multiphasic Personality In-ventory (MMPI; Hathaway & McKinley, 1940), MMPI2(Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989)and the Rorschach (Rorschach, 1942) in the assessment of psy-chopathology in applied settings have been well established(Archer & Krishnamurthy, 1993b; Greene, 2000). Empiricalsurveys of test usage found that the Rorschach and MMPI werethe two most extensively used (Archer, Maruish, Imhof, & Pi-otrowski, 1991; Watkins, Campbell, Nieberding, & Hallmark,1995) and researched (Archer & Krishnamurthy, 1993b) per-sonality assessment instruments.

Although it might be expected that the MMPI and theRorschach would be interrelated, a review of the literature re-vealed little congruence between these two widely used as-

sessment measures (Archer & Krishnamurthy, 1993a, 1993b;Ganellen, 1996a, 1996b, 1996c; Meyer, 1996, 1997; Meyer,Riethmiller, Brooks, Benoit, & Handler, 2000). The lack of convergent validity between these two instruments has becomea signicant conundrum in the eld of psychological assess-ment (Meyer, 1997) and has led to multiple explanatory models(Archer, 1996; Archer & Krishnamurthy, 1993b; Ganellen,1996a, 1996b, 1996c; Meyer, 1997; Meyer et al., 2000). Meyer (1997) and Meyer et al. (2000) have speculated that althoughthese two instruments do not produce signicant interrelation-ships under general conditions, meaningful relationships existunder specic psychometric conditions in which a similar re-sponse style is utilized by the participant during administrationof both instruments. Alternately, Archer and Krishnamurthy

(1993b) proposed that weak associations occurred as a resultof methodological limitations in previous studies. Archer andKrishnamurthy (1993b) postulated that although there is littleagreement among MMPIRorschach variables, the use of thesetests conjointly might provide a more valid assessment becauseeach instrument is contributing to the understanding of pathol-ogy in unique ways.

Received July 18, 2006; Revised February 23, 2007.Address correspondence to Frances Prevatt, Department of Educational Psy-

chologyand LearningSystems,Stone 307,Florida StateUniversity,Tallahassee,FL 32306; Email: [email protected]

The examination of incremental validity associated with theMMPI and Rorschach can be traced back to Kostlan (1954) whoexamined the utility of using the Rorschach, MMPI, sentencecompletion test, and patient history to predict personality de-scriptions. In Kostlans study, 20 clinicians interpreted differentcombinationsof test data, whichwere then compared against cri-terion judges who used a lengthy personality checklist. Kostlanfound that themost accurate personality descriptions were basedon the combination of using both the patients social history aswell as the MMPI. However, for the most part, clinicians wereno more accurate in their personality descriptions than if theyhad only used age, occupation, education, marital status, and thereferral problem. Since that time, there has been mixed evidenceregarding the incremental validity of the Rorschach and MMPI.

Studies that have failed to nd support for the combined useof the MMPI and Rorschach include Garb (1984), Archer andGordon(1988), and Archerand Krishnamurthy (1997). The rsttwo of these works were criticized for using inadequate or out-dated scoring systems (Ganellen, 1996b). In thestudy conductedby Garb, 32 studies were reviewed to examine the incrementalvalidity of personality assessment data. Garb found that a testbattery that included the Rorschach, Sentence Completion Test,and either an MMPI or a case history did not increase a clin-icians accuracy in diagnosing psychiatric patients when thesedata were added to demographic data. Likewise, Archer andGordon found that combining MMPI and Rorschach indexesdid not signicantly improve the classication accuracy for par-ticipants diagnosed with schizophrenia and depression. With a

sampleof adolescents, Archer and Krishnamurthy (1997) foundthat the classication rate was not improved when Rorschachvariables were added to a combination of MMPIAdolescentConduct Problems, Cynicism, and Immaturity scales (Butcher et al., 1992) for adolescents diagnosed with conduct disorder.

A review of the literature produced ve studies in supportof the combined use of the MMPI and Rorschach in predictingrelevant criteria. Ritsher (2004) investigated the relationship be-tween MMPI (Sc, BIZ, Sc3, and Sc6) and Rorschach (SCZI andPTI) psychosis variables and schizophrenia spectrum disordersin a Russian sample of psychiatric patients. Ritsher found thatnone of the MMPI variables showed an expected association

93

8/13/2019 31561278

2/10

94 DAO, PREVATT, HORNE

with schizophrenia spectrum disorders for any of the diagnosticsystems (e.g., Moscow school, Moscow International Classi-cation of Diseases [ICD]9, and ICD10). However, the SCZIand PTI had statistically signicant associations with several of the diagnostic categorizations, but these associations were mod-est in magnitude. As well, the SCZI and PTI had higher positivepredictive power (PPP) values and higher specicity scores thanthe MMPI Sc scale and lower sensitivity and false positive rates.Meyer (2000a) examined the incremental validity of the MMPIand Rorschach with a series of meta-analyses. Within six stud-ies, Meyer (2000a) found that the Rorschach Prognostic Ratingscale demonstrated incremental validity over the MMPI EgoStrength scale in the prediction of treatment outcome. In re-gards to diagnostic criteria, Meyer (2000b) found that the SCZIand the DEPI contributed meaningful information to the predic-tionof schizophrenicanddepressive disordersabove andbeyondthat obtained from the MMPI2 psychotic-related indexes. Sim-ilarly, Blais, Hilsenroth, Castlebury, Fowler, and Baity (2001)examined the incrementalvalidityof the Rorschachin predictingchart review ratings of antisocial, borderline, histrionic, andnar-cissistic personality disorders. The results suggest that both theMMPI2 andRorschach data added incrementally to the predic-tion of borderline and narcissistic personality disorder criteria;however, the ndings were less clear for the incremental valueof the Rorschach and MMPI2 data in predicting the number of histrionic and antisocial personality disorder criteria. In anarticle on the different methods for assessing incremental valid-ity of a Rorschach variable, Dawes (1999) reevaluated Meyer and Resnicks (1996) data set and found that the RorschachEgo Impairment Index provided a signicant contribution over and above other Rorschach and MMPI2 variables in predict-ing ratings of severity of diagnosis. In summary, the discrepantliterature ndings regarding the combined use of the MMPIand Rorschach suggest a clear need for further research to in-vestigate the magnitude of MMPIRorschach relationships andthe clinical utility provided when these instruments are usedtogether in predicting diagnostic criteria.

In this study, we examined how well the MMPI2 andRorschach can identify diagnostic classications of primarypsychotic disorder (PPD) or primary mood disorder withoutpsychotic features (PMD). There are a number of specicMMPI2 scales that have been associated with the detectionof psychosis. According to Greene (2000), Clinical Scales 6(Pa) and 8 (Sc) are indicative of schizophrenia symptomatol-ogy. The Pa scale contains items that assess content areas suchas suspiciousness, interpersonal sensitivity, persecutory ideas,and moral self-righteousness. A number of studies have foundthe Sc (Bagby et al., 2005; Ben-Porath, Butcher, & Graham,1991; Munley, Busby, & Jaynes, 1997) and Pa (Greenblatt& Davis, 1999; Munley et al., 1997) scales to be useful indistinguishing psychotic groups from other criterion groups.Along with the MMPI2 Clinical scales, the Content BIZ scale(Butcher, Graham, Williams,& Ben-Porath, 1990)which con-tains items that measure psychotic thought processes, hallucina-tions, delusions of persecution, paranoia, and unusual thoughtsand experienceshas also been reported in the literature topredict behaviors that are characteristic of psychosis (Archer,Aiduk, Grifn, & Elkins, 1996; Greene, 2000). A number of indexes that involve combining various Validity and/or Clinicalscales in a linear pattern have been proposed as additional meth-ods of determiningclients characteristics. Goldberg(1965) sug-

gested a linear index based on T scores [(Scale L + Scale 6 +Scale 8) (Scale 3 Scale 7)] as being a good indicator indifferentiating neurotic and psychotic proles. 1

Similar to the MMPI2, the Rorschach Comprehensive Sys-tem (RCS; Exner, 2003) has demonstrated some utility in dis-criminating between criterion-based diagnoses. The PerceptualThinking Index (PTI; Exner, 2000a, 2000b) has become the pre-ferred index for assessing cognition prior to interpreting other variables that may be related to thought disturbance (Exner,2000a). The PTI is comprised of eight Rorschach variables,which are arranged based on a combination of different valueson ve empirical criteria. It measures both perceptual odditiesand cognitive slippage (Smith, Baity, Knowles, & Hilsenroth,2001). Current research on the PTI is limited; however, recentstudies have indicated that the PTI is an effective index in differ-entiating psychotic from nonpsychotic patients (Dao & Prevatt,2006; Exner, 2000a; Kumar & Khess, 2005; Ritsher, 2004). ThePTI is composed of a number of Rorschach variables based ona combination of different values on ve empirical criteria, asfollows:

1. X + % < .70 and WDA% < .75 [X + % = the sum of allform qualityplus ( + ), ordinary (o), andunusual (u)responsesdivided by total responses (R). WDA% = the sum of + , o,and u responses given to whole (W) and common detail (D)areas divided by the sum of all responses given the to the Wand D areas].

2 . X % > .29 (X % = the sum of distorted form levelresponses divided by R).

3. Level 2 Special Score > 2 and FAB2 > 0 (Level 2 SpecialScore = severe cognitive disruption; FAB2 = Level 2 Fabu-lized Combination or implausible relationship between twoobjects).

4. R < 17 and WSum6 > 12 or R > 16 and WSum6 > 17(R = number of responses; WSum6 = the weighted sum of the 6 Special Scores indicating cognitive disruption).

5. Either: M > 1 or X % > .40 (M = human objects inmovement with distorted form level).

In this study, we investigated the incremental validity andthe clinical utility of the MMPI2 and Rorschach with regardto differential diagnosis in a sample of adult inpatients witha PPD. A comparison group included adult inpatients with aPMD. We conducted hierarchical, logistic regression analysesto investigate the additive benets of using both the MMPI2and Rorschach in detecting individuals diagnosed with a PPD.Hierarchical, logistic regression analyses have been endorsed asthe appropriate statistical method for determining incrementalvalidity (Blais et al., 2001; Cohen & Cohen, 1983; Hunsley &Meyer, 2003). The indexes measuring psychosis on the MMPI 2 and Rorschach have been found to be generally reliable andvalid. Furthermore, previous studies have typically found thatthey are uncorrelated with one another. Thus, we anticipatedthat the Rorschach data would contribute signicantly over the

1 The Goldberg Index (1965) was included in subsequent analyses based onthe advice of a reviewer. Given that the Rorschach PTI can be considered as acomplex index, the inclusion of the Goldberg Index allowed the Rorschach PTIto be evaluated against a comparable MMPI2 index. A T score greater than 45on the Goldberg Index suggests a psychotic prole pattern, and a T score scoreof 44 or below suggests a neurotic pattern.

8/13/2019 31561278

3/10

INCREMENTAL VALIDITY OF THE MMPI2 AND RORSCHACH 95

MMPI2 in predicting PPD patients. Similarly, we anticipatedthat the MMPI2 data would contribute signicantly over theRorschach in predicting PPD patients.

METHOD Participants

The sample consisted of 236 patients drawn from an archivalsearch of les at a 60-bed inpatient psychiatric facility in thesoutheastern United States. Of the 236 patients in this study, 94were used in a previous study that examined the psychometricproperties of the PTI (Dao & Prevatt, 2006). The aggregatesample ranged in age from 18 to 74 years ( M age = 33), and46% of all participants were male ( n = 109). As for ethniccomposition, 77% were White ( n = 182), 18% were AfricanAmerican ( n = 43), and 5% were Hispanic ( n = 11). The PPD(n = 108) group ranged in age from 18 to 74 years ( M age =35), and 56% were male ( n = 61). The ethnic composition of the PPD group consisted of 72% White ( n = 77), 24% AfricanAmerican ( n = 26), and 4% Hispanic ( n = 5). The PMD group(n = 128) ranged in age from 18 to 72 years ( M age = 32),and 45% were male ( n = 57). The ethnic composition of thePMD group consisted of 81% White ( n = 104), 14% AfricanAmerican ( n = 18), and 5% Hispanic ( n = 6).

To assess for potential confounding group demographicvariables between the PPD and the PMD groups, we per-formed chi-square tests on gender and ethnicity. For gen-der, the chi-square test indicated no signicant difference, 2(1 , N = 236) = 3.35 , p > . 05 (two-tailed), with = .12.A chi-square test comparing White to Others (i.e., AfricanAmerican and Hispanic) indicated no signicant difference, 2(1 , N = 236) = 3.25 , p > . 05 (two-tailed), with = .12. Inregards to age, unpaired t tests indicated no signicant differ-ence, t (234) = .55 ,p > . 05 (two-tailed), with d = .07.

PPD ( n = 108) and PMD ( n = 128) groups were formedbased on primary admission diagnoses. Diagnoses of the PPDgroup consisted of schizophrenia ( n = 94), psychotic disor-der not otherwise specied ( n = 13), and delusional disorder (n = 1). Diagnoses of the PMD group consisted of major de-pressive disorder without psychotic features ( n = 112) anddepressive disorder not otherwise specied without psychoticfeatures ( n = 16). A retrospective review of chart records in-dicated that every patient in the PPD group was prescribed atleast one antipsychotic medication, whereas all of those in thePMD group were prescribed at least one antidepressant medica-tion. The PPD group had 92 patients with comorbid diagnosescompared to 105 patients from the PMD group who had co-morbid diagnoses. The types of comorbid diagnoses in the PPDgroup consisted of Axis I and Axis II disorders. Axis I comor-bid disorders for the PPD group included posttraumatic stressdisorder, alcohol dependence, polysubstance dependence, gen-eralized anxiety disorder, and panic disorder. Axis II comorbiddisorders for the PPD group consisted of schizoid personalitydisorder, obsessivecompulsive personality disorder, and per-sonality disorder not otherwise specied. Similar to the PPDgroup, the types of comorbid diagnoses in the PMD group con-sisted of Axis I and Axis II comorbid disorders. A number of Axis I comorbid disorders were found that included gen-eralized anxiety disorder, alcohol dependence, polysubstancedependence, posttraumatic stress disorder, bulimia nervosa, andattention-decit/hyperactive disorder. Axis II comorbid disor-

ders for the PMD group consisted of borderline personality dis-order, antisocial personality disorder, and personality disorder not otherwise specied.

ProcedurePrior to the study, institutional review board approval was

obtained from thehospital facility as well as from ourUniversity

Human Subjects Committee. A search of medical records of adult inpatients admitted to the hospital from 2000 through2006 was performed. The initial sample consisted of 1,314 adultpsychiatric inpatients. The selection of cases proceeded in threephases. In the rst phase, cases were excluded if they did notmeet the following criterion:

1. Participants were excluded if they were not diagnosed withone of two of the following categories of psychiatric disor-ders: (a) PPD consisting of schizophrenia, psychotic disor-der not otherwise specied, and delusional disorder; or (b)PMD consisting of major depressive disorder without psy-chotic features and depressive disorder not otherwise spec-ied without psychotic features. Based on this criterion, atotal of 712 cases were excluded, leaving 602 cases havingdiagnoses of PPD or PMD.

In the second phase, cases were excluded if they did not meetall three of the following criteria:

1. Participants were excluded if they were not administeredboth theMMPI2and the Rorschach. Based on thiscriterion,307 cases were excluded, leaving 295 cases.

2. Participants were excluded if the MMPI2 was not scoredusing a computerizedscoring system. Based on thiscriterion,8 cases were excluded, leaving 287 cases.

3. Participants were excluded if the Rorschach was not admin-istered and scored according to the RCSs guidelines (Exner,1993).Based on this criterion, 6 cases were excluded, leavinga sample size of 281 participants.

In the third phase, we examined the remaining 281 MMPI2and Rorschach protocols for validity. Cases were excluded if they did not meet all three of the following conditions:

1. Participants wereexcludedif Rorschachprotocolswereilleg-ible or incomplete (i.e., structural summary, location sheets,etc.). Based on this criterion, 21 cases were excluded, leav-ing 260 cases (3 due to illegible handwriting and 18 due toabsence of location sheets).

2. Participants were excluded if Rorschach protocols consistedof less than the minimum of 14 responses. A number of stud-ies have cautioned against interpreting Rorschach protocolscontaining fewer than 14 responses (Exner, 1993; Exner &Weiner, 1995). Based on this criterion, 10 cases were ex-cluded, leaving 250 cases.

3. Participants were excluded if they did not provide a valid andaccurate MMPI2 prole assessed by the following criteria(Butcher, Graham, & Ben-Porath, 1995):a. The Cannot Say (?) score on the MMPI2 was greater

than 30. No cases were excluded based on this criterion.b. The T score exceeded 80 on the VRIN scale. Based on

this criterion, 10 cases were excluded, leaving 240 cases.

8/13/2019 31561278

4/10


c. The T score exceeded 80 on the L scale. No cases wereexcluded based on this criterion.

d. The T score exceeded 80 on the K scale. Based on thiscriterion, 4 cases were excluded, leaving 236 cases.

e. The raw score exceeded 30 on the F scale. No cases wereexcluded based on this criterion.

Clinical diagnoses were based on intake evaluations conductedindependently by a licensed psychiatrist and clinical socialworker via two interview schedules, the Structured ClinicalInterview for the Diagnostic and Statistical Manual of Men-tal Disorders (4th ed. [ DSMIV ]; American Psychiatric Asso-ciation, 2000) Axis I Disorders: Clinical Versions (SCIDCV;Michael, Spitzer, Gibbon, & Williams, 1996) and the StructuredClinical Interview for DSMIV Personality Disorders (SCID II; Michael, Spitzer, Gibbon, & Williams, 1997) and in ac-cordance with DSM-IV-TR . Intake diagnoses were establishedthrough consensus by the psychiatrist and clinical social worker within 48 hr of admission. In all cases, the intake diagnosis wasestablished before the Rorschach was administered. Psychol-ogy interns, who were blind to the diagnosis, administered theRorschach within the rst 7 days following a patients admis-sion to the hospital. Once the Rorschach was administered, theRorschach protocols were scored by the intern under the directsupervision of the director of psychological services who wasalso blind to the diagnosis. Psychology interns consisted of up-per level graduate psychology students who had completed atleast 2 years in a psychology-oriented doctoral program. Be-tween 2000 and 2006, a number of graduate psychology internsadministered Rorschach protocols. However, the supervisingclinical psychologist remained the same, thus reducing the po-tential problems due to multiple raters. The director of psycho-logical services, also a licensed clinical psychologist, had over 13 years of extensive training in providing Rorschach adminis-tration, scoring, interpretation, and supervision.

To estimate interrater reliability, 40 Rorschachprotocols werechosen at random using a numbers table and rescored indepen-dently by a licensed clinical psychologist who was blind to theoriginal Rorschach scores as well as to patients diagnoses. In-traclass correlation reliability analysis was conducted for thetotal PTI score, which was .75. Based on Cicchettis (1994)guidelines; this statistic suggests good reliability.

Patients completed the MMPI2 independently via paper-and-pencil under the supervision of unit staff members. Scor-ing of the MMPI2 answer sheets was accomplished using acomputer scoring method, which produced Validity, Clinical,Content, and Supplementary scales. Administration and scor-ing of the MMPI2 followed Greenes (2000) and Butcher etal.s(1995) recommendations, whereas Rorschachprotocolsfol-lowed Exners (2003) RCS guidelines, with the structural sum-mary produced through the Rorschach Interpretation AssistanceProgram Version 5 (Exner, Weiner, & PAR Staff, 2001).

Intake evaluations were conducted for each admitted patient,with approximately 80% of these patients admitted involun-tary. Psychological assessment was not required of all patientsthat entered the hospital. Furthermore, the MMPI2 and theRorschach was not given to all patients as part of the admissionprocedure. However, the MMPI2 and Rorschach was routinelyadministered as part of an assessment battery with patients whoexhibited cognitive disturbances and/or depressive symptoms.

Instruments MMPI2. The MMPI2 (Butcher et al., 1989) is a struc-

tured psychological instrument that consists of 567 questionsthat require a true or false response. A number of articles havereported the psychometric properties of the MMPI2 and itsassociated scales: major reviews (Butcher et al., 1995; Greene,Gwin & Staal, 1997), testretest reliability (Munley, 2002), pre-

dictive validity (Vendrig, Derksen, & de Mey, 2000), convergentvalidity (Hicklin & Widiger, 2000; Rossi, Van den Brande, To-bac, Sloore, & Hauben, 2003), anddiscriminant validity (Strass-berg & Russell, 2000; Wise, 2001).

In this study, we used the following MMPI2 Clinical andContent scales: Scale 6 (Pa), Scale 8 (Sc) ,Bizarre Mentation(BIZ) , and the Goldberg Index (Goldberg 1965). A number of studies have found the Sc scale (Bagby et al., 2005; Ben-Porathet al., 1991; Munley et al., 1997), the Pa scale (Greenblatt &Davis, 1999; Munley et al., 1997), and the BIZ scale (Greenblatt& Davis, 1999; Munley et al., 1997) to be useful in distinguish-ing psychotic groups from other criterion groups.

Rorschach inkblot test. The Rorschach Inkblot Test (RIT)

consists of 10 inkblots (5 black and white and 5 containingcolors). The RCS (Exner, 2003) scoring of the responses con-stitutes the basis for the Structural Summary. The StructuralSummary provides the following types of interpretive data: (a)frequency statistics for the numerous individual variables as-sessed by the RCS, (b) seven subsections comprised of com-binations and ratios of variables that evaluate various types of cognitive and affective processing, and (c) six indexes that as-sess cognitive-perceptual proclivities related to specic types of categorical psychopathology. The following studies have exam-ined the overall psychometric properties of the RCS: testretestreliability (Gr onner od, 2003), interrater reliability (Archer &Krishnamurthy, 1997; Exner, 1993; McDowell & Acklin, 1996;Meyer et al., 2000), convergent validity (Archer & Krishna-

murthy, 1993b, 1997; Greenwald, 1997), and discriminant va-lidity (Ball, Archer, Gordon, & French, 1991).

STATISTICAL ANALYSESFollowing previous convention (Blais et al., 2001), we cor-

related the Rorschach PTI with the total number of Rorschachresponses (R). No signicant bivariate correlation (two-tailed)was found ( p < .05) between the Rorschach PTI and R; thus,there was no need to partial out the effect of R for subsequentanalyses. Based on recommendations of Blais et al. (2001) andHunsley and Meyer (2003), we computed t -test statistics to ex-amine differences between means of the PPD and PMD groupsacross MMPI2 Clinical Pa, Sc, Content BIZ, and GoldbergIndex scales and Rorschach PTI. For the MMPI2, we usedraw scores for the Pa scale and non-K-corrected raw scoreswere used for the Sc scale. We computed the Goldberg In-dex using K-corrected T scores. We used variables that weresignicantly different across the two groups in the subsequentregression models to test incremental validity. We used hierar-chical, logistic regression with forced order entry of MMPI2and Rorschach variables. For the analyses, there were two dataentry blocks in the hierarchy. In the rst block, we evaluated theMMPI2 scales. In the second block, we evaluated the MMPI 2 variables and Rorschach PTI simultaneously. We conductedthe analyses twice. In the next set of regression analyses, we

8/13/2019 31561278

5/10


reversed the order of the entry such that the Rorschach PTI wasevaluated rst followed by forced order entry of the MMPI2variables.

Incremental validity was assessed using the Nagelkerke(1991) R2and R. The Nagelkerke R2 is the most frequentlyreported of the R2 estimates. Given that Nagelkerke R2 willimprove as the number of variables increases, we computeddiagnostic efciency statistics for overall correct classication(OCC), sensitivity (SENS), specicity (SPEC), PPP, negativepredictive power (NPP), and kappa (see Streiner, 2003) usingthe decision rule of a predicted probability of .50 or greater. Inthis study, OCC refers to the proportion of individuals correctlyidentied as having PPD or PMD, SENS is dened as the pro-portion of people diagnosed with PPD who are detected as such,SPEC is the proportion of people who do not meet diagnosticcriteria for PPD and are correctly identied as not PPD, PPP isthe percent of individuals classied as having PPD who trulyhave the particular disorder, NPP is the percent of individualsclassied as not having PPD who truly do not have the par-ticular disorder, and Kappa represents the level of agreementbetween the predictor(s) and the diagnostic criteria beyond thataccounted for by chance alone.

RESULTSIn the rst analysis, we used t tests to examine group dif-

ferences between the MMPI2 scales and Rorschach PTI (seeTable 1).Results showed that thePPDgroup scoredsignicantlyhigher than the PMD group on all four of the MMPI2 scales.As well, the PPD group scored signicantly higher than thePMD group on the Rorschach PTI Criteria 1, 2, 5, and the To-tal PTI score. To assess for practical importance, we computedCohens (1988) d as the effect size, and it indicated moderateto large effects for the signicant MMPI2 and Rorschach vari-ables. We used the following variables in the subsequent testsof incremental validity: Pa, Sc, BIZ, Goldberg Index, and PTI.We did not use specic criteria on the PTI because they would

TABLE 1.Descriptive and t -teststatistics and effect sizesfor MinnesotaMultiphasicPersonalityInventory2 (MMPI2)scales and RorschachPerceptual ThinkingIndex (PTI) for the total sample and the primary psychotic disorder (PPD) and primary mood disorder (PMD) groups.

Total Sample a PPD Group b PMD Group c

MMPI2/Rorschach M SD Min Max M SD Min Max M SD Min Max t d

Scale 6 (Pa) 13 .9 4.0 3.0 21 .0 15 .0 3.9 3.0 21 .0 13 .0 3.8 4.0 21 .0 3.97 .52Scale 8 (Sc) 25 .8 10 .4 2.0 48 .0 28 .5 8.8 2.0 48 .0 23 .6 11 .1 4.0 45 .0 3.70 .49Bizarre Mentation (BIZ) 8 .0 4.6 0.0 16 .0 9.8 3.7 1.0 15 .0 6.5 4.8 1.0 16 .0 5.83 .76Goldberg Index 39 .2 21 .5 10 .0 91 .0 44 .4 18 .0 10 .0 91 .0 35 .0 23 .2 1.0 87 .0 3.42 .45PTI Criteria 1: XA% < .70 .56 .50 0 .0 1.0 0.79 0 .41 0 .0 1.0 0.36 0 .48 0 .0 1.0 7.32 .96

and WDA% > .75PTI Criteria 2: X % > .29 .50 .50 0 .0 1.0 0.73 0 .45 0 .0 1.0 0.31 0 .47 0 .0 1.0 6.97 .92PTI Criteria 3: Level 2 Special .17 .38 0 .0 1.0 0.21 0 .41 0 .0 1.0 0.14 0 .35 0 .0 1.0 1.41 .19

Score > 2 and FAB2 > 0PTI Criteria 4: R < 17 and WSum6 > 12 .16 .37 0 .0 1.0 0.19 0 .39 0 .0 1.0 0.13 0 .34 0 .0 1.0 1.26 .17

or R > 16 and WSum6 > 17PTI Criteria 5: M > 1 or X % > .40 .42 .50 0 .0 1.0 0.72 0 .45 0 .0 1.0 0.18 0 .39 0 .0 1.0 9.87 1.30Total PTI 1 .99 1 .7 0.0 5.0 2.95 1 .6 0.0 5.0 1.13 1 .4 0.00 3 .0 9.31 1.22

Note. Non-K-corrected raw scores were used to compute Pa, Sc, and BIZ, whereas K-corrected T scores were used to compute the Goldberg Index. The MMPI2 raw scores equatewith T scores for males/females as follows: For Pa, 13 = 61T/59T, 14 = 64T/63T, and 15 = 68T/67T; for Sc, 24 = 67T/66T, 26 = 69T/68T, and 29 = 73T/71T; and for BIZ, 7 = 67T/67T,8 = 70T/70T, and 10 = 77T/76T.p < . 01. p < . 001. a N = 236. b n = 108. c n = 128.

create problems of multicollinearity in regression equations thatalso used the full PTI.

Tables 2 and 3 contain the results for hierarchical, logistic re-gression analyses with the MMPI2 scales and Rorschach PTI.We conducted preliminary analyses to assess for potential out-liers and observations of excessive inuence. No outliers wereidentied that hada largerstandardized Pearson residual than thethreshold of 2.5, which is considered an adequate case index for the identication of outliers for univariate and multivariate stud-ies. Pearson residual is dened to be the standardized differencebetween the observed frequency and the predicted frequency. Itmeasures the relative deviations between the observed and t-ted values (Tate, 1998). As can be seen in Table 2, the MMPI2variables were entered in Block 1 followed by the addition of theRorschach PTI in Block 2. In Block 1, the MMPI2 variablescontributed uniquely to the prediction of group membership, 2(4, N = 236) = 26.6, p < .05. The Nagelkerke (1991) R 2 , acomparable statistic to the R 2 in linear regression, was .34. TheOCC rate using the MMPI2 variables was 70%. The Hosmer Lemeshowtest (Hosmer & Lemeshow, 1989), a test of themodelgoodness of t, produced a fail to rejectdecision 2(8, N = 236)= 10.6, p > .05, a result consistent with the assumption that thespecied logistic model was correct. In Block 2, the RorschachPTI entered and added signicantly to the model with a changein 2(1, N = 236) = 45.1, p < .05. The Nagelkerke R2 was.48. The OCC rate using a combination of MMPI2 variablesand Rorschach PTI was 84%, with a good t of the data to themodel, 2(8, N = 236) = 15.4, p > .05. Diagnostic efciencystatistics improved with the addition of the Rorschach PTI inBlock 2.

Table 3 provides the result of the reverse model with theRorschach PTI entering on Block 1 before we evaluated theMMPI2 variables. Preliminary analyses revealed no outlierswith standardized Pearson residual larger than the threshold of 2.5. Entry of the Rorschach PTI was signicant in predictinggroup membership, 2(1, N = 236) = 61.4, p < .05, witha good t of the data to the model, 2(4, N = 236) = 8.4,

8/13/2019 31561278

6/10


TABLE 2.Hierarchical, logistic regression analyses assessing the incremental validity of the Rorschach Perceptual Thinking Index (PTI) to predict primarypsychotic disorder (PPD) versus primary mood disorder (PMD) groups over Minnesota Multiphasic Inventory2 scales.

Variable B SE B Nagelkerke R 2 (R ) Nagelkerke R 2 (R ) OCC SENS SPEC PPP NPP Kappa

Block 1 .34 (.58) .70 .67 .73 .66 .74 .40Constant 2.6 .67Scale 6 (Pa) 0 .09 .04Scale 8 (Sc) 0 .08 .02

Bizarre Mentation 0 .10

.03Goldberg Index 0 .03 .01Block 2 .48 (.69) .14 (.37) .84 .81 .87 .85 .84 .69

Constant 2.2 .70Scale 6 (Pa) 0 .05 .04Scale 8 (Sc) 0 .04 .03Bizarre Mentation 0 .09 .03Goldberg Index 0 .02 .01Perceptual 0 .66 .74Thinking Index

Note. N = 236. Non-K-corrected raw scores were used to compute Pa, Sc, and BIZ, whereas K-corrected T scores were used to compute the Goldberg Index. OCC = overall correctclassication; SENS = sensitivity; SPEC = specicity; PPP = positive predictive power; NPP = negative predictive power.p < . 05. p < . 001.

p > .05. The Nagelkerke R 2 was .41. The OCC rate usingthe Rorschach PTI was 85%. In Block 2, MMPI2 variablescontributed uniquely to the prediction of group membershipwith a change in chi-square, 2(4, N = 236) = 6.17, p < .05.The Nagelkerke R 2 was .48. The OCC rate with the MMPI2variables included decreased slightly to 84%, with a good t of the data to the model, 2(8, N = 236) = 15.4, p > .05. Table 3also contains the diagnostic efciency statistics for each blockof the analyses. Diagnostic efciency statistics did not improvewhen we added the MMPI2 variables to the model.

DISCUSSIONThere has been little systematic effort in areas of applied psy-

chology to evaluate the incremental validity of measures and

assessment procedures (Hunsley & Meyer, 2003). Accordingto Hunsley and Meyer, newly developed measures need to beexaminedrelativeto alternative measuresalready available to as-sess the same construct. Furthermore, these new measures needto add to the prediction of a criterion above what can be pre-dictedby other sources of data. This form of incremental validityis important when a new measure has been created or an older

TABLE 3.Hierarchical, logistic regression analyses assessing the incremental validity of Minnesota Multiphasic Personality Inventory2 Scales to predict primarypsychotic disorder (PPD) versus primary mood disorder (PMD) groups over the Rorschach Perceptual Thinking Index (PTI).

Nagelkerke R 2 (R ) Nagelkerke R 2 (R )Variable B SE B OCC SENS SPEC PPP NPP Kappa

Block 1 .41 (.64) .85 .81 .89 .86 .85 .70Constant 1.6 .09PTI 0 .71 .24

Block 2 .48 (.69) .07 (.26) .84 .81 .87 .85 .84 .69Constant 2.2 .70PTI 0 .66 .74Scale 6 (Pa) 0 .05 .04Scale 8 (Sc) 0 .04 .03Bizarre Mentation 0 .09 .03Goldberg Index 0 .02 .01

Note. N = 236. Non-K-corrected raw scores were used to compute Pa, Sc, and BIZ, whereas K-corrected T scores were used to compute the Goldberg Index. OCC = overall correctclassication; SENS = sensitivity; SPEC = specicity; PPP = positive predictive power; NPP = negative predictive power.p < . 05. p < . 001.

instrument is revised or updated. It is within this framework that

we sought to address the incremental validity and clinical utilityof the MMPI2 and Rorschach on scales purported to measurepsychosis among a sample of adult inpatients.

Initial analyses revealed that the MMPI2 variables as well asthe Rorschach PTI were able to distinguish patients diagnosedwith a PPD from those that were diagnosed with a PMD. ThePPD group scored signicantly higher than the PMD group onthe Rorschach PTI. This is expected given that the PTI con-sists of variables that assess implausible relationships, illogicalor circumstantial thinking, and dissociated or distorted think-ing (Exner, 2000a). The PPD group also scored higher than thePMD group on PTI Criteria 1, 2, and 5, with Criterion 5 havingthe largest effect size. Based on the effect sizes for the differentPTI criteria, the variables assessing Form Quality performed

better in this sample at differentiating PPD and PMD groupsthan the variables measuring cognitive Special Scores. On theMMPI2, the PPD group scored signicantly higher than thePMD group on the Clinical scales Pa and Sc, on the Contentscale BIZ, as well as on the Goldberg Index. These scales tapdifferent aspects of a psychosis (e.g., unusual and atypical ex-periences, bizarre thought processes, social alienation, peculiar

8/13/2019 31561278

7/10


perceptions, suspiciousness, interpersonal sensitivity, persecu-tory ideas, moral self-righteousness, hallucinations, and delu-sions of persecution). Across instruments, differences betweenthe PPD and PMD groups produced moderate to large effectsizes for signicant MMPI2 variables and for the RorschachPTI.

On its own, the PTI showed a better overall classication ratethan the MMPI2. The PTI was able to correctly classify PPDand PMD patients 84% of the time as compared to 70% whenthe MMPI2variables were used. To assess the clinical utility of the MMPI2 and Rorschach in differentiating psychotic patientsfrom nonpsychotic patients, we computed diagnostic efciencystatistics at each block of the hierarchical regression analyses.When the MMPI2 variables were entered rst into the analy-ses, diagnostic efciency statistics indicated that the addition of the Rorschach PTI contributed to the prediction of group mem-bership above what was predicted by the MMPI2 variables.When the Rorschach PTI was entered rst into the analyses,diagnostic efciency statistics showed that the MMPI2 scaleswere not able to add to the predictive capacity of the RorschachPTI.

The MMPI Clinical Pa, Sc, and Content BIZ scales consistof items that assess areas such as bizarre thought processes, pe-culiar perceptions, social alienation, poor family relationships,concentration, suspiciousness, interpersonal sensitivity, halluci-nations, and delusions of persecution. These results suggest thatthe areas tapped by the MMPI are already being assessed bythe Rorschach PTI. Alternatively, the Rorschach PTI appears tobe measuring additional symptoms common to psychosis thatare not being assessed by the MMPI-2. One can speculate as tothe additional variance being accounted for by the Rorschach.It is possible that the response style of the Rorschach accountsfor this difference. When participants are limited to true or falseresponses, there are some natural boundaries inherent in thestructure imposed by the test. Alternately, when participantsare allowed free responses, there is a loosening of boundaries,and circumstantial thinking, unconventional reasoning, and de-viant verbalizations may be more likely to emerge. Informationobtained by the MMPI2 is dependent on the quality of individ-uals conscious self-schema (Meyer, 1997), whereas informa-tion obtained by the Rorschach is not dependent on consciousawareness. Therefore, it is speculated that the PTI score is morelikely to capture additional characteristics of psychosis.

There wasan increase in the Nagelkerke R 2 from 41% to 48%but a decrease in OCC rate from 85% to 84% when the MMPI 2 variables were entered into the analysis after the RorschachPTI (see Table 3). It is rare to see an increase in R2 at thesame time as a decrease in OCC, so this nding deserves further discussion. One would expect that these two statistics wouldvary in the same direction. One can speculate that the inverserelation between these two statistics is due to the fact that, inrelatively small samples, the Nagelkerke R 2 in is not directlyrelated to classication rates and that these rates can decreaseeven though the overall standard error of prediction is reducedby adding a predictor (G. J. Meyer, personal communication,June 27, 2006). One of the goals of logistic regression is todifferentiate participants into their respective groups. Thus, de-spite a decrease in diagnostic efciency statistics, two ndingssuggest that there was better separation of PPD and PMD cases.A comparison of classication plots in Blocks 1 and 2 suggeststhat in Block 2, the model does a better job of discriminating

participants who have a relatively even chance of belonging ineither the PMD or PPD group. A calculation of the proportion of cases with predicted probabilities ranging from .45 to .63 using just the Rorschach PTI and the complete model supports thisassertion. The proportion of cases with predicted probability inthis range when the PTI was used solely was .45. When both theMMPI2 variables and Rorschach PTI was used, the proportiondecreased to .15.

Despite the positive results for the incremental validity of the Rorschach PTI over the MMPI variables, a number of au-thors have cautioned against the concept of incremental valid-ity (Anastasi, 1988; Wiggins, 1973; Hunsley & Meyer, 2003).According to the preceding authors, conclusions about the in-cremental validity of a measure are context specic. In other words, the prevalence rate of a particular disorder may not gen-eralize to other contexts in which the prevalence rate differssubstantially. In situations when the prevalence rate is very low,a valid measure may demonstrate only modest incremental va-lidity (Anastasi, 1988; Hunsley & Meyer; 2003). Furthermore,given that there is a greater chance of a false positive clinicaldiagnosis in contexts where the prevalence rates are low, theimportance of examining other diagnostic efciency statisticssuch as SENS and SPEC becomes important. Based on the di-agnostic efciency statistics descriptions, the coefcients for SENS and SPEC were determined independent of the base rate,or prevalence, of being diagnosed with PPD (Streiner, 2003).The coefcients for PPP and NPP, on the other hand, take intoaccount the prevalence rate of being diagnosed with PPD in thisparticular sample. This distinction is important given that thissample consisted of 46% of patients diagnosed with PPD and54% of patients diagnosed with PMD, percentages that may notbe comparable to the full range of diagnoses encountered in aninpatient setting.

In summary, this study presents evidence in support of boththe Rorschach PTI and the MMPI2 as diagnostic tools in theprediction of PPDs. However, the incremental validity of theRorschach appears superior to that of the MMPI2. Despitepositive ndings, there are several limitations to consider, therst of which involves the number of patients who were on an-tipsychoticand antidepressant medications. A number of studieshave found that antipsychotic and antidepressant medicationscan affect neurocognitive domains such as attention, memory,and general executive and perceptual organization (Bilder et al.,2002; Dealberto, McAay, Seeman, & Berkman, 1997). Giventhe potential cognitive effects of psychotropic medications onpatient functioning, the test scores might not reect the truecognitive functioning of PPD and PMD patients. Similarly, theinterpretations pertaining to the classication of PPD and PMDbased on the MMPI2 and Rorschach should be made with cau-tion given that a number of patients had comorbid diagnoses.

There may also be issues with validity of the evaluationsgiven that patients tested immediately after hospitalization maybe acutely psychotic and prone to disorganized or impoverishedprotocols. Additionally, there are still questions regarding theoverlap between these two measures and the reasons for lack of convergent validity. According to Meyer (1997)andMeyeret al.(2000), results may differ when response styles of patients aretaken into consideration. Thus, this study may underestimate theincremental validity of these measures. Future research shouldinvestigate the effect of response styles on the incremental va-lidity of the MMPI2 and Rorschach.

8/13/2019 31561278

8/10


ACKNOWLEDGMENTTam K. Dao is currently a postdoctural fellow at the Michael

E. De Bakey VAMC, Houston, Texas. He is supported by the Of-ce of Academic Afliations, VA Special MIRECC FellowshipProgram in Advanced Psychiatry and Psychology, Departmentof Veterans Affairs.

REFERENCESAmerican Psychiatric Association. (2000). Diagnostic and statistical man-

ual of mental disorders (4th ed., text revision). Washington, DC:Author.

Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.Archer, R. P. (1996). MMPI-Rorschach interrelationships: Proposed criteria

for evaluating explanatory models. Journal of Personality Assessment, 67,504515.

Archer, R. P., Aiduk,R., Grifn, R.,& Elkins, D. E. (1996). Incremental validityof the MMPI2 Content scales in a psychiatric sample. Assessment, 3, 7990.

Archer, R. P., & Gordon, R. A. (1988). MMPI and Rorschach indices of schizophrenic and depressive disorders among adolescent inpatients. Journalof Personality Assessment, 52, 276287.

Archer, R. P., & Krishnamurthy, R. (1993a). Combining the Rorschach and the

MMPI in the assessment of adolescents. Journal of Personality Assessment,60, 132140.

Archer, R. P., & Krishnamurthy, R. (1993b). A review of MMPI and Rorschachinterrelationships in adult samples. Journal of Personality Assessment, 61,277293.

Archer, R. P., & Krishnamurthy, R. (1997). MMPIA and Rorschach indicesrelated to depression and conduct disorder: An evaluation of the incre-mental validity hypothesis. Journal of Personality Assessment, 69, 517 533.

Archer, R. P., Maruish, M., Imhof, E. A., & Piotrowski, C. (1991). Psycho-logical test usage with adolescent clients: 1990 survey ndings. Professional Psychology: Research & Practice, 22, 247252.

Bagby, R. M., Marshall, M. B., Basso, M. R., Nicholson, R. A., Bacchiochi, J.,& Miller, L. S. (2005). Distinguishing bipolar depression, major depression,and schizophrenia with the MMPI2 Clinical and Content scales. Journal of

Personality Assessment, 84, 8995.Ball, J. D., Archer, R. P., Gordon, R. A., & French, J. (1991). Rorschach De-pression indices with children and adolescents: Concurrent validity ndings. Journal of Personality Assessment, 57, 465476.

Ben-Porath, Y. S., Butcher, J. N., & Graham, J. R. (1991). Contribution of theMMPI2 Content scales to the differential diagnosis of Schizophrenia andmajor depression. Psychological Assessment, 3, 634640.

Bilder, R., Goldman, R., Volavka, J., Czobor, P., Hoptman, M., Sheitman, B.,et al. (2002). Neurocognitive effects of clozapine, olanzapine, risperidone,and haloperidol in patients with chronic schizophrenia or schizoaffectivedisorder. American Journal of Psychiatry, 159, 10181028.

Blais, M. A., Hilsenroth, M. J., Castlebury, F., Fowler, C. J., & Baity, M. R.(2001). Predicting DSMIV cluster B personality disorder criteria fromMMPI2 and Rorschach data: A test of incremental validity. Journal of Personality Assessment, 76, 150168.

Butcher, J. N., Dahlstrom, W., Graham, J. R., Tellegen, A., & Kaemmer, B.(1989). MMPI2: Minnesota Multiphasic Personality Inventory2: Manual for administration and scoring . Minneapolis: University of Minnesota Press.

Butcher,J. N.,Graham,J. R.,& Ben-Porath, Y. S. (1995). Methodological prob-lems and issues in MMPI, MMPI2, and MMPIA research. Psychological Assessment, 7, 320329.

Butcher, J. N., Graham, J. R., Williams, C. L., & Ben-Porath, Y. S. (1990). Developmentand use of the MMPI2Content scales . Minneapolis: Universityof Minnesota Press.

Butcher, J. N., Williams, C. L., Graham, J. R., Archer, R. P., Tellegen, A., &Ben-Porath, Y. S.,et al.(1992). MMPIA(Minnesotamultiphasic personality InventoryAdolescent): Manual for administration, scoring, and interpreta-tion. Minneapolis: University of Minnesota Press.

Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluatingnormed and standardized instruments in psychology. Psychological Assess-ment, 6, 284290.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nded.). Hillsdale, NJ: Lawrence Erlbaum Associates.

Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analy-sis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence ErlbaumAssociates.

Dao, T. K., & Prevatt, F. (2006). A psychometric evaluation of the RorschachComprehensive Systems Perceptual Thinking Index. Journal of Personality Assessment, 86, 180189.

Dawes, R. M. (1999). Two methods for studying the incremental validity of aRorschach variable. Psychological Assessment, 11, 297302.

Dealberto, M., McAay, G. J., Seeman, T., & Berkman, L. (1997). Psychotropicdrug use and cognitive decline among older men and women. International Journal of Geriatric Psychiatry, 12, 567574.

Exner, J. E., Jr. (1993). The Rorschach: A Comprehensive System: Vol. 1. Basic foundations (3rd ed.). New York: Wiley.

Exner, J. E., Jr. (2000a). A primer for Rorschach interpretation . Asheville, NC:Rorschach Workshops.

Exner, J. E., Jr. (2000b). 2000 alumni newsletter . Asheville, NC: RorschachWorkshops.

Exner, J. E., Jr. (2003). The Rorschach: A comprehensive system: Vol. 1. Basic foundations (4th ed.). New York: Wiley.

Exner, J. E., Jr., & Weiner, I. B. (1995). The Rorschach: A comprehensivesystem: Vol. 3. Assessment of children and adolescents (2nd ed.). Oxford,England: Wiley.

Exner, J. E., Jr., Weiner, I. B., & PAR Staff. (2001). Rorschach interpretationassistance program: Version 5 for Windows [Computer software]. Odessa,FL: Psychological Assessment Resources.

Ganellen, R. J. (1996a). Comparing the diagnostic efciency of the MMPI,MCMIII, and Rorschach: A review. Journal of Personality Assessment, 37,219243.

Ganellen, R. J. (1996b). Exploring the MMPI-Rorschach relationships. Journalof Personality Assessment, 67, 529542.

Ganellen, R. J. (1996c). Integrating the Rorschach and the MMPI2 in person-ality assessment . Hillsdale, NJ: Lawrence Erlbaum Associates.

Garb, H. N. (1984). The incremental validity of information used in personalityassessment. Clinical Psychology Review, 4, 641655.

Goldberg, L. R. (1965). Diagnosticians vs. diagnostic signs: The diagnosis of psychosis vs. neurosis from the MMPI. Psychological Monographs, 79 (9),(Whole No. 602).

Greenblatt, R. L., & Davis, W. E. (1999). Differential diagnosis of PTSD,schizophrenia, and depression with the MMPI2. Journal of Clinical Psy-chology, 55, 217223.

Greene, R. L. (2000). The MMPI2:An InterpretiveManual (2nded.). NeedhamHeights, MA: Allyn & Bacon.

Greene, R. L., Gwin, R., & Staal, M. (1997). Current status of MMPI2 re-search: A methodologic overview. Journalof Personality Assessment, 68, 20 36.

Greenwald, D. F. (1997). Comparisons between the Rorschach Depression In-dex and depression-related measures in a nonpatient sample. Psychological Reports, 80, 11511154.

Gr onner od, C. (2003). Temporal stability in the Rorschach method: A meta-

analytic review. Journal of Personality Assessment, 80, 272293.Hathaway, S. R., & McKinley, J. C. (1940). A multiphasic personality schedule(Minnesota): I. Construction of the schedule. Journal of Psychology, 10,249254.

Hicklin, J., & Widiger, T.A. (2000). Convergent validity of alternative MMPI2personality disorder scales. Journal of Personality Assessment, 75, 502518.

Hosmer, D. W., & Lemeshow, S. (1989). Applied logistic regression . New York:Wiley.

Hunsley, J., & Meyer, G. J. (2003). The incremental validity of psycholog-ical testing assessment: Conceptual, methodological, and statistical issues. Psychological Assessment, 15, 446455.

Kostlan, A. (1954). A method for the empirical study of psychodiagnosis. Jour-nal of Consulting Psychology, 18, 8388.

8/13/2019 31561278

9/10


Kumar, R., & Khess, C. R. J. (2005). Diagnostic efciency of SchizophreniaIndex and Perceptual-Thinking Index in schizophrenia and mania. Journal of Projective Psychology & Mental Health, 12, 115122.

McDowell, C., & Acklin, M. W. (1996). Standardizing procedures for calculat-ing Rorschach interrater reliability: Conceptual and empirical foundations. Journal of Personality Assessment, 66, 308320.

Meyer, G. J. (1996). The Rorschach and MMPI: Toward a more scienticallydifferentiated understanding of cross-method assessment. Journal of Person-

ality Assessment, 67, 558578.Meyer, G. J. (1997). On the integration of personality assessment methods: TheRorschach and the MMPI2. Journal of Personality Assessment, 68, 297 330.

Meyer, G. J. (2000a). Incremental validity of the Rorschach prognostic rat-ing scale over the MMPI ego strength scale and IQ. Journal of Personality Assessment, 74, 356370.

Meyer, G. J. (2000b). On the science of Rorschach research. Journal of Person-ality Assessment, 75, 4681.

Meyer, G. J.,& Resnick,G. D.(1996, July). Assessing ego impairment: Do scor-ing procedures make a difference? Paper presented at the 15th InternationalResearch Conference, Boston.

Meyer, G. J., Riethmiller, R. J., Brooks, R. D., Benoit, W. A., & Handler, L.(2000). A replication of Rorschach and MMPI2 convergent validity. Journalof Personality Assessment, 74, 175215.

Michael, B., Spitzer, R. L., Gibbon, M., & Williams, J. B. (1996). Structured Clinical Interview for DSMIV Axis I Disorders: Clinician Version (SCIDCV). Washington, DC: American Psychiatric Press.

Michael, B., Spitzer, R. L., Gibbon, M., & Williams, J. B. (1997). Structured ClinicalInterview for DSMIVPersonalityDisorders (SCIDII) . Washington,DC: American Psychiatric Press.

Munley, P. H. (2002). Comparability of MMPI2 scales proles over time. Journal of Personality Assessment, 78, 145160.

Munley, P. H., Busby, R. M., & Jaynes, G. (1997). MMPI2 ndings inschizophrenia and depression. Psychological Assessment, 9, 508511.

Nagelkerke, N. J. (1991). A note on the general denition of the coefcient of determination. Biometrika, 78, 691692.

Ritsher, J. B. (2004). Association of Rorschach and MMPI psychosis indicatorsand schizophrenia spectrum diagnoses in a Russian clinical sample. Journalof Personality Assessment, 83, 4663.

Rorschach, H. (1942). Psychodiagnostik (Hans Huber, Trans.). Bern: Switzer-land: Verlag. (Original work published in 1921.)

Rossi, G., Van den Brande, I., Tobac, A., Sloore, H., & Hauben, C. (2003).

Convergent validity of the MCMIIII personality disorder scales and theMMPI2 scales. Journal of Personality Disorders, 17, 330340.Smith, S. R., Baity, M. R., Knowles, E. S., & Hilsenroth, M. J. (2001). Assess-

ment of disordered thinking in children and adolescents: The Rorschachperceptual-thinking index. Journal of Personality Assessment, 77, 447 463.

Strassberg, D. S., & Russell, S.W. (2000). MMPI2 content scalevalidity withina sample of chronic pain patients. Journal of Psychopathology & Behavioral Assessment, 22, 4760.

Streiner, D. L. (2003). Diagnosing tests: Using and misusing diagnostic andscreening tests. Journal of Personality Assessment, 81, 209219.

Tate, R. (1998). An introduction to modeling outcomes in the behavioral and social sciences . Edina, MN: Burgess International Group.

Vendrig, A. A., Derksen, J. J. L, & de Mey, H. R. (2000). MMPI2 Person-ality Psychopathology Five (PSY5) and prediction of treatment outcomefor patients with chronic back pain. Journal of Personality Assessment, 74,423438.

Watkins, C. E., Campbell, V. L., Nieberding, R., & Hallmark, R. (1995). Con-temporary practice of psychological assessment by clinical psychologists. Professional Psychology: Research & Practice, 26, 5460.

Wise, E. A. (2001). The comparative validity of MCMIIII and MMPI2 per-sonality disorder scales with forensic examinees. Journal of Personality Dis-orders, 15, 275279.

Wiggins, J. S. (1973). Personality and prediction: Principles of personalityassessment . Menlo Park, CA: Addison Wesley.

8/13/2019 31561278

10/10

31561278

Documents