factor analysis - university of...
TRANSCRIPT
FactorAnalysis
TheMeasurementModel
FactorAnalysis:TheMeasurementModel
D1 D8D7D6
D5D4D3D2
F1 F2
Di = !Fi + ei
Examplewith2factorsand8observedvariables
!
""""""""""#
Di,1
Di,2
Di,3
Di,4
Di,5
Di,6
Di,7
Di,8
$
%%%%%%%%%%&
=
!
""""""""""#
!11 !12
!21 !22
!31 !32
!41 !42
!51 !52
!61 !62
!71 !27
!81 !82
$
%%%%%%%%%%&
'Fi,1
Fi,2
(+
!
""""""""""#
ei,1
ei,2
ei,3
ei,4
ei,5
ei,6
ei,7
ei,8
$
%%%%%%%%%%&
Di = !Fi + ei
Di,1 = !11Fi,1 + !12Fi,2 + ei,1
Di,2 = !21Fi,1 + !22Fi,2 + ei,2 etc.
Thelambdavaluesarecalledfactorloadings.
Terminology
• Thelambdavaluesarecalledfactorloadings.• F1andF2aresometimescalledcommonfactors,becausetheyinfluencealltheobservedvariables.
• e1,…,e8sometimescalleduniquefactors,becauseeachoneinfluencesonlyasingleobservedvariable.
Di,1 = !11Fi,1 + !12Fi,2 + ei,1
Di,2 = !21Fi,1 + !22Fi,2 + ei,2 etc.
FactorAnalysiscanbe
• Exploratory:Thegoalistodescribeandsummarizethedatabyexplainingalargenumberofobservedvariablesintermsofasmallernumberoflatentvariables(factors).Thefactorsarethereasontheobservablevariableshavethecorrelationstheydo.
• Confirmatory:Statisticalestimationandtestingasusual
PartOne:Unconstrained(Exploratory)FactorAnalysis
F and e independent multivariate normal
V (D) = ! = "#"! + $
D = !F + e
Main interest is in the number of factors and the factor loadings !.
V (F) = !V (e) = " (usually diagonal)
ARe‐parameterization
! = "#"! + $
Write ! = SS! (square root matrix), so
!"!! = !SS!!!
= (!S)I(S!!!)= (!S)I(!S)!
= !2I!!2
Parametersarenotidentifiable
• Phicouldbeanysymmetricpositivedefinitematrixandthiswouldwork.
• Infinitelymany(Lambda,Phi)pairsgivethesameSigma,andhencethesamedistributionofthedata.
• Phiiscompletelyarbitrary(soisLambda)• Thisshowsthattheparametersofthegeneralmeasurementmodelarenotidentifiablewithoutsomerestrictionsonthepossiblevaluesoftheparametermatrices.
• Noticethatthegeneralunrestrictedmodelcouldbeveryclosetothetruth.Buttheparameterscannotbeestimatedsuccessfully,period.
! = "#"! + $ !"!! = !2I!!2
Restrictthemodel
• SetPhi=theidentity,soV(F)=I• Justifythisonthegroundsofsimplicity.
• Saythefactorsare“orthogonal”(atrightangles,uncorrelated).
!"!! = !2I!!2
Standardizetheobservedvariables
• Forj=1,…,kandindependentlyfori=1,…,n
•
• Assumeeachobservedvariablehasvarianceoneaswellasmeanzero.
• Sigmaisnowacorrelationmatrix.
Zij =Dij !Dj
sj
RevisedExploratoryFactorAnalysisModel
V (F) = IV (e) = ! (usually diagonal)
F and e independent multivariate normal
V (D) = ! = ""! + #
Z = !F + e
Meaningofthefactorloadings
• λijisthecorrelationbetweenvariableiandfactorj.
• Squareofλijisthereliabilityofvariableiasameasureoffactorj.
Corr(X6, F2) = Cov(X6, F2) = E(X6F2)= E ((!61F1 + !62F2)F2)= !61E(F1F2) + !62E(F 2
2 )= !61E(F1)E(F2) + !62V ar(F2)= !62
Communality
• istheproportionofvarianceinvariableithatcomesfromthecommonfactors.
• Itiscalledthecommunalityofvariablei.
• Thecommunalitycannotexceedone.
• Peculiar?
V ar(Xi) =p!
j=1
!ijFj + ei
=p!
j=1
!2ijV ar(Fj) + V ar(ei)
=p!
j=1
!2ij + "i
p!
j=1
!2ij
V ar(!i) = 1!!p
j=1 "2ij
Ifwecouldestimatethefactorloadings
• Wecouldestimatethecorrelationofeachobservablevariablewitheachfactor.
• Wecouldeasilyestimatereliabilities.• Wecouldestimatehowmuchofthevarianceineachobservablevariablecomesfromeachfactor.
• Thiscouldrevealwhattheunderlyingfactorsare,andwhattheymean.
• Numberofcommonfactorscanbeveryimportanttoo.
Examples• Amajorstudyofhowpeopledescribeobjects(using7‐pointscalesfromUglytoBeautiful,StrongtoWeak,FasttoSlowetc.revealed3factorsofconnotativemeaning:– Evaluation– Potency– Activity
• Factoranalysisofalargecollectionofpersonalityscalesrevealed2majorfactors:– Neuroticism– Extraversion
• Yetanotherstudyledto16personalityfactors,thebasisofthewidelyused16pftest.
RotationMatrices
• Have a co-ordinate system in terms of!i ,!j orthonormal vectors
• Roatate the axies through an angle !.
!
i
i!
j! j
i! = i cos ! + j sin !
j! = !i sin ! + j cos !
i! = (cos !)i + (sin !)jj! = (! sin !)i + (cos !)j
!i!
j!
"=
!cos ! sin !
! sin ! cos !
" !ij
"= R
!ij
"
RR! =!
cos ! sin !! sin ! cos !
" !cos ! ! sin !sin ! cos !
"
=!
cos2 ! + sin2 ! ! cos ! sin ! + sin ! cos !! sin ! cos ! + cos ! sin ! sin2 ! + cos2 !
"
=!
1 00 1
"= I
Thetransposerotatedtheaxiesbackthroughanangleofminustheta.
InGeneral
• ApxpmatrixRsatisfyingR‐inverse=R‐transposeiscalledanorthogonalmatrix.
• Geometrically,pre‐multiplicationbyanorthogonalmatrixcorrespondstoarotationinp‐dimensionalspace.
• IfyouthinkofasetoffactorsFasasetofaxies(underlyingdimensions),thenRFisarotationofthefactors.
• Callitanorthoganalrotation,becausethefactorsremainuncorrelated(atrightangles).
AnotherSourceofnon‐identifiability
! = ""! + #= "RR!"! + #= ("R)(R!"!) + #= ("R)("R)! + #= "2"!
2 + #
InfinitelymanyrotationmatricesproducethesameSigma.
NewModel
Z = !2F + e= (!R)F + e= !(RF) + e= !F! + e
F! is a set of rotated factors.
ASolution
• Placesomerestrictionsonthefactorloadings,sothattheonlyrotationmatrixthatpreservestherestrictionsistheidentitymatrix.Forexample,λij=0forj>i
• Thereareothersetsofrestrictionsthatwork.• Generally,theyresultinasetoffactorloadingsthatareimpossibletointerpret.Don’tworryaboutit.
• Estimatetheloadingsbymaximumlikelihood.Othermethodsarepossiblebutusedmuchlessthaninthepast.
• All(orthoganal)rotationsresultinthesamevalueofthelikelihoodfunction(themaximumisnotunique).
• Rotatethefactors(thatis,multiplytheloadingsbyarotationmatrix)soastoachieveasimplepatternthatiseasytointerpret.
Rotatethefactorsolution
• Rotatethefactorstoachieveasimplepatternthatiseasytointerpret.
• Therearevariouscriteria.Theyarealliterative,takinganumberofstepstoapproachsomecriterion.
• Themostpopularrotationmethodisvarimaxrotation.• Varimaxrotationtriestomaximizethe(squared)loadingofeachobservablevariablewithjustoneunderlyingfactor.
• Sotypicallyeachvariablehasabigloadingon(correlationwith)oneofthefactors,andsmallloadingsontherest.
• Lookattheloadingsanddecidewhatthefactorsmean(namethefactors).
AWarning
• Whenanon‐statisticianclaimstohavedonea“factoranalysis,”askwhatkind.
• Usuallyitwasaprincipalcomponentsanalysis.• Principalcomponentsarelinearcombinationsoftheobservedvariables.Theycomefromtheobservedvariablesbydirectcalculation.
• Intruefactoranalysis,it’stheobservedvariablesthatarisefromthefactors.
• Soprincipalcomponentsanalysisiskindoflikebackwardsfactoranalysis,thoughthespiritissimilar.
• Mostfactoranalysis(SAS,SPSS,etc.)doesprincipalcomponentsanalysisbydefault.