lect 03_en

8/3/2019 Lect 03_EN

http://slidepdf.com/reader/full/lect-03en 1/21

Research Methodology: Tools

Applied Data Analysis (with SPSS)

Lecture 03: Factor Analysis

March 2011

Prof. Dr. Jürg Schwarz [email protected]

MSc Business Administration

Page 2

Contents

Aims ___________________________________________________________________________________________________ 5 Introduction _____________________________________________________________________________________________ 6 Outline _________________________________________________________________________________________________ 8 Concepts of Factor Analysis ______________________________________________________________________________ 12 Factor Analysis with SPSS: A detailed example ______________________________________________________________ 16

8/3/2019 Lect 03_EN


Page 3

Table of contents

Aims ___________________________________________________________________________________________________ 5 Aims of the lecture .................................................................................................................................................................................................5

Introduction _____________________________________________________________________________________________ 6 Example of a construct...........................................................................................................................................................................................6

Outline _________________________________________________________________________________________________ 8 Concepts of Factor Analysis ______________________________________________________________________________ 12 Key steps involved in using factor analysis...........................................................................................................................................................12 Sample size: Rules of thumb................................................................................................................................................................................13 Principal component analysis vs. principal axis factoring......................................................................................................................................14

Types of factoring ......................................................................................................................................................................................................................14 Problematic issues ...............................................................................................................................................................................................15

Factor Analysis with SPSS: A detailed example ______________________________________________________________ 16 Item battery "motivation" from visitors’ questionnaire of Documenta.....................................................................................................................16 Before first step: Conduct basic statistics.............................................................................................................................................................17 First step: Correlation matrix of the variables .......................................................................................................................................................20

Factor analysis with SPSS: <Analyze><Dimension Reduction><Factor…>.............................................................................................................................22 Inverse of correlation Matrix.......................................................................................................................................................................................................23 Bartlett’s Test .............................................................................................................................................................................................................................24 Kaiser Meyer Olkin (KMO).........................................................................................................................................................................................................25

Page 4

Second step: Extraction of factors........................................................................................................................................................................27 Principal components analysis (PCA): Some facts....................................................................................................................................................................28 Principal axis factoring (PAF): Some facts ................................................................................................................................................................................29 Example Documenta: Extraction of factors................................................................................................................................................................................30 Graphical interpretation..............................................................................................................................................................................................................31 Loading plot in SPSS.................................................................................................................................................................................................................32 A loading must satisfy certain criteria ........................................................................................................................................................................................33 Communality ("Gemeinsamkeit")...............................................................................................................................................................................................33

Third step: Criteria for determining the number of factors.....................................................................................................................................35 Scree plot ...................................................................................................................................................................................................................................36 Kaiser criterion (Eigenvalues > 1)..............................................................................................................................................................................................38

Fourth step: Interpretation of factors.....................................................................................................................................................................39 Rotation......................................................................................................................................................................................................................................39 Content-related interpretation of factors ....................................................................................................................................................................................41

Fifth step: Calculation of Factor Values ................................................................................................................................................................42 SPSS..........................................................................................................................................................................................................................................42

8/3/2019 Lect 03_EN


Page 5

Aims

Aims of the lecture

You know the term "construct".

You know the key steps involved in using factor analysis.

You can conduct a factor analysis with SPSS

(Extraction method: Principal components analysis)

In particular, you know how to …

◦ interpret the correlation matrix

◦ derive the "right" number of factors (Scree plot, Kaiser criterion)

◦ use varimax rotation to better interpret the factor solution

◦ interpret the factors with regard to their meaning

Page 6

Introduction

Example of a construct

Item battery from visitor's questionnaire of "Documenta" (German exhibition of contemporary art)

8/3/2019 Lect 03_EN


Page 7

Question

Is there a structure in the item battery of the Documenta questionnaire?

Are there any sub-dimensions (also called factors)?

Conduct factor analysis

Items Factors

Interested in information

Interested in entertainment

I pay close attention to the documenta-flair

I'm looking specifically for current trends …

I want to experience something …

I would like to experience a cultural event

I want to have some fun

I want to continue my education in arts

I would like to see an overview of …

I visit the DOCUMENTA for professional …

Page 8

Outline

Constructs in social science

A construct is a theoretical approach to express what cannot be directly observed.

Examples: Motivation, intelligence, anxiety.

Technically a construct is an item battery.

Dimension (also called factor) is a certain sub-structure of a construct.

In the Documenta sample, the purpose item battery was used to operationalise and analyse

"motivation" as the psychological construct. It was measured using 8 items (indicators).

Interested in information

Interested in entertainment

I pay close attention to the documenta-flair

I'm looking specifically for current trends …

I want to experience something …

I would like to experience a cultu ral event

I want to have some fun

I want to continue my education in arts

I would like to see an overview of …

I visit the DOCUMENTA for professional …

Dimensions (factors) of motivationConstruct

"motivation"

factor <–> technical term

dimension <–> term of theory

item <–> in questionnaire

indicator <–> term of theory

8/3/2019 Lect 03_EN


Page 9

How to discover dimensions (factors)?

Conduct factor analysis!

Factor analysis is based on how closely various items are related and how they form factors.

Each factor (dimension) represents several different items.Factors turn out to be more efficient than individual items at representing outcomes.

The goal is to represent items that are related to one another by a more general term.

Example

"Interested in entertainment" is more general and comprises

"I want to experience something aesthetically beautiful"

"I would like to experience a cultural event"

"I want to have some fun"

"I pay close attention to the documenta-flair"

The general term reflects the underlying content.

Note: Never forget theoretical and empirical facts while conducting factor analysis.

Page 10

Basic idea of factor analysis

Assumption: Some variables are related

Dataset of the Documenta questionnaire

I want to experience something aesthetically beautiful => variable v01

I would like to experience a cultural event => variable v02

8/3/2019 Lect 03_EN


Page 11

Three possible causes for the correlation of v01 and v02

◦ Variable v01 influences variable v02

◦ Variable v02 influences variable v01

◦

Both variables are influenced by a factor

Table of correlations

v01 v02 v03 v04 v05 v06 v07 v08

v01 1

v02 0.20 1

v03 0.35 0.23 1

v04 0.30 0.29 0.25 1

v05 -0.10 0.00 -0.22 0.00 1

v06 0.15 0.22 0.04 0.07 0.35 1

v07 -0.05 0.15 -0.13 -0.06 0.35 0.32 1

v08 -0.18 -0.10 -0.22 -0.08 0.39 0.20 0.11 1

Factor "Interested in information"

Factor "Interested in entertainment"

Page 12

Concepts of Factor Analysis

Key steps involved in using factor analysis

1. Choice of variables

◦ Include only variables that are based on theory

◦ Not too small a sample (see also rule of thumbs)

2. Extraction of factors

◦ Calculation of correlation matrix, inverse correlation matrix

◦ Two different concepts: Principal component analysis vs. principal axis factoring

3. Criteria for determining the number of factors

◦ Tools: Eigenvalue, scree plot, several rules of thumb

4. Interpretation of factors

◦ Interpretation of factor loading, use of rotated factor matrix

5. Calculation of factor values

◦ Done by SPSS (three different concepts: Regression, Bartlett, Anderson-Rubin)

8/3/2019 Lect 03_EN


Page 13

Sample size: Rules of thumb

How large should the sample be?

There is no scientific answer.

Below are some rules of thumb, in descending order of popularity.

These are not mutually exclusive: Some researchers use both, STV and the Rule of 200.

1. Rule of 10: There should be at least 10 subjects for each item in the construct being used.

2. STV ratio: The subjects-to-variables ratio should be no lower than 5.

3. Rule of 150: There should be at least 150 - 300 cases, around 150 when there are a fewhighly correlated variables.

4. Rule of 200: There should be at least 200 cases, regardless of STV.

There is nearly universal agreement that sample size should be at least 50.

Page 14

Principal component analysis vs. principal axis factoring

Types of factoring

Principal component analysis (PCA) ("Hauptkomponentenanalyse")

Goal: Reproduce the data structure using the smallest number of factors

Features:

◦ Interpretation is difficult

◦ No causal relationship between factors and variables

◦ Factors are "general terms", called components

Note: PCA is default in SPSS

Principal axis factoring (PAF) ("Hauptachsenanalyse")

Goal: Determine the cause of the correlation structure

Features:

◦ Factors cause the correlations between variables

◦ Causal interpretation of the factors

In this lecture the words "components" and "factors" are used interchangeably

because most of the rules on inter retin PCA and PAF are the same.

8/3/2019 Lect 03_EN


Page 15

Problematic issues

Many decisions on extracting and interpreting the factors are subjective.

The same data set may produce different results, depending on the "decision path".

Even though variables must be interval-scaled, variables with a low scale level are often

included in real life, and this leads to wrong conclusions.

Missing value problem

In item batteries, there are often many missing values

Depending on how the missing values are treated,

the results may differ in terms of the

◦ Number of factors

◦

Interpretation of factors

There is no single solution to the problem of missing values.

Depending on the underlying data and context, another approach may be needed.

Page 16

Factor Analysis with SPSS: A detailed example

Item battery "motivation" from visitors’ questionnaire of Documenta

The facts

Sample of Documenta visitors (n = 775)

Item battery with eight items

(Note: Some items were "rotated")

Data set: documenta.savSyntax: documenta.sps

8/3/2019 Lect 03_EN


Page 17

Before first step: Conduct basic statistics

Missing

◦ No specific problem

Mean / median

◦ Differences between the mean and the median show that some distributionsmight not be symmetric

Std. Deviation

◦ No specific problem

Minimum / maximum

◦ Values between 1 and 5 show that the full range of the scales was used.

Page 18

Histogram v01 to v04

v01: Equally distributed. Should not be a problem.

v02: "I would like to experience a cultural event" is skewed left.

Most answers concentrate around "Strongly agree". This might be because the visitors

go the exhibition in order to have a cultural experience. Should not be a problem.


v04: "I pay close attention to the documenta-flair" has a peak at "Strongly disagree".

A reasonable part of the visitors seems to dislike the term "flair". Should not be a problem.

v01 v02 v03 v04

8/3/2019 Lect 03_EN


Page 19

Histogram v05 to v06


v06: Almost normally distributed. Should not be a problem.

v07: "I want to continue my education in art" is skewed left.Most of the answers are found near "Strongly agree". This might be because the visitors

show "social desirability" in their answers. Should not be a problem.

v08: "I visit the DOCUMENTA for professional reasons" has a peak at "Strongly disagree".

Most of the visitors are concentrated around "Strongly disagree". This might be because

most of the visitors are attending as private persons. This item should be eliminated.

v05 v06 v07 v08

Page 20

First step: Correlation matrix of the variables

Correlations

v01 v02 v03 v04 v05 v06 v07

Correlation Coefficient 1.000 .199 .345 .301 -.097 .146 -.051v01

Sig. (2-tailed) . .000 .000 .000 .007 .000 .156

Correlation Coefficient .199 1.000 .229 .293 .000 .222 .154v02

Sig. (2-tailed) .000 . .000 .000 .997 .000 .000

Correlation Coefficient .345 .229 1.000 .250 -.218 .043 -.125v03

Sig. (2-tailed) .000 .000 . .000 .000 .237 .001

Correlation Coefficient .301 .293 .250 1.000 .002 .069 -.058v04

Sig. (2-tailed) .000 .000 .000 . .966 .056 .109

Correlation Coefficient -.097 .000 -.218 .002 1.000 .352 .351v05

Sig. (2-tailed) .007 .997 .000 .966 . .000 .000

Correlation Coefficient .146 .222 .043 .069 .352 1.000 .322v06

Sig. (2-tailed) .000 .000 .237 .056 .000 . .000

Correlation Coefficient -.051 .154 -.125 -.058 .351 .322 1.000

Spearman's rho

v07

Sig. (2-tailed) .156 .000 .001 .109 .000 .000 .

There is evidence of two factors.

Attention: The variables in this example are nicely grouped in an obvious factor structure.

Normally this is not the case!

8/3/2019 Lect 03_EN


Page 21

What is important when assessing the correlation matrix?

Significance level of correlations

◦ The level of significance of the correlations indicates the likelihoodthat the actual correlations are a value other than zero.

◦

Depending on the sample size and intention of the analysis,the a priori level of significance will be at 1% or 5%.

In rare cases, particularly for large samples, the a priori level of significance may be 0.1 %.

Value of the correlation coefficients

◦ Factor analysis becomes problematic if there are many small correlation coefficients.In this case, the data structure is too heterogeneous.

◦ Best would be if there are clusters of highly correlated variables that are separated.These clusters are indication of an underlying factor structure.

Page 22

Further properties of the correlation structure

Using information only from the correlation matrix is insufficient because of a …

◦ lack of indication of causality

◦ lack of indication of how many factors are causing correlation

Use more tests!

Factor analysis with SPSS: <Analyze><Dimension Reduction><Factor…>

8/3/2019 Lect 03_EN


Page 23

Inverse of correlation Matrix

A correlation structure is suitable for a factor analysis only if the inverse forms a diagonal matrix.

The matrix is diagonal when the non-diagonal elements are close to zero as possible.

There is no generally accepted rule.

The inverse of the correlation matrix is essentially a visual aid for testing suitability.

In the Documenta example, the non-diagonal elements are significantly smaller

=> Correlation structure is well suited for a factor analysis.

Diagonal

Page 24

Bartlett’s Test

Null hypothesis H0:

The random sample comes from a universe in which all variables

are completely uncorrelated.

Prerequisite: The variables are normally distributed.

In the case of the Documenta data, the test statistic is very high (603.735),

and accordingly the null hypothesis may be rejected (Sig. = .000).

The variables are not completely uncorrelated.

Note: The statement "The variables are correlated" is incorrect.

The alternative hypothesis cannot be stated.

8/3/2019 Lect 03_EN


Page 25

Kaiser Meyer Olkin (KMO)

Kaiser, Meyer and Olkin have developed the "measure of sampling adequacy" (MSA) test,

which has become the standard test procedure for the factor analysis.

The Kaiser-Meyer-Olkin measure of sampling adequacy tests whether or not the partial

correlations among variables are small.

The MSA criterion indicates the degree to which the variables are related,and it thus helps in evaluating if using a factor analysis makes sense.

Page 26

KMO-Value

0.00 to 0.49 unacceptable

0.50 to 0.59 miserable

0.60 to 0.69 mediocre

0.70 to 0.79 middling

0.80 to 0.89 meritorious

0.90 to 1.00 marvellous

As a rule of thumb, KMO should be 0.60 or higher in order to proceed with a factor analysis.

Kaiser* suggests 0.50 as a cut-off value, and a desirable value of 0.8 or higher.

*Kaiser H. (1970) "A second generation little jiffy," Psychometrika, Springer, vol. 35(4), pages 401-415, December.

8/3/2019 Lect 03_EN


Page 27

Second step: Extraction of factors

Page 28

Principal components analysis (PCA): Some facts

◦ As PCA is more common than PFA, the simple term "factor analysis" also may refer to PCA.

◦ PCA is used when the research purpose is data reduction.

◦ Analyzes a correlation matrix in which the diagonal contains 1's.This is not equivalent to analyzing the covariance matrix.

◦ Factors, properly called components, reflect the common variance of variables plus theunique variance. That is, manifest variables may be conceptualized as reflecting a combina-tion of total variance explained by the components, plus error variance not explained by thecomponents.

◦ Components seek to reproduce the total variable variance as well as the correlations,That is, PCA accounts for the total variance of the variables.

◦ PCA is thus a variance-focused approach.

◦ For the first component, PCA creates a linear equation which extracts the maximum total vari-ance from the variables; for the second component PCA removes the variance explained bythe first component and creates a second linear equation which extracts the maximum re-maining variance; etc., continuing until the components can explain all the common and

unique variance in a set of variables.◦ Adding variables to the model will change the factor loadings.

http://faculty.chass.ncsu.edu/garson/PA765/factor.htm (Date of access: March, 2011)

8/3/2019 Lect 03_EN


Page 29

Principal axis factoring (PAF): Some facts

◦ Analyzes a correlation matrix in which the diagonal contains the communalities. This isequivalent to analyzing the covariance matrix.

◦ PFA is used when the research purpose is causal modelling.

◦ Factors reflect the common variance of variables, excluding unique variance. That is, mani-fest variables may be conceptualized as reflecting a combination of common variance ex-

plained by the factors, plus unique variance not explained by the factors.◦ Factors seek to reproduce the correlations of the variables. That is, PFA accounts for the co-

variation among the variables.

◦ PFA is thus a correlation-focused approach.

◦ PFA seeks the least number of factors which can account for the common variance shared bya set of variables. For the first factor, PFA creates a linear equation which extracts the maxi-mum common variance from the variables; for the second component PFA removes thecommon variance explained by the first component and creates a second linear equationwhich extracts the maximum remaining variance; etc., continuing until the factors can explain

all the common variance in a set of variables.

◦ Normally, factors are orthogonal to (uncorrelated with) one another.

◦ In principle it is possible to add variables without affecting the factor loadings.

http://faculty.chass.ncsu.edu/garson/PA765/factor.htm (Date of access: March, 2011)

Page 30

Example Documenta: Extraction of factors

Factor loading

8/3/2019 Lect 03_EN


Page 31

Graphical interpretation

Each variable can be described as a vector in a system of coordinates,

which are formed by its factors.

-0.5

0.0

0.5

1.0

-0.5 0.0 0.5 1.0

v05

v02

Factor 1

Factor 2

v04v01

v03

v07 v06

:

Variables v01 to v07 can bedescribed by factors 1 and 2.

The factor loading is the cosine ofthe angle between the factor and

one variable.

Example: v06

ARCCOS(0.223) = 77.11v06 forms an angle of 77.11° withthe first factor.

Page 32

Loading plot in SPSS

8/3/2019 Lect 03_EN


Page 33

A loading must satisfy certain criteria

A factor can be interpreted if at least 4 variables have a loading of more than 0.60.

The variables with the highest loading are the "marker variables".

A factor can be interpreted if at least 10 variables have a loading of more than 0.40.

The variables with the highest loading are the "marker variables".

If fewer than 10 variables have a loading of more than 0.40 and thesample size is less than 300, the loading structure is likely to be random.

Normative: A factor loading of less than 0.2 cannot be considered

=> such items are omitted and the analysis must be recalculated.

Communality ("Gemeinsamkeit")Variables can normally not be explained completely by means of factors.

The number of explanatory factors is usually significantly smaller than the number of variables.

Communality is the part of a variable’s total variance that can be explained by the factors.

Communality indicates the extent to which this variable can be explained by the factors.

Page 34

Documenta Example

Example variable v01

Communality after extraction 0.479 => 47.9% of the variance of v01 is explained byFactors 1 and 2.

0.479 = 0.6912 + 0.0392

47.9% = 47.8% + 0.1%

8/3/2019 Lect 03_EN


Page 35

Third step: Criteria for determining the number of factors

There is no strictly mathematical criterion.

Common sense: Limit the number of factors to those whose meaning is understandable.

Scree test: The scree test plots the components as the X Axis and the corresponding eigenval-

ues as the Y Axis. The scree test requires all components after the elbow to be dropped.

Kaiser criterion: The Kaiser rule is to drop all components with eigenvalues under 1.0.

The Kaiser criterion is the default in SPSS and most computer programs, but it is not recom-

mended when used as the sole cut-off criterion for estimating the number of factors.

Variance explained criterion: Some researchers simply use the rule of keeping enough factors

to account for 90% (sometimes 80%) of the variation.

Page 36

Scree plot

Elbow criterion:

◦ When the factors correspond to error or random numbers, the slope will be flat.

◦ Keep the number of factors above the elbow.

The elbow of the Documenta scree plot occurs at 3, so keep 2 factors.

= Factor Scree ("Geröll")

elbow

flat slope

h t t p : / / e n . w

i k

i p e d i a . o r g / w i k i / S c r e e

8/3/2019 Lect 03_EN


Page 37

Typical argument

"I have chosen 2 factors because it is

consistent with theory and is at least notinconsistent with the scree plot."

Later on you will have to complete

scree test with Kaiser criterion

(Eigenvalue > 1) test and other criteria.

Page 38

Kaiser criterion (Eigenvalues > 1)

An eigenvalue indicates how much of the total variance of all variables is covered by the factor.

The Kaiser rule is to drop all components with eigenvalues under 1.0.

The Kaiser criterion is the default in SPSS and most computer programs but is not recom-

mended when used as the only cut-off criterion for estimating the number of factors.

Sum of eigenvalues = Maximum number of factors

Documenta example: At most 2 factors <=> Kaiser criterion (eigenvalues > 1)

8/3/2019 Lect 03_EN


Page 39

Fourth step: Interpretation of factors

Rotation

Rotation makes the output more readable.

Varimax rotation maintains the independence of the factors.

Factor 1

Factor 2Factor 2Factor 1

Page 40

8/3/2019 Lect 03_EN


Page 41

Content-related interpretation of factors

Factor 1 Factor 2

v01 I want to experience something aesthetically beautiful +

v02 I would like to experience a cultural event +

v03 I want to have some fun +

v04 I pay close attention to the documenta-flair +v05 I'm looking specifically for current trends in visual art +

v06 I want to continue my education in arts +

v07 I would like to see an overview of contemporary art +

What is the subject of Factor 1?

Hints from the item text: "experience", "have fun", "Documenta -flair" => Entertainment

What is the subject of Factor 2?

Hints from the item text: "current trend", "education", "overview" => Information

There are also hints from the context of the study.

Page 42

Fifth step: Calculation of Factor Values

SPSS

Regression Method: The resulting scores have a mean of 0 and a variance equal to the squared

multiple correlation between the estimated factor scores and the true factor values. The scores

may be correlated even when factors are orthogonal.

Bartlett Scores: The resulting scores have a mean of 0. The sum of squares of the unique fac-tors over the range of variables is minimized.

Anderson-Rubin Method: A modification of the Bartlett method which ensures orthogonality of

lect 03_en

Documents