Subject: Statistics
Paper: Multivariate Analysis
Module: Factor Analysis : Estimation Techniques 1
1 / 17
Development Team
Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta
Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta
Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad
Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta
2 / 17
Development Team
Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta
Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta
Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad
Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta
2 / 17
Development Team
Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta
Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta
Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad
Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta
2 / 17
Development Team
Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta
Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta
Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad
Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta
2 / 17
Factor Model
I X= (X1, X2, . . . , Xm)0 ←observed random vector.
I F= (F1, F2, . . . , Fp)0 ← set of common factors.
I = (1, 2, . . . , m)0 ← the specific factors (or errors) Model : X=µ+LF+
I p usually much smaller than m.
I Assume thatE(F) =0 andCov(F) =Ip
I Also E() =0 and Cov() = Ψ =diag((ψjj))
I Cov(F,) =0.
3 / 17
Factor Model
I X= (X1, X2, . . . , Xm)0 ←observed random vector.
I F= (F1, F2, . . . , Fp)0 ← set of common factors.
I = (1, 2, . . . , m)0 ← the specific factors (or errors) Model : X=µ+LF+
I p usually much smaller than m.
I Assume thatE(F) =0 andCov(F) =Ip
I Also E() =0 and Cov() = Ψ =diag((ψjj))
I Cov(F,) =0.
3 / 17
Factor Model
I X= (X1, X2, . . . , Xm)0 ←observed random vector.
I F= (F1, F2, . . . , Fp)0 ← set of common factors.
I = (1, 2, . . . , m)0 ← the specific factors (or errors) Model : X=µ+LF+
I p usually much smaller than m.
I Assume thatE(F) =0 andCov(F) =Ip
I Also E() =0 and Cov() = Ψ =diag((ψjj))
I Cov(F,) =0.
3 / 17
Factor Model
I X= (X1, X2, . . . , Xm)0 ←observed random vector.
I F= (F1, F2, . . . , Fp)0 ← set of common factors.
I = (1, 2, . . . , m)0 ← the specific factors (or errors) Model : X=µ+LF+
I p usually much smaller than m.
I Assume thatE(F) =0 andCov(F) =Ip
I Also E() =0 and Cov() = Ψ =diag((ψjj))
I Cov(F,) =0.
3 / 17
Factor Model
I X= (X1, X2, . . . , Xm)0 ←observed random vector.
I F= (F1, F2, . . . , Fp)0 ← set of common factors.
I = (1, 2, . . . , m)0 ← the specific factors (or errors) Model : X=µ+LF+
I p usually much smaller than m.
I Assume thatE(F) =0 andCov(F) =Ip
I Also E() =0 and Cov() = Ψ =diag((ψjj))
I Cov(F,) =0.
3 / 17
Factor Model
I X= (X1, X2, . . . , Xm)0 ←observed random vector.
I F= (F1, F2, . . . , Fp)0 ← set of common factors.
I = (1, 2, . . . , m)0 ← the specific factors (or errors) Model : X=µ+LF+
I p usually much smaller than m.
I Assume thatE(F) =0 andCov(F) =Ip
I Also E() =0 and Cov() = Ψ =diag((ψjj))
I Cov(F,) =0.
3 / 17
Factor Model
I X= (X1, X2, . . . , Xm)0 ←observed random vector.
I F= (F1, F2, . . . , Fp)0 ← set of common factors.
I = (1, 2, . . . , m)0 ← the specific factors (or errors) Model : X=µ+LF+
I p usually much smaller than m.
I Assume thatE(F) =0 andCov(F) =Ip
I Also E() =0 and Cov() = Ψ =diag((ψjj))
I Cov(F,) =0.
3 / 17
Implication
The assumptions lead to
I E(X) =µ+LE(F) +E() =µ
Σ = Cov(X) =E(X−µ)(X−µ)0
= E[(LF+)(LF+)0]
= LE(FF0)L0+E(F0)L0+LE(F0) +E(0)
= LL0+ Ψ
I Also (X−µ)F0= (LF+)F0 =LFF0+F0
⇒Cov(X,F) =E(X−µ)F0 =LE(FF0) +E(F0) =L.
4 / 17
Implication
The assumptions lead to
I E(X) =µ+LE(F) +E() =µ
Σ = Cov(X) =E(X−µ)(X−µ)0
= E[(LF+)(LF+)0]
= LE(FF0)L0+E(F0)L0+LE(F0) +E(0)
= LL0+ Ψ
I Also (X−µ)F0= (LF+)F0 =LFF0+F0
⇒Cov(X,F) =E(X−µ)F0 =LE(FF0) +E(F0) =L.
4 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Estimation of the parameters
I The parameters involved are (L,Ψ,µ).
I There are mpunknowns inL andm unknowns in each of Ψ andµ, a total ofm(p+ 2)parameters.
I Let X1,X2, . . . ,Xn be a sample of size n.
I µis estimated as µˆ =X= n1Pn i=1Xi.
I Henceforth we will work withXi−Xso that we have mp+m parameters ofL andΨ to estimate.
I Usually either of two estmation techniques are used :
I Principal Component Method
I Maximum Likelihood Method
5 / 17
Principal Component Method
I Σ =Cov(X) is p.d.
I Let λ1 ≥λ2≥. . .≥λm be the eigen-values ofΣ with p1,p2, . . . ,pm the corresponding eigen-vectors.
I Then the spectral decomposition of Σcan be obtained as Σ = λ1p1p01+λ2p2p02. . .+λmpmp0m
= √ λ1p1
√λ2p2 . . . √ λmpm
√λ1p1
√λ2p2
...
√λmpm
= L˜L˜0.
I Here we have a representation of Σsimilar to that of the model, except thatΨ = 0since we have m factors and hence no reduction.
6 / 17
Principal Component Method
I Σ =Cov(X) is p.d.
I Let λ1 ≥λ2≥. . .≥λm be the eigen-values ofΣ with p1,p2, . . . ,pm the corresponding eigen-vectors.
I Then the spectral decomposition of Σcan be obtained as Σ = λ1p1p01+λ2p2p02. . .+λmpmp0m
= √ λ1p1
√λ2p2 . . . √ λmpm
√λ1p1
√λ2p2
...
√λmpm
= L˜L˜0.
I Here we have a representation of Σsimilar to that of the model, except thatΨ = 0since we have m factors and hence no reduction.
6 / 17
Principal Component Method
I Σ =Cov(X) is p.d.
I Let λ1 ≥λ2≥. . .≥λm be the eigen-values ofΣ with p1,p2, . . . ,pm the corresponding eigen-vectors.
I Then the spectral decomposition of Σcan be obtained as Σ = λ1p1p01+λ2p2p02. . .+λmpmp0m
= √ λ1p1
√λ2p2 . . . √ λmpm
√λ1p1
√λ2p2
...
√λmpm
= L˜L˜0.
I Here we have a representation of Σsimilar to that of the model, except thatΨ = 0since we have m factors and hence no reduction.
6 / 17
Principal Component Method
I Σ =Cov(X) is p.d.
I Let λ1 ≥λ2≥. . .≥λm be the eigen-values ofΣ with p1,p2, . . . ,pm the corresponding eigen-vectors.
I Then the spectral decomposition of Σcan be obtained as Σ = λ1p1p01+λ2p2p02. . .+λmpmp0m
= √ λ1p1
√λ2p2 . . . √ λmpm
√λ1p1
√λ2p2
...
√λmpm
= L˜L˜0.
I Here we have a representation of Σsimilar to that of the model, except thatΨ = 0since we have m factors and hence no reduction.
6 / 17
Principal Component Method
I The principal component method suggests dropping the last (m−p) columns of L˜ and approximating Σas
Σ ≈ √ λ1p1
√λ2p2 . . . p λppp
√√λ1p1
λ2p2 ... pλppp
= LL0.
I Since the λj’s are ordered, this means dropping the contributions of the smallest(m−p) eigen-values.
7 / 17
Principal Component Method
I The principal component method suggests dropping the last (m−p) columns of L˜ and approximating Σas
Σ ≈ √ λ1p1
√λ2p2 . . . p λppp
√√λ1p1
λ2p2 ... pλppp
= LL0.
I Since the λj’s are ordered, this means dropping the contributions of the smallest(m−p) eigen-values.
7 / 17
Principal Component Method
I The difference matrix is L˜L˜0−LL0
= p
λp+1pp+1 . . . √ λmpm
pλp+1pp+1
...
√λmpm
.
I The diagonal elements ofΨare then obtained from the diagonal elements of L˜L˜0−LL0, i.e.
ψjj =σjj−
p
X
k=1
ljk2 .
I Ψis then constructed by taking ψjj on the diagonal and 0’s elsewhere.
8 / 17
Principal Component Method
I The difference matrix is L˜L˜0−LL0
= p
λp+1pp+1 . . . √ λmpm
pλp+1pp+1
...
√λmpm
.
I The diagonal elements ofΨare then obtained from the diagonal elements of L˜L˜0−LL0, i.e.
ψjj =σjj−
p
X
k=1
ljk2 .
I Ψis then constructed by taking ψjj on the diagonal and 0’s elsewhere.
8 / 17
Principal Component Method
I The difference matrix is L˜L˜0−LL0
= p
λp+1pp+1 . . . √ λmpm
pλp+1pp+1
...
√λmpm
.
I The diagonal elements ofΨare then obtained from the diagonal elements of L˜L˜0−LL0, i.e.
ψjj =σjj−
p
X
k=1
ljk2 .
I Ψis then constructed by taking ψjj on the diagonal and 0’s elsewhere.
8 / 17
Principal Component Method
I How to use the technique ?
I Estimate Σby the sample covariance matrix S = n−11 Pn
i=1(Xi−X)(Xi−X)0.
I Obtain the eigen-values and eigen-vectors of S, λˆ1≥ˆλ2 ≥. . .≥λˆm andpˆ1,pˆ2, . . . ,pˆm.
I Then Lˆ = h p
ˆλ1pˆ1
pλˆ2pˆ2 . . .
qˆλppˆp
i
and ψˆjj =sjj−
p
X
k=1
ˆljk2
with Ψ =ˆ diag(( ˆψjj)).
9 / 17
Principal Component Method
I How to use the technique ?
I Estimate Σby the sample covariance matrix S = n−11 Pn
i=1(Xi−X)(Xi−X)0.
I Obtain the eigen-values and eigen-vectors of S, λˆ1≥ˆλ2 ≥. . .≥λˆm andpˆ1,pˆ2, . . . ,pˆm.
I Then Lˆ = h p
ˆλ1pˆ1
pλˆ2pˆ2 . . .
qˆλppˆp
i
and ψˆjj =sjj−
p
X
k=1
ˆljk2
with Ψ =ˆ diag(( ˆψjj)).
9 / 17
Principal Component Method
I How to use the technique ?
I Estimate Σby the sample covariance matrix S = n−11 Pn
i=1(Xi−X)(Xi−X)0.
I Obtain the eigen-values and eigen-vectors of S, λˆ1≥ˆλ2 ≥. . .≥λˆm andpˆ1,pˆ2, . . . ,pˆm.
I Then Lˆ = h p
ˆλ1pˆ1
pλˆ2pˆ2 . . .
qˆλppˆp
i
and ψˆjj =sjj−
p
X
k=1
ˆljk2
with Ψ =ˆ diag(( ˆψjj)).
9 / 17
Principal Component Method
I How to use the technique ?
I Estimate Σby the sample covariance matrix S = n−11 Pn
i=1(Xi−X)(Xi−X)0.
I Obtain the eigen-values and eigen-vectors of S, λˆ1≥ˆλ2 ≥. . .≥λˆm andpˆ1,pˆ2, . . . ,pˆm.
I Then Lˆ = h p
ˆλ1pˆ1
pλˆ2pˆ2 . . .
qˆλppˆp
i
and ψˆjj =sjj−
p
X
k=1
ˆljk2
with Ψ =ˆ diag(( ˆψjj)).
9 / 17
How good is the fit ?
I The residual matrix is S−( ˆLLˆ0+ ˆΨ).
I Notice that this has diagonal elements 0.
I For a good fit the non-diagonal elements of the residual matrix must be small.
I Since
sum of squares of S−( ˆLLˆ0+ ˆΨ)≤λˆp+1+ ˆλp+2+. . .+ ˆλm
small ˆλp+1,λˆp+2, . . . ,λˆm ensures that the fit is good.
10 / 17
How good is the fit ?
I The residual matrix is S−( ˆLLˆ0+ ˆΨ).
I Notice that this has diagonal elements 0.
I For a good fit the non-diagonal elements of the residual matrix must be small.
I Since
sum of squares of S−( ˆLLˆ0+ ˆΨ)≤λˆp+1+ ˆλp+2+. . .+ ˆλm
small ˆλp+1,λˆp+2, . . . ,λˆm ensures that the fit is good.
10 / 17
How good is the fit ?
I The residual matrix is S−( ˆLLˆ0+ ˆΨ).
I Notice that this has diagonal elements 0.
I For a good fit the non-diagonal elements of the residual matrix must be small.
I Since
sum of squares of S−( ˆLLˆ0+ ˆΨ)≤λˆp+1+ ˆλp+2+. . .+ ˆλm
small ˆλp+1,λˆp+2, . . . ,λˆm ensures that the fit is good.
10 / 17
How good is the fit ?
I The residual matrix is S−( ˆLLˆ0+ ˆΨ).
I Notice that this has diagonal elements 0.
I For a good fit the non-diagonal elements of the residual matrix must be small.
I Since
sum of squares of S−( ˆLLˆ0+ ˆΨ)≤λˆp+1+ ˆλp+2+. . .+ ˆλm
small ˆλp+1,λˆp+2, . . . ,λˆm ensures that the fit is good.
10 / 17
The choice of p
I But what should be the choice ofp ?
I It is obvious from the spectral decomposition that the
contribution of the1st factor tosjj, the variability ofXj isˆlj12 .
I Observe that the total variability of the observed values is tr S =s11+s22+. . .+smm.
I Thus contribution of the 1st factor to the total variability is ˆl112 + ˆl221+. . .+ ˆl2m1 = (
q
λˆ1pˆ1)0( q
λˆ1pˆ1) = ˆλ1
I Thus proportion of variability explained by the 1st factor is λˆ1/(s11+s22+. . .+smm).
11 / 17
The choice of p
I But what should be the choice ofp ?
I It is obvious from the spectral decomposition that the
contribution of the1st factor tosjj, the variability ofXj isˆlj12 .
I Observe that the total variability of the observed values is tr S =s11+s22+. . .+smm.
I Thus contribution of the 1st factor to the total variability is ˆl112 + ˆl221+. . .+ ˆl2m1 = (
q
λˆ1pˆ1)0( q
λˆ1pˆ1) = ˆλ1
I Thus proportion of variability explained by the 1st factor is λˆ1/(s11+s22+. . .+smm).
11 / 17
The choice of p
I But what should be the choice ofp ?
I It is obvious from the spectral decomposition that the
contribution of the1st factor tosjj, the variability ofXj isˆlj12 .
I Observe that the total variability of the observed values is tr S =s11+s22+. . .+smm.
I Thus contribution of the 1st factor to the total variability is ˆl112 + ˆl221+. . .+ ˆl2m1 = (
q
λˆ1pˆ1)0( q
λˆ1pˆ1) = ˆλ1
I Thus proportion of variability explained by the 1st factor is λˆ1/(s11+s22+. . .+smm).
11 / 17
The choice of p
I But what should be the choice ofp ?
I It is obvious from the spectral decomposition that the
contribution of the1st factor tosjj, the variability ofXj isˆlj12 .
I Observe that the total variability of the observed values is tr S =s11+s22+. . .+smm.
I Thus contribution of the 1st factor to the total variability is ˆl112 + ˆl221+. . .+ ˆl2m1 = (
q
λˆ1pˆ1)0( q
λˆ1pˆ1) = ˆλ1
I Thus proportion of variability explained by the 1st factor is λˆ1/(s11+s22+. . .+smm).
11 / 17
The choice of p
I But what should be the choice ofp ?
I It is obvious from the spectral decomposition that the
contribution of the1st factor tosjj, the variability ofXj isˆlj12 .
I Observe that the total variability of the observed values is tr S =s11+s22+. . .+smm.
I Thus contribution of the 1st factor to the total variability is ˆl112 + ˆl221+. . .+ ˆl2m1 = (
q
λˆ1pˆ1)0( q
λˆ1pˆ1) = ˆλ1
I Thus proportion of variability explained by the 1st factor is λˆ1/(s11+s22+. . .+smm).
11 / 17
The choice of p
I In general, proportion of variability explained by thejth factor is
ˆλj/(s11+s22+. . .+smm).
I The cumulative proportion of variability explained by the first p factors is
(ˆλ1+. . .+ ˆλp)/(s11+s22+. . .+smm).
I Choose p such that this proportion is high (say90% or95%).
12 / 17
The choice of p
I In general, proportion of variability explained by thejth factor is
ˆλj/(s11+s22+. . .+smm).
I The cumulative proportion of variability explained by the first p factors is
(ˆλ1+. . .+ ˆλp)/(s11+s22+. . .+smm).
I Choose p such that this proportion is high (say90% or95%).
12 / 17
The choice of p
I In general, proportion of variability explained by thejth factor is
ˆλj/(s11+s22+. . .+smm).
I The cumulative proportion of variability explained by the first p factors is
(ˆλ1+. . .+ ˆλp)/(s11+s22+. . .+smm).
I Choose p such that this proportion is high (say90% or95%).
12 / 17
Example (Johnson & Wichern (2009)
I In a consumer preference study, customers were asked to rate the attributes of a new product on a 7-point scale. The attributes were Taste (X1), Good buy for money (X2), Flavour (X3), Suitable for snack (X4), Provides lots of energy (X5).The correlation matrix (R) thus obtained was
1.00 0.02 0.96 0.42 0.01 0.02 1.00 0.13 0.71 0.85 0.96 0.13 1.00 0.50 0.11 0.42 0.71 0.50 1.00 0.79 0.01 0.85 0.11 0.79 1.00
I The first eigen-value of R is λˆ1 = 2.85.
I Since tr(R) = 5, the first factor explains(2.85/5)% = 57% of the variability.
13 / 17
Example (Johnson & Wichern (2009)
I In a consumer preference study, customers were asked to rate the attributes of a new product on a 7-point scale. The attributes were Taste (X1), Good buy for money (X2), Flavour (X3), Suitable for snack (X4), Provides lots of energy (X5).The correlation matrix (R) thus obtained was
1.00 0.02 0.96 0.42 0.01 0.02 1.00 0.13 0.71 0.85 0.96 0.13 1.00 0.50 0.11 0.42 0.71 0.50 1.00 0.79 0.01 0.85 0.11 0.79 1.00
I The first eigen-value of R is λˆ1 = 2.85.
I Since tr(R) = 5, the first factor explains(2.85/5)% = 57% of the variability.
13 / 17
Example (Johnson & Wichern (2009)
I In a consumer preference study, customers were asked to rate the attributes of a new product on a 7-point scale. The attributes were Taste (X1), Good buy for money (X2), Flavour (X3), Suitable for snack (X4), Provides lots of energy (X5).The correlation matrix (R) thus obtained was
1.00 0.02 0.96 0.42 0.01 0.02 1.00 0.13 0.71 0.85 0.96 0.13 1.00 0.50 0.11 0.42 0.71 0.50 1.00 0.79 0.01 0.85 0.11 0.79 1.00
I The first eigen-value of R is λˆ1 = 2.85.
I Since tr(R) = 5, the first factor explains(2.85/5)% = 57% of the variability.
13 / 17
Example (Johnson & Wichern (2009)
I The second eigen-value of R isλˆ2= 1.81.
I Together, the first two factors explain
(2.85 + 1.81)/5 % = 93.2% of the variability.
I So one can work with 2 factors instead of the 5 variables.
I The factor loadings, communalities and uniquenesses are obtained as
F1 F2 h2j ψjj
X1 0.56 0.82 0.98 0.02 X2 0.78 -0.53 0.88 0.12 X3 0.65 0.75 0.98 0.02 X4 0.94 -0.10 0.89 0.11 X5 0.80 -0.54 0.93 0.07
14 / 17
Example (Johnson & Wichern (2009)
I The second eigen-value of R isλˆ2= 1.81.
I Together, the first two factors explain
(2.85 + 1.81)/5 % = 93.2% of the variability.
I So one can work with 2 factors instead of the 5 variables.
I The factor loadings, communalities and uniquenesses are obtained as
F1 F2 h2j ψjj
X1 0.56 0.82 0.98 0.02 X2 0.78 -0.53 0.88 0.12 X3 0.65 0.75 0.98 0.02 X4 0.94 -0.10 0.89 0.11 X5 0.80 -0.54 0.93 0.07
14 / 17
Example (Johnson & Wichern (2009)
I The second eigen-value of R isλˆ2= 1.81.
I Together, the first two factors explain
(2.85 + 1.81)/5 % = 93.2% of the variability.
I So one can work with 2 factors instead of the 5 variables.
I The factor loadings, communalities and uniquenesses are obtained as
F1 F2 h2j ψjj
X1 0.56 0.82 0.98 0.02 X2 0.78 -0.53 0.88 0.12 X3 0.65 0.75 0.98 0.02 X4 0.94 -0.10 0.89 0.11 X5 0.80 -0.54 0.93 0.07
14 / 17
Example (Johnson & Wichern (2009)
I The second eigen-value of R isλˆ2= 1.81.
I Together, the first two factors explain
(2.85 + 1.81)/5 % = 93.2% of the variability.
I So one can work with 2 factors instead of the 5 variables.
I The factor loadings, communalities and uniquenesses are obtained as
F1 F2 h2j ψjj
X1 0.56 0.82 0.98 0.02 X2 0.78 -0.53 0.88 0.12 X3 0.65 0.75 0.98 0.02 X4 0.94 -0.10 0.89 0.11 X5 0.80 -0.54 0.93 0.07
14 / 17
Principal Factor Solution
I The principal factor solution is a technique often used to solve for the eigen-values and eigen-vectors.
I Notice that the diagonal elements of R should be 1 =h2jj+ψj or h2jj = 1−ψj.
I Supposeψj =ψj∗.
I Then for h∗2jj = 1−ψj∗, write the correlation matrix R∗ as h∗211 r12 . . . r1m
r21 h∗222 . . . r2m . . . . rm1 rm2 . . . h∗2mm
15 / 17
Principal Factor Solution
I The principal factor solution is a technique often used to solve for the eigen-values and eigen-vectors.
I Notice that the diagonal elements of R should be 1 =h2jj+ψj or h2jj = 1−ψj.
I Supposeψj =ψj∗.
I Then for h∗2jj = 1−ψj∗, write the correlation matrix R∗ as h∗211 r12 . . . r1m
r21 h∗222 . . . r2m . . . . rm1 rm2 . . . h∗2mm
15 / 17
Principal Factor Solution
I The principal factor solution is a technique often used to solve for the eigen-values and eigen-vectors.
I Notice that the diagonal elements of R should be 1 =h2jj+ψj or h2jj = 1−ψj.
I Supposeψj =ψj∗.
I Then for h∗2jj = 1−ψj∗, write the correlation matrix R∗ as h∗211 r12 . . . r1m
r21 h∗222 . . . r2m . . . . rm1 rm2 . . . h∗2mm
15 / 17
Principal Factor Solution
I The principal factor solution is a technique often used to solve for the eigen-values and eigen-vectors.
I Notice that the diagonal elements of R should be 1 =h2jj+ψj or h2jj = 1−ψj.
I Supposeψj =ψj∗.
I Then for h∗2jj = 1−ψj∗, write the correlation matrix R∗ as h∗211 r12 . . . r1m
r21 h∗222 . . . r2m . . . . rm1 rm2 . . . h∗2mm
15 / 17
Principal Factor Solution
I Using the principal component method on R∗ to get L∗ = p
λ∗1p∗1 p
λ∗2p∗2 . . . p
λ∗mp∗m , whereλ∗j’s and p∗j’s are respectively the eigen-values and eigen-vectors ofR∗.
I Also ψ∗jj = 1−Pp k=1ˆl∗2jk.
I Updating with this new ψ∗jj, iterate till convergence.
I A possible initial solution is
h∗2jj = 1−ψj∗= 1− 1 rjj, whererjj are elements ofR∗−1.
16 / 17
Principal Factor Solution
I Using the principal component method on R∗ to get L∗ = p
λ∗1p∗1 p
λ∗2p∗2 . . . p
λ∗mp∗m , whereλ∗j’s and p∗j’s are respectively the eigen-values and eigen-vectors ofR∗.
I Also ψ∗jj = 1−Pp k=1ˆl∗2jk.
I Updating with this new ψ∗jj, iterate till convergence.
I A possible initial solution is
h∗2jj = 1−ψj∗= 1− 1 rjj, whererjj are elements ofR∗−1.
16 / 17
Principal Factor Solution
I Using the principal component method on R∗ to get L∗ = p
λ∗1p∗1 p
λ∗2p∗2 . . . p
λ∗mp∗m , whereλ∗j’s and p∗j’s are respectively the eigen-values and eigen-vectors ofR∗.
I Also ψ∗jj = 1−Pp k=1ˆl∗2jk.
I Updating with this new ψ∗jj, iterate till convergence.
I A possible initial solution is
h∗2jj = 1−ψj∗= 1− 1 rjj, whererjj are elements ofR∗−1.
16 / 17
Principal Factor Solution
I Using the principal component method on R∗ to get L∗ = p
λ∗1p∗1 p
λ∗2p∗2 . . . p
λ∗mp∗m , whereλ∗j’s and p∗j’s are respectively the eigen-values and eigen-vectors ofR∗.
I Also ψ∗jj = 1−Pp k=1ˆl∗2jk.
I Updating with this new ψ∗jj, iterate till convergence.
I A possible initial solution is
h∗2jj = 1−ψj∗= 1− 1 rjj, whererjj are elements ofR∗−1.
16 / 17
Summary
I Methods of estimation of factor models are introduced.
I The Principal Component technique of estimating the factor model parameters is discussed.
I A numerical illustration is used to expalin the method.
I The principal factor solution technique is discussed.
17 / 17
Summary
I Methods of estimation of factor models are introduced.
I The Principal Component technique of estimating the factor model parameters is discussed.
I A numerical illustration is used to expalin the method.
I The principal factor solution technique is discussed.
17 / 17
Summary
I Methods of estimation of factor models are introduced.
I The Principal Component technique of estimating the factor model parameters is discussed.
I A numerical illustration is used to expalin the method.
I The principal factor solution technique is discussed.
17 / 17
Summary
I Methods of estimation of factor models are introduced.
I The Principal Component technique of estimating the factor model parameters is discussed.
I A numerical illustration is used to expalin the method.
I The principal factor solution technique is discussed.
17 / 17