Module: Canonical Correlation

Subject: Statistics

Paper: Multivariate Analysis

Module: Canonical Correlation

Development Team

Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta

Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta

Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad

Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta

Development Team

Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta

Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta

Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad

Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta

Development Team

Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta

Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta

Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad

Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta

Development Team

Principal investigator: Dr. Bhaswati Ganguli, Professor, Department of Statistics, University of Calcutta

Paper co-ordinator: Dr. Sugata SenRoy,Professor, Department of Statistics, University of Calcutta

Content writer: Souvik Bandyopadhyay, Senior Lecturer, Indian Institute of Public Health, Hyderabad

Content reviewer: Dr. Kalyan Das,Professor, Department of Statistics, University of Calcutta

Correlations

I Q : What is canonical correlation ?

I It’s a technique which seeks to identify and quantify the relationship between two sets of variables.

I Special cases

I Simple correlation

I Multiple Correlation

I In simple correlation, we study the relationship between two variables X andY.

I In multiple correlation, we study the relationship between a variable Y and a set of variables (X1, . . . , Xp).

←To do this we take a linear combination ofX₁, . . . , X_p and find its correlation with Y.

Correlations

I Q : What is canonical correlation ?

I It’s a technique which seeks to identify and quantify the relationship between two sets of variables.

I Special cases

I Simple correlation

I Multiple Correlation

I In simple correlation, we study the relationship between two variables X andY.

I In multiple correlation, we study the relationship between a variable Y and a set of variables (X1, . . . , Xp).

←To do this we take a linear combination ofX₁, . . . , X_p and find its correlation with Y.

Correlations

I Q : What is canonical correlation ?

I It’s a technique which seeks to identify and quantify the relationship between two sets of variables.

I Special cases

I Simple correlation

I Multiple Correlation

I In simple correlation, we study the relationship between two variables X andY.

I In multiple correlation, we study the relationship between a variable Y and a set of variables (X1, . . . , Xp).

←To do this we take a linear combination ofX₁, . . . , X_p and find its correlation with Y.

Correlations

I Q : What is canonical correlation ?

I It’s a technique which seeks to identify and quantify the relationship between two sets of variables.

I Special cases

I Simple correlation

I Multiple Correlation

I In simple correlation, we study the relationship between two variables X andY.

I In multiple correlation, we study the relationship between a variable Y and a set of variables (X1, . . . , Xp).

←To do this we take a linear combination ofX₁, . . . , X_p and find its correlation with Y.

Correlations

I Q : What is canonical correlation ?

I It’s a technique which seeks to identify and quantify the relationship between two sets of variables.

I Special cases

I Simple correlation

I Multiple Correlation

I In simple correlation, we study the relationship between two variables X andY.

I In multiple correlation, we study the relationship between a variable Y and a set of variables (X1, . . . , Xp).

←To do this we take a linear combination ofX₁, . . . , X_p and find its correlation with Y.

Correlations

I Q : What is canonical correlation ?

I It’s a technique which seeks to identify and quantify the relationship between two sets of variables.

I Special cases

I Simple correlation

I Multiple Correlation

I In simple correlation, we study the relationship between two variables X andY.

I In multiple correlation, we study the relationship between a variable Y and a set of variables (X1, . . . , Xp).

←To do this we take a linear combination ofX₁, . . . , X_p and find its correlation with Y.

Correlations

I Q : What is canonical correlation ?

I It’s a technique which seeks to identify and quantify the relationship between two sets of variables.

I Special cases

I Simple correlation

I Multiple Correlation

I In simple correlation, we study the relationship between two variables X andY.

I In multiple correlation, we study the relationship between a variable Y and a set of variables (X1, . . . , Xp).

←To do this we take a linear combination ofX₁, . . . , X_p and find its correlation with Y.

Canonical Correlation

I In canonical correlation we study the relationship between two sets of variables (X₁, . . . , X_r)and (X₁^∗, . . . , X_s^∗).

I Canonical correlation requires that each set of variables be reduced to a single variable and then finding their correlation.

I Usually these two variables are found by taking linear combinations of the variables in each set, under certain pre-fixed criteria.

I The variables obtained by these linear combinations are known as canonical variables and the correlation between them as canonical correlation.

Canonical Correlation

I In canonical correlation we study the relationship between two sets of variables (X₁, . . . , X_r)and (X₁^∗, . . . , X_s^∗).

I Canonical correlation requires that each set of variables be reduced to a single variable and then finding their correlation.

I Usually these two variables are found by taking linear combinations of the variables in each set, under certain pre-fixed criteria.

I The variables obtained by these linear combinations are known as canonical variables and the correlation between them as canonical correlation.

Canonical Correlation

I In canonical correlation we study the relationship between two sets of variables (X₁, . . . , X_r)and (X₁^∗, . . . , X_s^∗).

I Canonical correlation requires that each set of variables be reduced to a single variable and then finding their correlation.

I Usually these two variables are found by taking linear combinations of the variables in each set, under certain pre-fixed criteria.

I The variables obtained by these linear combinations are known as canonical variables and the correlation between them as canonical correlation.

Canonical Correlation

I In canonical correlation we study the relationship between two sets of variables (X₁, . . . , X_r)and (X₁^∗, . . . , X_s^∗).

I Canonical correlation requires that each set of variables be reduced to a single variable and then finding their correlation.

I Usually these two variables are found by taking linear combinations of the variables in each set, under certain pre-fixed criteria.

I The variables obtained by these linear combinations are known as canonical variables and the correlation between them as canonical correlation.

The Model

I Let there ber variables in the1^st groupX1= (X1, . . . , Xr) ands variables in the2^nd groupX₂ = (X_r+1, . . . , X_r+s).

I Assume, without loss of generality, thatr ≤s, i.e. take the smaller group of variables as the 1^st group.

I Also let

E(X₁) =µ₁ andE(X₂) =µ₂ V ar(X1) = Σ11,V ar(X2) = Σ22

andCov(X1,X2) = Σ12 (orΣ⁰₂₁)

I DefineX= X₁

X2

,µ=

µ₁ µ₂

andΣ =

Σ₁₁ Σ₁₂ Σ21 Σ22

.

The Model

I Let there ber variables in the1^st groupX1= (X1, . . . , Xr) ands variables in the2^nd groupX₂ = (X_r+1, . . . , X_r+s).

I Assume, without loss of generality, thatr ≤s, i.e. take the smaller group of variables as the 1^st group.

I Also let

E(X₁) =µ₁ andE(X₂) =µ₂ V ar(X1) = Σ11,V ar(X2) = Σ22

andCov(X1,X2) = Σ12 (orΣ⁰₂₁)

I DefineX= X₁

X2

,µ=

µ₁ µ₂

andΣ =

Σ₁₁ Σ₁₂ Σ21 Σ22

.

The Model

I Let there ber variables in the1^st groupX1= (X1, . . . , Xr) ands variables in the2^nd groupX₂ = (X_r+1, . . . , X_r+s).

I Assume, without loss of generality, thatr ≤s, i.e. take the smaller group of variables as the 1^st group.

I Also let

E(X₁) =µ₁ andE(X₂) =µ₂ V ar(X1) = Σ11,V ar(X2) = Σ22

andCov(X1,X2) = Σ12 (orΣ⁰₂₁)

I DefineX= X₁

X2

,µ=

µ₁ µ₂

andΣ =

Σ₁₁ Σ₁₂ Σ21 Σ22

.

The Model

I Let there ber variables in the1^st groupX1= (X1, . . . , Xr) ands variables in the2^nd groupX₂ = (X_r+1, . . . , X_r+s).

I Assume, without loss of generality, thatr ≤s, i.e. take the smaller group of variables as the 1^st group.

I Also let

E(X₁) =µ₁ andE(X₂) =µ₂ V ar(X1) = Σ11,V ar(X2) = Σ22

andCov(X1,X2) = Σ12 (orΣ⁰₂₁)

I DefineX= X₁

X2

,µ=

µ₁ µ₂

andΣ =

Σ₁₁ Σ₁₂ Σ21 Σ22

.

The Problem

I Σ₁₂ containsrselements which gives the correlations between each variable of the 1^st group with those of the2^nd group

← gives a perfect picture of the relationships

← however, not easily comprehendible.

I Instead, can we look at just a few values, which would make understanding easier ?

The Problem

I Σ₁₂ containsrselements which gives the correlations between each variable of the 1^st group with those of the2^nd group

← gives a perfect picture of the relationships

← however, not easily comprehendible.

I Instead, can we look at just a few values, which would make understanding easier ?

The Method

I Let aand bbe respectively r andsdimensional vectors.

I Define the variables,

U =a⁰X₁ and V =b⁰X₂.

Then E(U) =a⁰µ₁, E(V) =b⁰µ₂, V ar(U) =a⁰Σ11a, V ar(V) =b⁰Σ22b, with Cov(U, V) =a⁰Σ₁₂b.

so that

Corr(U, V) = a⁰Σ₁₂b

√a⁰Σ₁₁a×b⁰Σ₂₂b.

The Method

I Let aand bbe respectively r andsdimensional vectors.

I Define the variables,

U =a⁰X₁ and V =b⁰X₂.

Then E(U) =a⁰µ₁, E(V) =b⁰µ₂, V ar(U) =a⁰Σ11a, V ar(V) =b⁰Σ22b, with Cov(U, V) =a⁰Σ₁₂b.

so that

Corr(U, V) = a⁰Σ₁₂b

√a⁰Σ₁₁a×b⁰Σ₂₂b.

How to choose a and b

I The coefficients a₁,b₁ of the first pair(U₁, V₁)are so chosen as to maximizeCorr(U, V).

I The 2nd pair(U2, V2) are chosen to maximizeCorr(U, V) subject to their combining vectorsa2 andb2 being orthogonal to a₁ and b₁, respectively.

I In general, the jth pair (U_j, V_j) are chosen to maximize Corr(U, V)subject toa_j and b_j being orthogonal to a_k and bk, respectively, for all k < j.

I This can be done till j=r.

I Usually for all j= 1, . . . , r,

a⁰_ja_j = 1 and b⁰_jb_j = 1.

How to choose a and b

I The coefficients a₁,b₁ of the first pair(U₁, V₁)are so chosen as to maximizeCorr(U, V).

I The 2nd pair(U2, V2) are chosen to maximizeCorr(U, V) subject to their combining vectorsa2 andb2 being orthogonal to a₁ and b₁, respectively.

I In general, the jth pair (U_j, V_j) are chosen to maximize Corr(U, V)subject toa_j and b_j being orthogonal to a_k and bk, respectively, for all k < j.

I This can be done till j=r.

I Usually for all j= 1, . . . , r,

a⁰_ja_j = 1 and b⁰_jb_j = 1.

How to choose a and b

I The coefficients a₁,b₁ of the first pair(U₁, V₁)are so chosen as to maximizeCorr(U, V).

I The 2nd pair(U2, V2) are chosen to maximizeCorr(U, V) subject to their combining vectorsa2 andb2 being orthogonal to a₁ and b₁, respectively.

I In general, the jth pair (U_j, V_j) are chosen to maximize Corr(U, V)subject toa_j and b_j being orthogonal to a_k and bk, respectively, for all k < j.

I This can be done till j=r.

I Usually for all j= 1, . . . , r,

a⁰_ja_j = 1 and b⁰_jb_j = 1.

How to choose a and b

I The coefficients a₁,b₁ of the first pair(U₁, V₁)are so chosen as to maximizeCorr(U, V).

I The 2nd pair(U2, V2) are chosen to maximizeCorr(U, V) subject to their combining vectorsa2 andb2 being orthogonal to a₁ and b₁, respectively.

I In general, the jth pair (U_j, V_j) are chosen to maximize Corr(U, V)subject toa_j and b_j being orthogonal to a_k and bk, respectively, for all k < j.

I This can be done till j=r.

I Usually for all j= 1, . . . , r,

a⁰_ja_j = 1 and b⁰_jb_j = 1.

How to choose a and b

I The coefficients a₁,b₁ of the first pair(U₁, V₁)are so chosen as to maximizeCorr(U, V).

I The 2nd pair(U2, V2) are chosen to maximizeCorr(U, V) subject to their combining vectorsa2 andb2 being orthogonal to a₁ and b₁, respectively.

I In general, the jth pair (U_j, V_j) are chosen to maximize Corr(U, V)subject toa_j and b_j being orthogonal to a_k and bk, respectively, for all k < j.

I This can be done till j=r.

I Usually for all j= 1, . . . , r,

a⁰_ja_j = 1 and b⁰_jb_j = 1.

Result

I Let λ1 ≥λ2≥. . .≥λr be the eigenvalues of

Σ^−1/2₁₁ Σ₁₂Σ⁻¹₂₂Σ₂₁Σ^−1/2₁₁ andp₁,p₂, . . . ,p_r the corresponding eigen-vectors.

I In fact, λ₁ ≥λ₂≥. . .≥λ_r are also the r largest (out ofs) eigen-values of Σ^−1/2₂₂ Σ₂₁Σ⁻¹₁₁Σ₁₂Σ^−1/2₂₂ with q₁,q₂, . . . ,q_r the corresponding eigen-vectors.

Result

Corr(U₁, V₁) =ρ₁=p λ₁, witha₁= Σ^−1/2₁₁ p₁ andb₁= Σ^−1/2₂₂ q₁.

I Thus U₁ =p⁰₁Σ^−1/2₁₁ X₁ andV₂=q⁰₁Σ^−1/2₂₂ X₂.

Result

I Let λ1 ≥λ2≥. . .≥λr be the eigenvalues of

Σ^−1/2₁₁ Σ₁₂Σ⁻¹₂₂Σ₂₁Σ^−1/2₁₁ andp₁,p₂, . . . ,p_r the corresponding eigen-vectors.

I In fact, λ₁ ≥λ₂≥. . .≥λ_r are also the r largest (out ofs) eigen-values of Σ^−1/2₂₂ Σ₂₁Σ⁻¹₁₁Σ₁₂Σ^−1/2₂₂ with q₁,q₂, . . . ,q_r the corresponding eigen-vectors.

Result

Corr(U₁, V₁) =ρ₁=p λ₁,

witha₁= Σ^−1/2₁₁ p₁ andb₁= Σ^−1/2₂₂ q₁.

−1/2 −1/2

Result

I Let λ1 ≥λ2≥. . .≥λr be the eigenvalues of

Σ^−1/2₁₁ Σ₁₂Σ⁻¹₂₂Σ₂₁Σ^−1/2₁₁ andp₁,p₂, . . . ,p_r the corresponding eigen-vectors.

I In fact, λ₁ ≥λ₂≥. . .≥λ_r are also the r largest (out ofs) eigen-values of Σ^−1/2₂₂ Σ₂₁Σ⁻¹₁₁Σ₁₂Σ^−1/2₂₂ with q₁,q₂, . . . ,q_r the corresponding eigen-vectors.

Result

Corr(U₁, V₁) =ρ₁=p λ₁,

witha₁= Σ^−1/2₁₁ p₁ andb₁= Σ^−1/2₂₂ q₁.

I Thus U₁ =p⁰₁Σ^−1/2₁₁ X₁ andV₂=q⁰₁Σ^−1/2₂₂ X₂.

Result

I Let λ1 ≥λ2≥. . .≥λr be the eigenvalues of

Σ^−1/2₁₁ Σ₁₂Σ⁻¹₂₂Σ₂₁Σ^−1/2₁₁ andp₁,p₂, . . . ,p_r the corresponding eigen-vectors.

I In fact, λ₁ ≥λ₂≥. . .≥λ_r are also the r largest (out ofs) eigen-values of Σ^−1/2₂₂ Σ₂₁Σ⁻¹₁₁Σ₁₂Σ^−1/2₂₂ with q₁,q₂, . . . ,q_r the corresponding eigen-vectors.

Result

Corr(U₁, V₁) =ρ₁=p λ₁,

witha₁= Σ^−1/2₁₁ p₁ andb₁= Σ^−1/2₂₂ q₁.

−1/2 −1/2

Result (contd.)

Result

In general, forj = 1, . . . , r,

Corr(U_j, V_j) =ρ_j =p λ_j,

withaj = Σ^−1/2₁₁ pj andbj = Σ^−1/2₂₂ qj.

I Usually, only the first one or two of the canonical correlations are looked at.

Result (contd.)

Result

In general, forj = 1, . . . , r,

Corr(U_j, V_j) =ρ_j =p λ_j,

withaj = Σ^−1/2₁₁ pj andbj = Σ^−1/2₂₂ qj.

I Usually, only the first one or two of the canonical correlations are looked at.

Proof

I Since Σ11and Σ22 are p.d., define

c= Σ^1/2₁₁ a,d= Σ^1/2₂₂ b so that a= Σ^−1/2₁₁ c,b= Σ^−1/2₂₂ d.

Corr(U, V) = c⁰Σ^−1/2₁₁ Σ12Σ^−1/2₂₂ d q

c⁰Σ^−1/2₁₁ Σ₁₁Σ^−1/2₁₁ c×d⁰Σ^−1/2₂₂ Σ₂₂Σ^−1/2₂₂ d i.e. Corr(U, V) = c⁰Σ^−1/2₁₁ Σ12Σ^−1/2₂₂ d

√

c⁰c×d⁰d By Cauchy-Schwarz inequality,

c⁰Σ^−1/2₁₁ Σ₁₂Σ^−1/2₂₂ d≤(c⁰Σ^−1/2₁₁ Σ₁₂Σ⁻¹₂₂Σ₂₁Σ^−1/2₁₁ c)1/2(d⁰d)1/2

Proof (contd.)

I Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ is symmetric, with largest eigen-value λ₁ and corresponding eigen-vector p₁. Thus

c⁰Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ c≤λ1c⁰c=λ1.

I Equality occurs atc=p₁.

I Also equality in the Cauchy-Schwarz inequality occurs if d is proportional toΣ^−1/2₂₂ Σ21Σ^−1/2₁₁ p1.

I This means b is proportional to

Σ^−1/2₂₂ Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁ = Σ^−1/2₂₂ q₁ (say), where, q = Σ^−1/2Σ Σ^−1/2p .

Proof (contd.)

I Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ is symmetric, with largest eigen-value λ₁ and corresponding eigen-vector p₁. Thus

c⁰Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ c≤λ1c⁰c=λ1.

I Equality occurs atc=p₁.

I Also equality in the Cauchy-Schwarz inequality occurs if d is proportional toΣ^−1/2₂₂ Σ21Σ^−1/2₁₁ p1.

I This means b is proportional to

Σ^−1/2₂₂ Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁ = Σ^−1/2₂₂ q₁ (say), where, q₁ = Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁.

Proof (contd.)

I Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ is symmetric, with largest eigen-value λ₁ and corresponding eigen-vector p₁. Thus

c⁰Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ c≤λ1c⁰c=λ1.

I Equality occurs atc=p₁.

I Also equality in the Cauchy-Schwarz inequality occurs if d is proportional toΣ^−1/2₂₂ Σ21Σ^−1/2₁₁ p1.

I This means b is proportional to

Σ^−1/2₂₂ Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁ = Σ^−1/2₂₂ q₁ (say), where, q = Σ^−1/2Σ Σ^−1/2p .

Proof (contd.)

I Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ is symmetric, with largest eigen-value λ₁ and corresponding eigen-vector p₁. Thus

c⁰Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ c≤λ1c⁰c=λ1.

I Equality occurs atc=p₁.

I Also equality in the Cauchy-Schwarz inequality occurs if d is proportional toΣ^−1/2₂₂ Σ21Σ^−1/2₁₁ p1.

I This means b is proportional to

Σ^−1/2₂₂ Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁ = Σ^−1/2₂₂ q₁ (say), where, q₁ = Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁.

Proof (contd.)

I Notice that from properties of eigen-values, Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ p1 =λ1p1

so that premultiplying both sides byΣ^−1/2₂₂ Σ21Σ^−1/2₁₁ yields for q₁ = Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁

Σ^−1/2₂₂ Σ21Σ⁻¹₁₁Σ12Σ^−1/2₂₂ q1 =λ1q1

i.e. λ1 is an eigen-value ofΣ^−1/2₂₂ Σ21Σ⁻¹₁₁Σ12Σ^−1/2₂₂ with corresponding eigen-vectorq₁.

I Thus with a1 = Σ^−1/2₁₁ p1 and b1 = Σ^−1/2₂₂ q1,

0 0 p

Proof (contd.)

I Notice that from properties of eigen-values, Σ^−1/2₁₁ Σ12Σ⁻¹₂₂Σ21Σ^−1/2₁₁ p1 =λ1p1

so that premultiplying both sides byΣ^−1/2₂₂ Σ21Σ^−1/2₁₁ yields for q₁ = Σ^−1/2₂₂ Σ₂₁Σ^−1/2₁₁ p₁

Σ^−1/2₂₂ Σ21Σ⁻¹₁₁Σ12Σ^−1/2₂₂ q1 =λ1q1

i.e. λ1 is an eigen-value ofΣ^−1/2₂₂ Σ21Σ⁻¹₁₁Σ12Σ^−1/2₂₂ with corresponding eigen-vectorq₁.

I Thus with a1 = Σ^−1/2₁₁ p1 and b1 = Σ^−1/2₂₂ q1,

Corr(U₁, V₁) =max_a,bCorr(a⁰X₁,b⁰X₂) =p λ₁.

Properties

I E(U_j) =p⁰_jΣ^−1/2₁₁ µ₁ andE(V_j) =q⁰_jΣ^−1/2₂₂ µ₂ for all j.

I V ar(Uj) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pj =p⁰_jpj = 1 for all j.

I V ar(V_j) =q⁰_jΣ^−1/2₂₂ Σ₂₂Σ^−1/2₂₂ q_j =q⁰_jq_j = 1 for all j.

I Cov(Uj, Uk) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pk=p⁰_jpk= 0 for all k6=j.

I Cov(Vj, Vk) =q⁰_jΣ^−1/2₂₂ Σ22Σ^−1/2₂₂ qk =q⁰_jqk = 0for all k6=j.

I Cov(U_j, V_k) =p⁰_jΣ^−1/2₁₁ Σ₁₂Σ^−1/2₂₂ q_k=p⁰_jp_k= 0 for all j6=k.

Properties

I E(U_j) =p⁰_jΣ^−1/2₁₁ µ₁ andE(V_j) =q⁰_jΣ^−1/2₂₂ µ₂ for all j.

I V ar(Uj) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pj =p⁰_jpj = 1 for all j.

I V ar(V_j) =q⁰_jΣ^−1/2₂₂ Σ₂₂Σ^−1/2₂₂ q_j =q⁰_jq_j = 1 for all j.

I Cov(Uj, Uk) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pk=p⁰_jpk= 0 for all k6=j.

I Cov(Vj, Vk) =q⁰_jΣ^−1/2₂₂ Σ22Σ^−1/2₂₂ qk =q⁰_jqk = 0for all k6=j.

I Cov(U_j, V_k) =p⁰_jΣ^−1/2₁₁ Σ₁₂Σ^−1/2₂₂ q_k=p⁰_jp_k= 0 for all j6=k.

Properties

I E(U_j) =p⁰_jΣ^−1/2₁₁ µ₁ andE(V_j) =q⁰_jΣ^−1/2₂₂ µ₂ for all j.

I V ar(Uj) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pj =p⁰_jpj = 1 for all j.

I V ar(V_j) =q⁰_jΣ^−1/2₂₂ Σ₂₂Σ^−1/2₂₂ q_j =q⁰_jq_j = 1 for all j.

I Cov(Uj, Uk) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pk=p⁰_jpk= 0 for all k6=j.

I Cov(Vj, Vk) =q⁰_jΣ^−1/2₂₂ Σ22Σ^−1/2₂₂ qk =q⁰_jqk = 0for all k6=j.

I Cov(U_j, V_k) =p⁰_jΣ^−1/2₁₁ Σ₁₂Σ^−1/2₂₂ q_k=p⁰_jp_k= 0 for all j6=k.

Properties

I E(U_j) =p⁰_jΣ^−1/2₁₁ µ₁ andE(V_j) =q⁰_jΣ^−1/2₂₂ µ₂ for all j.

I V ar(Uj) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pj =p⁰_jpj = 1 for all j.

I V ar(V_j) =q⁰_jΣ^−1/2₂₂ Σ₂₂Σ^−1/2₂₂ q_j =q⁰_jq_j = 1 for all j.

I Cov(Uj, Uk) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pk=p⁰_jpk= 0 for all k6=j.

I Cov(Vj, Vk) =q⁰_jΣ^−1/2₂₂ Σ22Σ^−1/2₂₂ qk =q⁰_jqk = 0for all k6=j.

I Cov(U_j, V_k) =p⁰_jΣ^−1/2₁₁ Σ₁₂Σ^−1/2₂₂ q_k=p⁰_jp_k= 0 for all j6=k.

Properties

I E(U_j) =p⁰_jΣ^−1/2₁₁ µ₁ andE(V_j) =q⁰_jΣ^−1/2₂₂ µ₂ for all j.

I V ar(Uj) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pj =p⁰_jpj = 1 for all j.

I V ar(V_j) =q⁰_jΣ^−1/2₂₂ Σ₂₂Σ^−1/2₂₂ q_j =q⁰_jq_j = 1 for all j.

I Cov(Uj, Uk) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pk=p⁰_jpk= 0 for all k6=j.

I Cov(Vj, Vk) =q⁰_jΣ^−1/2₂₂ Σ22Σ^−1/2₂₂ qk =q⁰_jqk = 0for all k6=j.

I Cov(U_j, V_k) =p⁰_jΣ^−1/2₁₁ Σ₁₂Σ^−1/2₂₂ q_k=p⁰_jp_k= 0 for all j6=k.

Properties

I E(U_j) =p⁰_jΣ^−1/2₁₁ µ₁ andE(V_j) =q⁰_jΣ^−1/2₂₂ µ₂ for all j.

I V ar(Uj) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pj =p⁰_jpj = 1 for all j.

I V ar(V_j) =q⁰_jΣ^−1/2₂₂ Σ₂₂Σ^−1/2₂₂ q_j =q⁰_jq_j = 1 for all j.

I Cov(Uj, Uk) =p⁰_jΣ^−1/2₁₁ Σ11Σ^−1/2₁₁ pk=p⁰_jpk= 0 for all k6=j.

I Cov(Vj, Vk) =q⁰_jΣ^−1/2₂₂ Σ22Σ^−1/2₂₂ qk =q⁰_jqk = 0for all k6=j.

I Cov(U_j, V_k) =p⁰_jΣ^−1/2₁₁ Σ₁₂Σ^−1/2₂₂ q_k=p⁰_jp_k= 0 for all j6=k.

Summary

I The idea of canonical correlation is discussed.

I The expressions for the canonical correlations are derived.

I Some properties of canonical variables are discussed.

Summary

I The idea of canonical correlation is discussed.

I The expressions for the canonical correlations are derived.

I Some properties of canonical variables are discussed.

Summary

I The idea of canonical correlation is discussed.

I The expressions for the canonical correlations are derived.

I Some properties of canonical variables are discussed.