Paper: Regression Analysis III Module: Linear Mixed Model

Subject: Statistics

Paper: Regression Analysis III Module: Linear Mixed Model

1 / 13

Development Team

Principal investigator: Dr. Bhaswati Ganguli,Professor, Department of Statistics, University of Calcutta

Paper co-ordinator: Dr. Bhaswati Ganguli,Professor, Department of Statistics, University of Calcutta

Content writer: Sayantee Jana, Graduate student, Department of Mathematics and Statistics, McMaster University Sujit Kumar Ray,Analytics professional, Kolkata

Content reviewer: Department of Statistics, University of Calcutta

2 / 13

Linear Mixed Model (LMM) :

I Linear Mixed Model : y

¯=xβ

¯+zu

¯+

¯ E(¯) =0

¯ andD(

¯) = Ω β

¯ = fixed effect u¯ = random effect

x ,z → Known design matrices.

u¯∼N(0

¯, D)

3 / 13

Linear Mixed Model (LMM) :

I E(y

¯

|u¯) =xβ

¯+zu

¯ V ar(y

¯

|u¯) =σ²In

E(y¯) =xβ

¯ V ar(y

¯) =σ²In+z⁰Dz [V ar(y

¯)depends only onz not onx or β if D( ¯

¯) =R, then V ar(y

¯) =R+z⁰Dz]

I advantages : ^q(q+1)₂ = no. of parameters. q << n

I disadvantages : dimension

4 / 13

Linear Mixed Model (LMM) :

I link : η=g(µ) η =xβ

¯+zu

¯

µ=g⁻¹(η) =g⁻¹(xβ

¯+zu

¯) E(y) =E(µ) =E[g⁻¹(xβ

¯+zu

¯)]

V ar(y) =V ar(µ) +a(φ)E[V(µ)]

5 / 13

Linear Mixed Model (LMM) :

I Example : loglink : g(µ) =log(µ)

Conditional distribution of Y|u∼P(µ)⇒V ar(Y|u) =µ η :g(µ)⇒log(µ) =xβ

¯+zu

¯

⇒µ=e^xβ

¯

+zu

¯ E(Y) =E[e^xβ

¯

+zu

¯]

=e^xβ

¯E(e^zu

¯)

=e^xβ

¯µu(z)

6 / 13

Best Linear Unbiased Estimator (BLUP)

I BLUP is a method of estimating random effects in a model.

I P in BLUP stands for ‘predictor’ because conventionally, estimators for random effects are called predictors and estimators of fixed effects are called estimators.

I BLUP is called ‘best’ because it has the minimum MSE among all linearly unbiased estimators.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I When the covariance of the random effects, u

¯ tends to zero Henderson’s mixed model equations tend formally to the GLSE for estimatingβ

¯ and u

¯and u

¯is regarded as fixed effects.

7 / 13

Best Linear Unbiased Estimator (BLUP)

I BLUP is a method of estimating random effects in a model.

I P in BLUP stands for ‘predictor’ because conventionally, estimators for random effects are called predictors and estimators of fixed effects are called estimators.

I BLUP is called ‘best’ because it has the minimum MSE among all linearly unbiased estimators.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I When the covariance of the random effects, u

¯ tends to zero Henderson’s mixed model equations tend formally to the GLSE for estimatingβ

¯ and u

¯and u

¯is regarded as fixed effects.

7 / 13

Best Linear Unbiased Estimator (BLUP)

I BLUP is a method of estimating random effects in a model.

I P in BLUP stands for ‘predictor’ because conventionally, estimators for random effects are called predictors and estimators of fixed effects are called estimators.

I BLUP is called ‘best’ because it has the minimum MSE among all linearly unbiased estimators.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I When the covariance of the random effects, u

¯ tends to zero Henderson’s mixed model equations tend formally to the GLSE for estimatingβ

¯ and u

¯and u

¯is regarded as fixed effects.

7 / 13

Best Linear Unbiased Estimator (BLUP)

I BLUP is a method of estimating random effects in a model.

I P in BLUP stands for ‘predictor’ because conventionally, estimators for random effects are called predictors and estimators of fixed effects are called estimators.

I BLUP is called ‘best’ because it has the minimum MSE among all linearly unbiased estimators.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I When the covariance of the random effects, u

¯ tends to zero Henderson’s mixed model equations tend formally to the GLSE for estimatingβ

¯ and u

¯and u

¯is regarded as fixed effects.

7 / 13

Best Linear Unbiased Estimator (BLUP)

I BLUP is a method of estimating random effects in a model.

I P in BLUP stands for ‘predictor’ because conventionally, estimators for random effects are called predictors and estimators of fixed effects are called estimators.

I BLUP is called ‘best’ because it has the minimum MSE among all linearly unbiased estimators.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I When the covariance of the random effects, u

¯ tends to zero Henderson’s mixed model equations tend formally to the GLSE for estimatingβ

¯ and u

¯and u

¯is regarded as fixed effects.

7 / 13

Methods of deriving BLUP

I BLUP can be derived in many different ways and it is robust.

I Here we introduce four methods of deriving BLUP.

I Henderson’s justification

I Bayesian derivation.

I ClassicalSchool.

I Goldberger’s derivation

8 / 13

Methods of deriving BLUP

I BLUP can be derived in many different ways and it is robust.

I Here we introduce four methods of deriving BLUP.

I Henderson’s justification

I Bayesian derivation.

I ClassicalSchool.

I Goldberger’s derivation

8 / 13

Methods of deriving BLUP

I BLUP can be derived in many different ways and it is robust.

I Here we introduce four methods of deriving BLUP.

I Henderson’s justification

I Bayesian derivation.

I ClassicalSchool.

I Goldberger’s derivation

8 / 13

Henderson’s justification

I He described the BLUPs as “joint maximum likelihood estimates”.

I He assumed that both u

¯ and

¯are normally distributed.

I He maximized the joint density of y

¯ and u

¯ with respect toβ and u ¯

¯to obtain the BLUPs

9 / 13

Henderson’s justification

I He described the BLUPs as “joint maximum likelihood estimates”.

I He assumed that both u

¯ and

¯are normally distributed.

I He maximized the joint density of y

¯ and u

¯ with respect toβ and u ¯

¯to obtain the BLUPs

9 / 13

Henderson’s justification

I He described the BLUPs as “joint maximum likelihood estimates”.

I He assumed that both u

¯ and

¯are normally distributed.

I He maximized the joint density of y

¯ and u

¯ with respect toβ and u ¯

¯to obtain the BLUPs

9 / 13

Bayesian derivation

I Assume thatβ

¯ has a uniform improper prior distribution.

I Also assume that both u

¯ has a prior distribution with mean 0 and positive variance independent ofβ

¯.

I Then the posterior mode is the BLUP.

10 / 13

Bayesian derivation

I Assume thatβ

¯ has a uniform improper prior distribution.

I Also assume that both u

¯ has a prior distribution with mean 0 and positive variance independent ofβ

¯.

I Then the posterior mode is the BLUP.

10 / 13

Bayesian derivation

I Assume thatβ

¯ has a uniform improper prior distribution.

I Also assume that both u

¯ has a prior distribution with mean 0 and positive variance independent ofβ

¯.

I Then the posterior mode is the BLUP.

10 / 13

Classical school

I This method uses the estimation of residuals form a simple normal model.

I Properties of the residuals as estimators of the unkown errors.

I linear

I unbiased

I possesses minimum MSE amongst the class of LUEs

11 / 13

Classical school

I This method uses the estimation of residuals form a simple normal model.

I Properties of the residuals as estimators of the unkown errors.

I linear

I unbiased

I possesses minimum MSE amongst the class of LUEs

11 / 13

Classical school

I This method uses the estimation of residuals form a simple normal model.

I Properties of the residuals as estimators of the unkown errors.

I linear

I unbiased

I possesses minimum MSE amongst the class of LUEs

11 / 13

Goldberger’s Derivation

I He assume that linear model - y=xβ

¯+ where the disturbances ¯

¯satisfies E(

¯) = 0 andV ar(

¯) =Ω.

I He derived the best linear unbiased predictor of the future observation given a new observable vector of regressors and unobservable prediction of disturbances.

I He was the first to coin the term.

I Henderson was the first to use the acronym BLUP in 1973.

12 / 13

Goldberger’s Derivation

I He assume that linear model - y=xβ

¯+ where the disturbances ¯

¯satisfies E(

¯) = 0 andV ar(

¯) =Ω.

I He derived the best linear unbiased predictor of the future observation given a new observable vector of regressors and unobservable prediction of disturbances.

I He was the first to coin the term.

I Henderson was the first to use the acronym BLUP in 1973.

12 / 13

Goldberger’s Derivation

I He assume that linear model - y=xβ

¯+ where the disturbances ¯

¯satisfies E(

¯) = 0 andV ar(

¯) =Ω.

I He derived the best linear unbiased predictor of the future observation given a new observable vector of regressors and unobservable prediction of disturbances.

I He was the first to coin the term.

I Henderson was the first to use the acronym BLUP in 1973.

12 / 13

Goldberger’s Derivation

I He assume that linear model - y=xβ

¯+ where the disturbances ¯

¯satisfies E(

¯) = 0 andV ar(

¯) =Ω.

I He derived the best linear unbiased predictor of the future observation given a new observable vector of regressors and unobservable prediction of disturbances.

I He was the first to coin the term.

I Henderson was the first to use the acronym BLUP in 1973.

12 / 13

Summary

I Mixed models have a fixed effect and a random effect.

I The estimator for random effects in a model is called BLUP.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I There are several methods of deriving BLUP.

I For further details the students are refered to Robinson’s paper “The BLUP is a good thing”.

13 / 13

Summary

I Mixed models have a fixed effect and a random effect.

I The estimator for random effects in a model is called BLUP.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I There are several methods of deriving BLUP.

I For further details the students are refered to Robinson’s paper “The BLUP is a good thing”.

13 / 13

Summary

I Mixed models have a fixed effect and a random effect.

I The estimator for random effects in a model is called BLUP.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I There are several methods of deriving BLUP.

I For further details the students are refered to Robinson’s paper “The BLUP is a good thing”.

13 / 13

Summary

I Mixed models have a fixed effect and a random effect.

I The estimator for random effects in a model is called BLUP.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I There are several methods of deriving BLUP.

I For further details the students are refered to Robinson’s paper “The BLUP is a good thing”.

13 / 13

Summary

I Mixed models have a fixed effect and a random effect.

I The estimator for random effects in a model is called BLUP.

I The BLUPs are the solutions of the mixed model equations by Henderson.

I There are several methods of deriving BLUP.

I For further details the students are refered to Robinson’s paper “The BLUP is a good thing”.

13 / 13