Xycoon logo
Maximum Likelihood
Home    Site Map    Site Search    Free Online Software    
horizontal divider
vertical whitespace

Maximum Likelihood Estimation (MLE) for Multiple Regression

[Home] [Up] [Introduction] [Criterion] [OLS] [Assumptions] [Inference] [Multiple Regression] [OLS] [MLE]


II.II.2 Maximum Likelihood Estimation (MLE) for Multiple Regression

MLE is needed when one introduces the following assumptions

(II.II.2-1)

(in this work we only focus on the use of MLE in cases where y and e are normally distributed).

The pdf of y is given by

(II.II.2-2)

and the log likelihood function

(II.II.2-3)

Maximizing the log likelihood function is (here) equivalent to minimizing the SSR

(II.II.2-4)

Therefore it follows

(II.II.2-5)

and of course

(II.II.2-6)

so that the ML estimator of ß is a BLUE and also a best unbiased estimator (for nonlinear models).

The ML estimator for the variance parameter however is biased

(II.II.2-7)

so that we have to use

(II.II.2-8)

The large sample properties of the ML estimator can be deduced on using a Taylor expansion of the likelihood around the true parameter value

(II.II.2-9)

This expression can be shown to be true under the so-called regularity condition which implies that the information matrix times 1/T converges to a probability matrix in the limit

(II.II.2-10)

Now it follows from (II.II.2-9) that

(II.II.2-11)

From (II.II.2-1)it can be seen that y is identically and independently distributed. Thus the pdf can be written as a derivative of the log likelihood

(II.II.2-12)

and from the derivation (c.q. proof) of the Cramér-Rao lower bound it follows that each of the T observations has a zero expected value and finite variance. Therefore

(II.II.2-13)

According to (II.II.2-10) and the central limit theorem it can be shown that

(II.II.2-14)

Using (II.II.2-11) and (II.II.2-14) it is easily derived that

(II.II.2-15)

Applying Cramér's theorem (I.VI-36) and (I.VI-37), proves that

(II.II.2-16)

(Q.E.D.) since

(II.II.2-17)

vertical whitespace




Home
Up
Introduction
Criterion
OLS
Assumptions
Inference
Multiple Regression
OLS
MLE
horizontal divider
horizontal divider

© 2000-2022 All rights reserved. All Photographs (jpg files) are the property of Corel Corporation, Microsoft and their licensors. We acquired a non-transferable license to use these pictures in this website.
The free use of the scientific content in this website is granted for non commercial use only. In any case, the source (url) should always be clearly displayed. Under no circumstances are you allowed to reproduce, copy or redistribute the design, layout, or any content of this website (for commercial use) including any materials contained herein without the express written permission.

Information provided on this web site is provided "AS IS" without warranty of any kind, either express or implied, including, without limitation, warranties of merchantability, fitness for a particular purpose, and noninfringement. We use reasonable efforts to include accurate and timely information and periodically updates the information without notice. However, we make no warranties or representations as to the accuracy or completeness of such information, and it assumes no liability or responsibility for errors or omissions in the content of this web site. Your use of this web site is AT YOUR OWN RISK. Under no circumstances and under no legal theory shall we be liable to you or any other person for any direct, indirect, special, incidental, exemplary, or consequential damages arising from your access to, or use of, this web site.

Contributions and Scientific Research: Prof. Dr. E. Borghers, Prof. Dr. P. Wessa
Please, cite this website when used in publications: Xycoon (or Authors), Statistics - Econometrics - Forecasting (Title), Office for Research Development and Education (Publisher), http://www.xycoon.com/ (URL), (access or printout date).

Comments, Feedback, Bugs, Errors | Privacy Policy