A line through the origin

Topic: Linear regression · more questions on this topic
‹ Normal equations for a straight line How much do the ads explain? ›

Show that the least-squares line through the origin, y=bx, has b=\sum x_iy_i/\sum x_i^2. Under the model Y_i=\beta x_i+\varepsilon_i with independent \operatorname{N}(0,\sigma^2) errors, find the mean and variance of \widehat\beta.

Worked answer (click to show)

L(b)=\sum(y_i-bx_i)^2 has derivative -2\sum x_i(y_i-bx_i), which is zero at b=\sum x_iy_i/\sum x_i^2. The second derivative 2\sum x_i^2 is positive, so this is the minimum.

The estimator is \widehat\beta=\sum x_iY_i/\sum x_i^2, a fixed linear combination of the Y_i. Its mean is \sum x_i(\beta x_i)/\sum x_i^2=\beta, so it is unbiased, and its variance is

\operatorname{Var}\widehat\beta=\frac{\sum x_i^2\sigma^2}{\left(\sum x_i^2\right)^2}=\frac{\sigma^2}{\sum x_i^2}.