Least Squares Regression Tutorial

Where does the name "Least Squares Regression" come from?
The method of Least Squares Regression, as we said before, is a method in which we try to fit a line(or plane) through a set of points. The method of Least Squares Regression uses a formula which tries to minimize the distance squared of each point to the fitted line(or plane), hence "Least Squares".---- The original developer of the Least Squares Regression Method was in fact working on the problem we used in our example below(relating father's and son's heights). He noticed a tendency for the son of an extremely tall father to be a bit shorter; similarly a very short father tended to have a son who was a bit taller. He called this phenomenon "regression to normality." Thus "Regression" stuck with the method of "Least Squares" and we got Least Squares Regression!!

The mathmatical concept of Least Squares Regression is a procedure in which one attempts to fit a line through a set of data points. This line is then used to predict where a data point is most likely to be.

For example in Figure 1.1 we could say that the x-axis(horizontal axis) represents a father's height while the y-axis(vertical axis) represents his son's height. Each dot is a data point representing a father's height paired with his son's height. After collecting several sets of father/son heights(data points) we use the method of least squares to fit the best line through the points. Now that we have the line, if we know the father's height, we can predict the son's height or vice versa.

Similarly if we extend this data to also take into account the mother's height, we must plot this new data on a third axis(refered to as the z-axis, normally represented as coming out toward oneself), thus taking us into the third dimension. In 3-D we require a plane to fit the data, not just a line, hence our project.