Linear Regression and the Line of Best Fit
Linear regression finds the straight line that best summarises a set of points. Getting the line is a button press. Knowing whether the line means anything is the part worth learning, and it is where most of the exam marks sit.
What "best fit" actually means
Given a scatter of points, infinitely many lines could be drawn through them. "Best" needs a definition, and the standard one is least squares.
For any candidate line, measure the vertical distance from each point to the line. Square each distance, then add them all up. The best-fit line is the one that makes that total as small as possible.
Two details in that definition are deliberate:
- Vertical distances, not perpendicular ones. Regression
assumes
xis known accurately and the uncertainty is iny. This is also why regressing y on x gives a different line from regressing x on y. - Squared, not absolute. Squaring stops errors above and below cancelling, and makes one large miss count for more than several small ones. That is a choice, and it is why a single outlier can drag the whole line.
Running a regression, step by step
- Open the calculator and switch to the Data tab.
- Press Paste data and paste two columns straight from a spreadsheet, or press Add row and type the pairs in. Tabs, commas and spaces all work as separators, and a header row is ignored.
- Choose Linear from the model list.
- Read off the equation, R squared, and r, which appears for linear fits.
- Press Plot this fit to draw the line over your points.
The fitted equation is added as an ordinary expression, so you can edit it,
change its colour, or evaluate it at a particular x by hovering
over the curve.
Reading the output
A linear fit reports y = mx + b. Both numbers have meaning.
- The slope m is the change in
yfor each one-unit increase inx. Ifxis hours studied andyis exam score, a slope of 4.2 says each extra hour is associated with 4.2 more marks. - The intercept b is the predicted
ywhenxis zero. Sometimes that is meaningful, often it is not. If your data covers heights from 150 to 190 cm, the intercept describes someone of height zero and should not be interpreted at all.
r and R squared, properly explained
The correlation coefficient r
r runs from -1 to 1 and measures how close the points lie to a
straight line.
r = 1: perfectly on an upward line.r = -1: perfectly on a downward line.r = 0: no linear relationship at all.
The sign carries the direction. An r of -0.9 is exactly as
strong a relationship as +0.9, just the other way round.
The coefficient of determination R squared
For a linear fit, R² really is r squared. It runs
from 0 to 1 and answers a specific question: what proportion of the
variation in y does the model account for?
R² = 0.87 means 87% of the variation in y is
explained by the linear relationship with x, and the remaining
13% is something else — other factors, measurement error, or randomness.
Unlike r, R² is defined for every model family, so
it is what you use to compare a quadratic fit against an exponential one on
the same data.
Careful with that comparison. Adding parameters can only
ever increase R², so a quartic will always score at least as
well as a linear fit on the same points. That does not make it a better
model. Four points can be fitted perfectly by a cubic, giving
R² = 1 and telling you nothing whatsoever.
When a line is the wrong model
A high R² does not confirm that a line was the right choice.
Always look at the plot.
The tell is the pattern of residuals. If the points sit above the line at both ends and below it in the middle, the relationship is curved and a different model will do better. Try exponential, power or quadratic from the same menu and compare.
The calculator offers eight families:
| Model | Shape | Typical use |
|---|---|---|
| Linear | y = mx + b | Constant rate of change |
| Quadratic | y = ax² + bx + c | One turning point, such as projectile height |
| Cubic, quartic | Higher polynomials | Several turning points; easy to overfit |
| Exponential | y = abˣ | Constant percentage growth or decay |
| Logarithmic | y = a + b ln x | Rapid early change that flattens off |
| Power | y = axᵇ | Scaling relationships in physics and biology |
| Sinusoidal | y = a sin(bx + c) + d | Anything seasonal or periodic |
Three of these — exponential, logarithmic and power — require data in a
valid range. An exponential fit needs positive y values,
because it works by fitting a line to the logarithms. Where the data cannot
support the model, the calculator says so rather than drawing a curve that
would mislead you.
Four traps to avoid
1. Treating correlation as causation
The classic example: ice cream sales correlate strongly with drowning
deaths. Neither causes the other; hot weather drives both. A regression
cannot distinguish that from a genuine causal link, and no value of
R² ever will.
2. Extrapolating past your data
A model fitted to temperatures between 10 and 30 degrees says nothing reliable about -40. Predicting inside the measured range is reasonable; outside it you are assuming the pattern continues, and the data gives you no grounds for that.
3. Letting one outlier steer the line
Because least squares penalises large errors disproportionately, a single stray point can pull the line noticeably. Investigate outliers before deciding what to do. If it is a transcription error, fix it. If it is real, it stays, and it may be the most interesting thing in the dataset.
4. Overfitting with a higher-degree model
A quartic through five points will pass through all of them exactly.
R² = 1, and the model has learned the noise rather than the
pattern. It will predict badly on anything new. Prefer the simplest model
that describes the data adequately.
Questions
What is the difference between r and R squared?
r is the correlation coefficient, running from −1 to 1, and its sign tells you the direction of the relationship. R squared is literally r squared for a linear fit, running from 0 to 1, and it tells you the proportion of the variation in y that the model accounts for. r carries direction; R squared carries strength only.
Is an R squared of 0.8 good?
It depends entirely on the field. In a physics experiment, 0.8 would suggest something went wrong. In social science or economics, 0.8 would be unusually strong. There is no universal threshold, which is why quoting one without context is meaningless.
Does a high R squared mean one variable causes the other?
No, and this is the most important caveat in the whole topic. Regression measures association. Two variables can move together because one causes the other, because both are driven by something else, or by coincidence. The statistics cannot tell you which.
What is the least squares method?
It picks the line that makes the sum of the squared vertical distances from the points to the line as small as possible. Squaring means errors above and below do not cancel out, and that large misses count disproportionately.
Can I use a regression line to predict beyond my data?
Cautiously at best. Predicting inside the range you measured is interpolation and is usually reasonable. Predicting outside it is extrapolation, and it assumes the pattern continues, which nothing in the data supports.