Simple Linear Regression

Concept

Simple linear regression models a numeric target from one feature with a straight line. Its prediction is often written as:

y_hat = b0 + b1 * x

y_hat is the prediction, x is the feature, b0 is the intercept, and b1 is the slope or coefficient. The equation describes how the model converts one input value into an estimated target.

Intuition

Imagine plotting house size on the horizontal axis and price on the vertical axis. Actual houses do not land exactly on one line: location, condition, and negotiation also matter. Linear regression chooses a line that captures the broad direction in the observed examples. The line is a summary of a relationship, not a claim that every home follows a fixed price rule.

If the slope is positive, larger x values increase the prediction. If it is negative, larger x values decrease it. A slope of 150 for size_sqft means the fitted model adds about $150 to its prediction for each additional square foot, within the pattern and units of this dataset.

Example: Size and Price

import pandas as pd
from sklearn.linear_model import LinearRegression

homes = pd.DataFrame({
    "size_sqft": [900, 1200, 1500, 1800, 2100],
    "price": [185000, 235000, 290000, 340000, 395000],
})

X = homes[["size_sqft"]]
y = homes["price"]

model = LinearRegression()
model.fit(X, y)

new_home = pd.DataFrame({"size_sqft": [1600]})
predicted_price = model.predict(new_home)

print(model.coef_)
print(model.intercept_)
print(predicted_price)

X remains a two-dimensional table, even with one feature, because scikit-learn expects rows and columns. model.coef_ contains the fitted slope and model.intercept_ contains the point where the line meets the target axis.

Interpreting the Line

If the fitted slope were near 175, the model would increase its predicted price by roughly $175 for an additional square foot. The intercept is the prediction when size is zero. A zero-square-foot home is not meaningful, so the intercept may be necessary for the equation without being a useful business claim.

The fitted line is different from the observed points. A 1,600-square-foot home may sell above or below the line because of features this simple model does not include. That vertical difference is introduced formally as a residual in the next lesson.

Prediction Is Not Causation

A positive size coefficient does not prove that adding a square foot causes every home's price to increase by the fitted amount. Size is associated with location, property quality, and other factors. The coefficient describes the model's predictive relationship in this dataset, not a controlled causal experiment.

When to Use It

Simple linear regression is useful for a first look at one numeric relationship, a transparent baseline model, or an explanatory visualization. It can underfit when important features are missing or the relationship is curved. Multiple linear regression later adds more inputs.

Best Practices

Plot the observed values and fitted line, inspect the range of x, and state the target units when interpreting a slope. Use held-out data for evaluation, as Module 1 established. Treat coefficient interpretation as conditional on the data and model rather than a universal law.

Interview Perspective

Question: What does a linear-regression coefficient mean? Answer: holding the model form fixed, it is the change in the prediction associated with a one-unit feature increase. What the interviewer is testing: whether you can interpret a slope while avoiding causal overclaiming. Follow-up: when might the intercept be unhelpful?

Practice Questions

1. Is predicting weekly sales from advertising spend a regression task? Why?
2. Interpret a slope of -2.5 when x is delivery distance and y is an on-time score.
3. Why does X = df[["size_sqft"]] use double brackets?
4. A fitted line predicts $300,000 for a home that sold for $325,000. What additional information might explain the gap?
5. Why should you avoid interpreting a slope as causation?

Quick Quiz

1. What does y_hat represent? Answer: the model's predicted target value.
2. What does a positive slope do? Answer: it raises the prediction as x increases.
3. Does a straight line guarantee exact predictions? Answer: no.

Next Lesson

Next, measure the gaps between predictions and actual values with residuals and loss.