Logistic Regression Intuition

Concept

Despite its name, logistic regression is normally a classification model. Like linear regression, it forms a weighted combination of features. Unlike linear regression, it transforms that combination so the output lies between 0 and 1 and can be interpreted as a probability-like estimate for a class.

From a Linear Score to a Probability

A plain linear equation can produce any number, including -4 or 1.7, which is unsuitable as a churn probability. Logistic regression first builds a score:

z = b0 + b1 * x1 + b2 * x2 + ...

It then applies the sigmoid function:

p = 1 / (1 + e^(-z))

You do not need to calculate this by hand. Intuitively, a very negative score becomes a probability near 0, a score near zero becomes near 0.5, and a very positive score becomes near 1. The smooth curve lets the model represent increasing or decreasing class likelihood without producing impossible probabilities.

Example: Churn

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = customers[["tenure_months", "monthly_charges", "support_calls"]]
y = customers["churn"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42
)

model = LogisticRegression()
model.fit(X_train, y_train)

predicted_classes = model.predict(X_test)
predicted_probabilities = model.predict_proba(X_test)

fit() learns from historical features and known churn labels. predict() returns the final class under the model's current decision rule. predict_proba() returns one probability-like value for each class. For a binary classifier, the second column commonly corresponds to the class labeled 1, but always check model.classes_ rather than assuming column order.

Coefficients: A Careful Intuition

With other included features fixed, a positive coefficient pushes the model's score, and therefore its estimated positive-class probability, upward. A negative coefficient pushes it downward. The probability change is not a fixed amount for every row because the sigmoid curve is nonlinear. Coefficients do not prove that changing a feature causes churn.

Why the Name Is Confusing

The historical name comes from modeling a transformed probability-related quantity, not from predicting a continuous target. The practical rule is simpler: LogisticRegression is used to classify observations, while LinearRegression estimates numeric values.

Best Practices

Inspect model.classes_, keep X/y timing valid, and use held-out data. Treat probabilities as model estimates that need business context and evaluation. Later lessons explain threshold choice and class-specific metrics.

Interview Perspective

Question: Why is logistic regression used for classification? Answer: it maps a weighted feature score through a sigmoid into a 0-to-1 probability-like output used for class decisions. What the interviewer is testing: whether you distinguish the model's name from its task. Follow-up: how does predict_proba() differ from predict()?

Practice Questions

1. Why is an unconstrained linear output unsuitable as a probability?
2. What does a positive logistic-regression coefficient generally do to the positive-class score?
3. Which method returns class labels: predict() or predict_proba()?
4. Why should you inspect model.classes_ before selecting a probability column?
5. Does a churn probability of 0.8 prove the customer will churn? Explain.

Quick Quiz

1. What range does the sigmoid produce? Answer: values between 0 and 1.
2. Is LogisticRegression usually a regression task? Answer: no, it is commonly a classifier.
3. Does a positive coefficient prove causation? Answer: no.

Next Lesson

Next, turn class probabilities into decisions with thresholds and decision boundaries.