Regression vs Classification

Understanding Regression vs Classification

Regression and classification are supervised learning tasks: each training example pairs input features with an observed target. Features describe information available when making a prediction; the target is what you want to estimate. The distinction depends on the target’s meaning, not whether its stored values happen to be numbers. Predicting delivery duration in minutes is regression. Predicting whether a delivery will be late is classification, even when the labels are encoded as 0 and 1.

Regression estimates a numerical quantity whose differences have practical meaning. An error of ten minutes is larger than an error of two minutes. Classification predicts membership in defined categories, such as on time, delayed, or cancelled. Category codes do not imply numerical distances: assigning cancelled the code 2 does not make it twice as delayed as category 1. Both tasks can use the same input features, but they answer different questions.

Many classifiers also produce estimated class probabilities. A late-delivery probability of 0.70 is not a predicted delay of 0.70 minutes; it expresses estimated uncertainty about category membership. A decision threshold converts a probability into an action or label. Lowering the threshold for issuing delay warnings generally catches more late deliveries but also produces more false alarms. Despite its name, logistic regression is commonly used for classification. The scikit-learn guide separately covers classification and regression variants of several algorithm families.

Evaluation must match the question. Mean absolute error, or MAE, averages absolute numerical errors and retains the target’s units. Root mean squared error, or RMSE, squares errors before averaging and taking a square root, giving large errors more influence. Classification accuracy measures the fraction of correct labels. Precision asks how many predicted positives were actually positive; recall asks how many actual positives were detected. These summarize different consequences, not interchangeable definitions of quality.

Task choice also involves information loss and assumptions. Turning minutes into late/not-late labels hides how late each delivery was. Ordered categories, such as one-to-five-star ratings, preserve rank without necessarily having equal spacing. Treating them as regression assumes meaningful numerical gaps; ordinary classification ignores their ordering unless explicitly accommodated. Neither task, by itself, establishes that changing a feature causes an outcome.

Practical walkthrough

Begin with the decision the prediction will support. A scheduling system might need expected delivery minutes to allocate capacity. A notification system might instead need the chance of exceeding a promised duration. Define the target, units, positive class, and prediction time before choosing an estimator. Here, “late” means strictly more than 30 minutes, so a delivery lasting exactly 30 minutes is not late.

Next, audit the available columns. Distance and departure time might be known at dispatch; recorded arrival time is not. Exclude information that becomes available only after the outcome. Inspect missing values and category meanings rather than trusting column types. The pandas guide’s sections on selection, missing data, and categorical data provide relevant tools for these checks.

Separate training data from evaluation data before learning transformations. Fit imputation, encoding, and scaling on training data only, then apply those fitted transformations unchanged to held-out rows. For future-delivery predictions, use a time-respecting split; keep related records together when overlap would leak information. Use validation data for model and threshold choices, reserving the test set for final assessment.

Start with a simple baseline, such as a training-set median for MAE or the most frequent training class for accuracy. Compare models against that baseline using metrics tied to operational costs. Inspect individual errors as well as averages: missing a severe delay may matter more than sending an unnecessary warning.

Finally, run the example below. It evaluates fixed illustrative predictions rather than training a model, isolating how one set of outcomes supports two different evaluation questions.

Worked example

from math import sqrt

# Fixed illustrative held-out outcomes and predictions; no fitting occurs.
actual_minutes = [20, 25, 35, 40]
predicted_minutes = [22, 29, 31, 38]
late_probabilities = [0.20, 0.60, 0.40, 0.80]

errors = [p - y for y, p in zip(actual_minutes, predicted_minutes)]
mae = sum(abs(e) for e in errors) / len(errors)
rmse = sqrt(sum(e * e for e in errors) / len(errors))

actual_late = [minutes > 30 for minutes in actual_minutes]
predicted_late = [probability >= 0.50 for probability in late_probabilities]

tp = sum(y and p for y, p in zip(actual_late, predicted_late))
fp = sum(not y and p for y, p in zip(actual_late, predicted_late))
fn = sum(y and not p for y, p in zip(actual_late, predicted_late))
accuracy = sum(y == p for y, p in zip(actual_late, predicted_late)) / len(actual_late)
precision = tp / (tp + fp) if tp + fp else None
recall = tp / (tp + fn) if tp + fn else None

print(f'MAE: {mae:.2f} minutes')
print(f'RMSE: {rmse:.2f} minutes')
print('Actual late:', [int(value) for value in actual_late])
print('Predicted late:', [int(value) for value in predicted_late])
print(f'Accuracy: {accuracy:.2f}')
print('Precision:', 'undefined' if precision is None else f'{precision:.2f}')
print('Recall:', 'undefined' if recall is None else f'{recall:.2f}')

Expected output

MAE: 3.00 minutes
RMSE: 3.16 minutes
Actual late: [0, 0, 1, 1]
Predicted late: [0, 1, 0, 1]
Accuracy: 0.50
Precision: 0.50
Recall: 0.50

The absolute duration errors are 2, 4, 4, and 2 minutes, giving MAE 3.00. RMSE is the square root of 10, displayed as 3.16. The classification predictions are separate illustrative estimates, not probabilities derived from the duration predictions. At the 0.50 probability threshold, one on-time delivery triggers a false alarm and one late delivery is missed. Two of four labels are correct; one of two warnings is correct; one of two late deliveries is detected. These four invented cases demonstrate arithmetic, not real-world performance. Neither the probabilities’ calibration nor a model’s reliability can be established from this tiny example alone.

Common mistakes and how to avoid them

• Choosing regression because category labels are integers.

  Check what subtraction means. Differences between arbitrary category codes are meaningless; use classification and document the label mapping.

• Reporting only accuracy when late deliveries are rare.

  Compare against a majority-class baseline and inspect false positives, false negatives, precision, and recall. High accuracy can coexist with missing every late delivery.

• Selecting a warning threshold using the final test set.

  Choose thresholds on validation data according to error costs. Reusing test outcomes for tuning makes the reported evaluation optimistic.

• Treating small regression error as proof of useful warnings.

  Evaluate the downstream labels separately. A small error near the late boundary can change a warning, while a larger error elsewhere may not.

Review questions

Further reading

• pandas user guide
• scikit-learn user guide