Linear regression with residual checks and a naive baseline

A linear regression can improve on a naive baseline and still miss important structure. Evaluate both predictive error and residual behavior before concluding that the fitted relationship adequately represents the task.

Our original synthetic dataset contains320 independent observations of a load index and an illustrative duration in minutes. The generator's mean is curved, 5+2*x+0.8*(x-5)**2, with noise whose standard deviation increases with load. These are authored teaching observations, not measured service times.

Fit the declared model and baseline

Use160 training rows,80 validation rows and80 test rows. The linear model includes only load index and an intercept. The baseline predicts the training target mean for every observation. The generator's known mean column is excluded from predictors.

The fitted intercept is approximately12.095890 and coefficient2.128093. The constant baseline is23.000506 minutes. Scikit-learn LinearRegression supplies the ordinary least-squares implementation.

The prespecified linear model beats the mean baseline on both reported metrics in this fixture. That comparison is useful, but it does not establish that the remaining error is acceptable for an actual decision.

Check the residuals with the right interpretation

Define residual as actual minus predicted. On training data, the residual sum is approximately4.4e-13 and its dot product with load approximately1.4e-12: effectively zero at ordinary numerical tolerance.

Those properties follow from the fitted least-squares normal equations with an intercept. They are not independent evidence that the model has captured all patterns. The training residual correlation with squared load is approximately0.224089, consistent with remaining curvature in this original generator.

import numpy as np
from linear_cases import regression

r = regression()
assert abs(r['residual_checks']['training_residual_sum'])<1e-9
assert abs(r['residual_checks']['training_residual_dot_load'])<1e-8
assert np.isclose(r['residual_checks']['training_residual_correlation_squared_load'],.2240886455376283)
test = r['results']['test']
assert np.isclose(test['linear_rmse'],6.290465782080996)
assert test['linear_rmse']<test['baseline_rmse']
assert np.isclose(test['residual_mean'],-2.1336279601092425)
print(r)

Run from the supervised-model lab. The recorded test residual mean is about-2.1336 minutes, indicating average overprediction in that finite test sample. A zero training residual mean does not force zero held-out bias.

Investigate patterns without declaring a cause prematurely

For a real dataset, inspect residuals against fitted values, important features and time. Curvature, changing spread, outliers or clustered errors can suggest a misspecified relationship, heterogeneous noise or data-quality problems.

The known synthetic generator lets us explain this example's curved mean and changing noise. In observational business data, a similar visual pattern would be a diagnostic clue, not proof of a specific causal mechanism.

Make the next comparison on development evidence

A quadratic feature is a plausible candidate here because the task is deliberately generated with curvature. Fit any expanded model on training data and choose among declared alternatives using validation evidence. Preserve the original test assessment rather than repeatedly trying features until that score improves.

Exercise: fit a training-only quadratic model and compare its validation residual pattern and errors with the linear reference. State how you would assess the selected procedure on untouched evidence and why extrapolation beyond the observed load range remains a separate concern.

NeuraPath's Data Science course connects regression formulas with evaluation. A complete result explains the baseline improvement and the residual evidence that limits the model's interpretation.

Continue learning

This article is part of the Supervised learning methods sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Continue with Logistic regression: probabilities, logits and decision thresholds.
• Then apply it in Ridge versus lasso when predictors are correlated.