Understanding Data & Features
A feature is an input measurement available when a prediction must be made. Suppose a delivery service wants to estimate trip duration before dispatch. Distance and parcel count could be features; actual trip duration is the target. Each row represents one delivery. A tracking identifier may help organize records without being a useful predictor. Therefore, not every column in a dataset belongs in the model's input.
For supervised learning, X conventionally holds features and y holds targets. X usually has shape (number of samples, number of features), while a single target per sample gives y one dimension. Their row order must agree: the first feature row must describe the delivery associated with the first target. Separating them explicitly reduces the chance of accidentally supplying the answer as an input.
Feature meaning matters as much as storage type. Distance is numerical, service category is categorical, and a tracking number is an identifier even if it contains only digits. Treating category codes as ordinary numbers can introduce an unjustified ordering. Many estimators need categories encoded into numeric inputs. Missing values also require a deliberate policy: an unknown distance is not necessarily a distance of zero.
Feature engineering creates representations from available measurements. Distance per parcel might express something useful about a route, but dividing two columns does not automatically improve predictions. The ratio assumes a meaningful denominator and needs a policy for zero parcels. Extra features can introduce noise, redundancy, and maintenance costs. Compare plausible candidates using validation data rather than assuming that more columns always help.
Preprocessing sometimes learns from data. Replacing missing distances with an average requires estimating that average; scaling similarly estimates quantities such as means and spreads. Learn those quantities only from training rows, then reuse them unchanged on validation or test rows. Otherwise, evaluation data influences the representation used for training. This is data leakage. The scikit-learn guide treats preprocessing, imputation, pipelines, and leakage prevention as connected concerns. A pipeline helps apply transformations consistently, but cannot rescue an unavailable-at-prediction-time feature. For example, delivery completion time would reveal information that does not exist at dispatch, regardless of how carefully the dataset was split.
Practical walkthrough
Start with a concrete prediction contract: estimate duration in minutes at dispatch, using distance in kilometers and parcel count. Assume both measurements are recorded at dispatch, except that distance can occasionally be missing. The example requires Python 3.10 or later and uses only the standard library. All records are created in memory; nothing is downloaded or loaded from disk.
First, reserve the last two records as held-out examples and use the first three for training. This fixed split makes the demonstration reproducible, not statistically convincing. In an actual delivery study, a chronological split may better represent predicting future deliveries. Repeated observations from the same route or driver can also require careful grouping to avoid overly optimistic evaluation.
Next, estimate the replacement distance from the observed training distances: two and six kilometers. Their mean is four. Do not include either held-out distance in this calculation. The function then converts each record into an ordered feature row, replacing a missing distance with that stored mean and adding distance per parcel. It rejects nonpositive parcel counts rather than silently producing an undefined ratio.
Build X and y separately for each partition. Print the feature names alongside both matrices so their column meanings remain inspectable. Notice that the feature function never accesses duration_min, even though the original records contain it.
Finally, check the held-out missing distance: it becomes four, not an average recomputed from held-out rows. Stop here intentionally. This exercise demonstrates representation and preprocessing boundaries; fitting a predictor or reporting accuracy on five invented deliveries would distract from those boundaries.
Worked example
from statistics import mean
records = [
{"distance_km": 2.0, "parcels": 1, "duration_min": 12},
{"distance_km": None, "parcels": 2, "duration_min": 20},
{"distance_km": 6.0, "parcels": 3, "duration_min": 30},
{"distance_km": None, "parcels": 4, "duration_min": 25},
{"distance_km": 10.0, "parcels": 2, "duration_min": 40},
]
train = records[:3]
test = records[3:]
observed = [r["distance_km"] for r in train
if r["distance_km"] is not None]
if not observed:
raise ValueError("Training data needs an observed distance")
fill_distance = mean(observed)
feature_names = ["distance_km", "parcels", "km_per_parcel"]
def features(rows, replacement):
result = []
for row in rows:
distance = row["distance_km"]
if distance is None:
distance = replacement
parcels = row["parcels"]
if parcels <= 0:
raise ValueError("Parcel count must be positive")
result.append([distance, parcels, distance / parcels])
return result
X_train = features(train, fill_distance)
y_train = [r["duration_min"] for r in train]
X_test = features(test, fill_distance)
y_test = [r["duration_min"] for r in test]
print("Features:", feature_names)
print("Training distance mean:", fill_distance)
print("X_train:", X_train)
print("y_train:", y_train)
print("X_test:", X_test)
print("y_test:", y_test)
Expected output
Features: ['distance_km', 'parcels', 'km_per_parcel']
Training distance mean: 4.0
X_train: [[2.0, 1, 2.0], [4.0, 2, 2.0], [6.0, 3, 2.0]]
y_train: [12, 20, 30]
X_test: [[4.0, 4, 1.0], [10.0, 2, 5.0]]
y_test: [25, 40]
The output contains three training rows and two held-out rows, each with three features. The learned replacement is (2.0 + 6.0) / 2 = 4.0. Consequently, the second training row uses four kilometers, giving 4.0 / 2 = 2.0 kilometers per parcel. The first held-out row also uses four kilometers, but its four parcels produce a ratio of 1.0. Targets remain separate lists in matching order. Mean replacement is simple, but it conceals uncertainty and can distort relationships. The ratio is constant across these training rows, so this tiny dataset provides no evidence that the engineered feature would help a predictor.
Common mistakes and how to avoid them
Including every stored column in X.
List inputs explicitly. Exclude the target and check when each measurement becomes available; post-delivery measurements cannot support a dispatch-time prediction.
Computing replacement values before splitting.
Split first, estimate replacements on training rows, and reuse them. During cross-validation, repeat this fitting separately inside each training fold.
Assuming a ratio must be useful.
Check units, denominator validity, and variation. Compare against a baseline using validation data; keep the final test set out of feature-selection decisions.
Changing feature order between training and prediction.
Use one shared transformation and fixed column order. A numeric matrix does not tell an estimator that distance and parcel count were accidentally swapped.
Review questions
Why is duration_min excluded from the feature function?
It is the answer being predicted and is unavailable at dispatch. Including it would leak the target rather than teach prediction from available inputs.
Why does the held-out distance of ten not affect the replacement mean?
The replacement is a learned preprocessing parameter. Held-out observations must not influence it, even though calculating a mean does not use target values.
Can engineered features be calculated without fitting?
Yes. A predetermined row-wise ratio needs no population estimate. Its inputs must still be available at prediction time, and its denominator must be valid.
Does a useful distance feature prove that distance causes longer trips?
No. Predictive associations may reflect route conditions or other variables. Establishing a causal effect requires assumptions and evidence beyond predictive performance.
