scikit-learn Introduction

Understanding scikit-learn Introduction

scikit-learn provides algorithms and supporting tools for learning patterns from data. Its user guide separates supervised learning, unsupervised learning, preprocessing, and model evaluation. These pieces belong together: a useful workflow must represent inputs consistently, learn only from permitted training data, and evaluate predictions on unseen observations. Installing the package uses the name `scikit-learn`; importing its modules uses `sklearn`.

In supervised learning, `X` contains input features and `y` contains target values. Usually, `X` has shape `(n_samples, n_features)`: each row is one observation and each column is one measured property. For ordinary single-target tasks, `y` is one-dimensional, with one entry per row. Classification predicts discrete labels; regression predicts numerical quantities. Feature columns must retain the same meaning and order during training and prediction.

The central abstraction is an estimator, an object configured with constructor arguments and trained through `fit`. These configuration choices are hyperparameters: they control learning rather than being learned directly from examples. After fitting, learned attributes commonly end in an underscore, such as a scaler’s `mean_`. Calling `fit` generally changes the estimator’s stored state and returns the estimator itself, not predictions or an evaluation result.

The interface is consistent, but not every estimator implements the same three methods. Predictors provide `predict`; transformers provide `transform`, often alongside `fit_transform`. Some classifiers expose `predict_proba`, but this is not universal. Likewise, `score` is estimator-specific: classifiers generally return accuracy, while regressors generally return the coefficient of determination, R². A score of 0.85 therefore has no clear interpretation without knowing the estimator, metric, and evaluation data.

A `Pipeline` connects transformations to a final estimator. During fitting, it learns preprocessing from training inputs and passes transformed inputs onward. During prediction, it reuses those learned transformations without fitting again. This prevents a common source of leakage: allowing held-out data to influence preprocessing statistics. Pipelines do not repair every leakage problem, however; a feature containing information unavailable at prediction time remains invalid.

Choosing an algorithm still requires assumptions. Nearest-neighbor classification, used below, assumes nearby feature vectors tend to share labels. Its behavior depends on distance, feature scaling, and neighborhood size. A shared interface makes alternatives easier to compare; it does not make their assumptions interchangeable.

Practical walkthrough

Begin by defining what one row represents and what must be predicted. Here, each observation has one numerical measurement and a binary label. The values are deliberately constructed for inspecting the workflow, not for estimating real-world performance. The example assumes scikit-learn is already installed and requires no files or downloads.

Reserve test observations before learning anything from the inputs. The code explicitly supplies separate training and test lists so the separation is visible and deterministic. In real work, choose a split that reflects deployment: unrelated observations may permit a random split, while repeated subjects or chronological records require group-aware or time-aware separation.

Construct a pipeline with `StandardScaler` followed by `KNeighborsClassifier(n_neighbors=1)`. The scaler learns each training feature’s mean and standard deviation, then subtracts that mean and divides by that standard deviation. With only one feature, this common rescaling does not change neighbor ordering. With multiple features, scaling can substantially change distances; treating standardized features comparably is itself a modeling choice.

Call `fit` using training inputs and labels only. Nearest-neighbor fitting primarily stores training examples for later searches rather than estimating a linear decision boundary. With one neighbor, prediction finds the nearest stored observation and returns its label. Larger neighborhoods aggregate more labels, potentially reducing sensitivity to individual examples but smoothing away local distinctions.

Finally, predict the held-out labels and calculate accuracy explicitly. Inspect the scaler’s learned mean to verify which data informed preprocessing. Do not change the neighborhood size repeatedly based on these test results. Select hyperparameters using validation or cross-validation within the training data, then evaluate the selected pipeline on the untouched test set.

Worked example

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

X_train = [[0.0], [2.0], [8.0], [10.0]]
y_train = [0, 0, 1, 1]
X_test = [[0.5], [9.5], [7.0]]
y_test = [0, 1, 0]

model = Pipeline([
    ("scale", StandardScaler()),
    ("classify", KNeighborsClassifier(n_neighbors=1)),
])
model.fit(X_train, y_train)
predictions = model.predict(X_test)

print(f"Training mean: {model.named_steps['scale'].mean_[0]:.1f}")
print("Predictions:", predictions.tolist())
print(f"Test accuracy: {accuracy_score(y_test, predictions):.3f}")

Expected output

Training mean: 5.0
Predictions: [0, 1, 1]
Test accuracy: 0.667

The training measurements average to 5.0; test measurements do not contribute to that learned value. The nearest training measurements to 0.5, 9.5, and 7.0 are respectively 0.0, 10.0, and 8.0. Their stored labels produce predictions 0, 1, and 1. Two predictions match the supplied test labels, so accuracy is 2/3, displayed as 0.667. The final observation demonstrates that proximity need not imply a matching label. This tiny, constructed test set illustrates mechanics only: it cannot establish reliable generalization, and neither its predictions nor its errors demonstrate a causal relationship between the measurement and the label.

Common mistakes and how to avoid them

• Passing a flat list as a single feature column.

  Use nested rows such as [[0.0], [2.0]]. A single observation with two features instead looks like [[0.0, 2.0]]. Check both dimensions before fitting.

• Scaling all observations before separating training and test data.

  Split first, then fit preprocessing on training data only. Pass the whole pipeline into cross-validation so each fold learns its own preprocessing.

• Treating score as a universal percentage of correctness.

  Check the estimator’s score definition. Choose explicit metrics suitable for the task; accuracy alone can conceal poor performance on a rare class.

• Assuming excellent training accuracy proves useful learning.

  A one-neighbor classifier can reproduce distinct training examples’ labels. Evaluate unseen observations and compare against a simple baseline before judging predictive usefulness.

Review questions

Further reading

• scikit-learn user guide