Types of Machine Learning

Understanding Types of Machine Learning

Machine learning types differ primarily in the feedback available during learning, not in the programming language or model size. A feature is an input measurement, such as an email's length. A target is the answer a supervised model should predict. The same collection of emails can support different learning tasks depending on whether training receives targets, only features, or feedback from actions.

Supervised learning uses paired inputs and targets, commonly written X and y. Classification predicts a category, such as spam or legitimate; regression predicts a numerical quantity, such as delivery time. Many algorithms adjust parameters to reduce a loss: a numerical penalty for disagreement with training targets. Others store examples or calculate summaries. The objective is useful prediction on unseen examples, not merely memorizing training answers. This assumes training data adequately represent the conditions where predictions will be used.

Unsupervised learning uses inputs without supplied target answers for the task. Clustering groups examples according to a chosen similarity rule. For example, k-means alternates between assigning points to nearby centers and updating those centers to reduce squared distances. Dimensionality reduction builds a smaller representation; PCA identifies directions of greatest variance. These objectives are mathematical, not guarantees of meaningful categories. Feature scaling and algorithm assumptions strongly influence the structure discovered.

Reinforcement learning concerns an agent choosing actions in an environment. The agent observes a state, acts, receives a reward, and encounters a subsequent state. It learns a policy: a rule for selecting actions. The objective is expected cumulative reward, often discounting distant rewards. Unlike a supervised target, a reward need not identify the correct action. Delayed consequences complicate learning, while exploration trades immediate reward against information about alternatives. A poorly specified reward can encourage unwanted behavior.

These categories are not exhaustive or mutually exclusive within a system. Semi-supervised learning combines labeled and unlabeled training examples, relying on assumptions connecting input structure with labels. Self-supervised learning constructs prediction targets from the data themselves, such as hiding words and predicting them. The scikit-learn guide separates supervised methods, unsupervised methods, and evaluation; its organization reinforces that choosing a learning type does not remove the need to assess results.

Practical walkthrough

Start by describing the information available when learning happens. For an email application, ask whether people have supplied spam labels. If so, predicting those labels is supervised classification. If you only want groups of similar messages without existing categories, clustering is an unsupervised task. If an agent chooses sequential actions whose consequences generate rewards, formulate states, actions, transitions, and rewards before considering reinforcement learning.

Next, specify the output and learning rule. In the example below, each message has one artificial feature: a length measurement. Human-provided categories are A and B. We use a nearest-centroid classifier: calculate the mean feature value for each training category, then assign a new value to whichever mean is closest. Labels determine which observations contribute to each mean, making the procedure supervised even though prediction uses distances.

Keep evaluation examples separate before fitting anything. The code defines training and test collections explicitly and calculates centers from training values alone. No scaling is necessary for this single-feature demonstration. With several differently measured features, scaling may matter; learn scaling parameters only from training data and reuse them unchanged for validation and testing. During cross-validation, fit preprocessing separately inside each training fold.

Finally, inspect predictions against held-out targets. Select model settings using validation data, not repeated test inspection. This tiny example demonstrates mechanics rather than providing a credible performance estimate. Real email classification needs richer features, representative labels, and evaluation that reflects costly mistakes. Neither a successful prediction nor a discovered cluster establishes why an email has a particular property.

Worked example

from collections import defaultdict

# Artificial message-length measurements and supplied categories.
train_x = [2, 4, 8, 10]
train_y = ["A", "A", "B", "B"]
test_x = [5, 8]
test_y = ["A", "B"]

values_by_class = defaultdict(list)
for x, label in zip(train_x, train_y):
    values_by_class[label].append(x)

centers = {
    label: sum(values) / len(values)
    for label, values in values_by_class.items()
}

def predict(x):
    # Resolve equal distances alphabetically for reproducibility.
    return min(centers, key=lambda label: (abs(x - centers[label]), label))

for label in sorted(centers):
    print(f"Center {label}: {centers[label]:.1f}")

for x, actual in zip(test_x, test_y):
    print(f"Length {x}: predicted={predict(x)}, actual={actual}")

Expected output

Center A: 3.0
Center B: 9.0
Length 5: predicted=A, actual=A
Length 8: predicted=B, actual=B

Run this script with Python 3; it uses only the standard library. Category A's center is (2 + 4) / 2 = 3, while B's is (8 + 10) / 2 = 9. For length 5, distances are 2 and 4, so the classifier chooses A. For length 8, distances are 5 and 1, so it chooses B. The boundary is length 6; the explicit tie rule assigns that value to A. Both held-out predictions match their targets, but two constructed cases cannot establish generalization. Unlike k-means, this algorithm receives category memberships rather than discovering them.

Common mistakes and how to avoid them

• Calling any distance-based method unsupervised.

  Inspect the training inputs. Nearest-centroid classification uses known labels to construct centers; k-means learns assignments without target labels. Similar calculations can serve different learning types.

• Treating cluster numbers as verified categories.

  Cluster identifiers are arbitrary. Examine membership, sensitivity to scaling, and usefulness for the intended task before assigning names such as 'high risk' or 'loyal customer'.

• Fitting preprocessing on the entire dataset before evaluation.

  Split first. Fit imputers, scalers, and dimensionality reduction on training data only, even when they do not use labels. Apply the fitted transformations to held-out examples.

• Equating reward with a correct-action label.

  Specify what actions change and when rewards arrive. Evaluate cumulative consequences and safety constraints; maximizing an immediate reward can undermine the intended longer-term objective.

Review questions

Further reading

• scikit-learn user guide