Supervised Learning: What It Is and How It Works

From image recognition to spam filtering, discover how supervised learning powers many of the AI applications we encounter daily in this informative guide.

Table of contents

• What is supervised learning?
• Supervised vs. unsupervised learning
• How supervised learning works
• Types of supervised learning
• Applications of supervised learning
• Advantages of supervised learning
• Disadvantages of supervised learning

What is supervised learning?

Supervised learning is a type of machine learning (ML) that trains models using data labeled with the correct answer. The term supervised means these labels provide clear guidance on the relationship between inputs and outputs. This process helps the model make accurate predictions on new, unseen data.

Machine learning is a subset of artificial intelligence (AI) that uses data and statistical methods to build models that mimic human reasoning rather than relying on hard-coded instructions. Supervised learning takes a guided, data-driven approach to identifying patterns and relationships in labeled datasets. It extrapolates from its evaluations to predict outcomes for new, unseen data. It learns by comparing its predictions against the known labels and adjusting its model to minimize errors.

Supervised vs. unsupervised learning

In contrast to supervised learning, which uses labeled data, unsupervised learning finds patterns in unlabeled data.

Without the “supervision” provided by explicit right answers in the training data, unsupervised learning treats everything it sees as data to analyze for patterns and groupings. The three main types are:

• Clustering: This technique groups data points that are most adjacent to each other. It is useful for customer segmentation or document sorting.
• Association: Determining when things tend to co-occur, most notably to co-locate items frequently bought together or suggest what to stream next.
• Dimensionality reduction: Shrinking datasets to be easier to process while preserving all or most of the details.

On the other hand, supervised learning makes sense when you want the model to make decisions. Major applications include:

• Yes or no decisions: Marking data as either one class or another. Often used for filtering like spam or fraud detection.
• Classification: Figuring out which of several classes something belongs to, such as identifying objects within an image or recognizing speech.
• Regression: Predicting continuous values based on historical data, such as forecasting house prices or weather conditions.

Other types of ML sit between these two: semi-supervised, reinforcement, and self-supervised learning.

How supervised learning works

Supervised learning involves a structured process of choosing and formatting data, running the model, and testing its performance.

Here’s a brief overview of the supervised learning process:

1 Labeling: Labeled data is essential for learning the correct association between inputs and outputs. For instance, if you’re creating a model to analyze sentiment in product reviews, start by having human evaluators read the reviews and mark them as positive, negative, or neutral.

2 Data collection and cleaning: Ensure your training data is comprehensive and representative. Clean the data by removing duplicates, correcting errors, and handling any missing values to prepare it for analysis.

3 Feature selection and extraction: Identify and select the most influential attributes, making the model more efficient and effective. This step may also involve creating new features from existing ones to better capture the underlying patterns in the data, such as converting date of birth to age.

4 Data splitting: Divide the dataset into training and testing sets. Use the training set to train the model, and the testing set to see how well it generalizes to new, unseen data.

5 Algorithm selection: Choose a supervised learning algorithm based on the task and data characteristics. You can also run and compare multiple algorithms to find the best one.

6 Model training: Train the model using the data to improve its predictive accuracy. During this phase, the model learns the relationship between inputs and outputs by iteratively minimizing the error between its predictions and the actual labels provided in the training data. Depending on the algorithm’s complexity and the dataset’s size, this could take seconds to days.

7 Model evaluation: Evaluating the model’s performance ensures that it produces reliable and accurate predictions on new data. This is a key difference from unsupervised learning: Since you know the expected output, you can evaluate how well the model performed.

8 Model tuning: Adjust and retrain the model’s parameters to fine-tune performance. This iterative process, called hyperparameter tuning, aims to optimize the model and prevent issues like overfitting. This process should be repeated after each adjustment.

9 Deployment and monitoring: Deploy the trained model to make predictions on new data in a real-world setting. For example, deploy the trained spam detection model to filter emails, monitor its performance, and adjust as needed.

10 Fine-tuning over time: As you gather more real-world data, continue to train the model to become more accurate and relevant.

Types of supervised learning

There are two main types of supervised learning: classification and regression. Each type has its own sub-types and specific use cases. Let’s explore them in more detail:

Classification

Classification involves predicting which category or class an input belongs to. Various sub-types and concepts are used to handle different classification problems. Here are some popular types:

• Binary classification: The model predicts one of two possible classes. This is useful when the outcome is binary, meaning there are only two possible states or categories. This approach is used in decisions where a clear distinction is needed.
• Multi-class classification: Like binary, but with more than two choices for which there is only one right answer. This approach is used when there are multiple categories that an input can belong to.
• Multi-label classification: Each input can belong to multiple classes simultaneously. Unlike binary or multi-class classification, where each input is assigned to a single class, multi-label classification allows for assigning multiple labels to a single input. This is a more complex analysis because rather than just choosing whichever class the input is most likely to belong to, you need to decide a probability threshold for inclusion.
• Logistic regression: An application of regression (see below) to binary classification. This approach can tell you the confidence of its prediction rather than a simple this-or-that.

There are several ways to measure the quality of a classification model, including:

• Accuracy: How many of the total predictions were correct?
• Precision: How many of the positives are actually positive?
• Recall: How many of the actual positives did it mark as positive?
• F1 score: On a scale of 0% to 100%, how well does the model balance precision and recall?

Regression

Regression involves predicting a continuous value based on input features, outputting a number that can also be called a prediction. Various types of regression models are used to capture the relationships between these input features and the continuous output. Here are some popular types:

• Linear regression: Models the relationship between the input features and the output as a straight line. The model assumes a linear relationship between the dependent variable (the output) and the independent variables (the inputs). The goal is to find the best-fitting line through the data points that minimizes the difference between the predicted and actual values.
• Polynomial regression: More complex than linear regression because it uses polynomials such as squared and cubed to capture more complex relationships between the input and output variables. The model can fit nonlinear data by using these higher-order terms.
• Ridge and lasso regression: Addresses the problem of overfitting, which is the tendency of a model to read too much into the data it’s trained on at the expense of generalizing. Ridge regression reduces the model’s sensitivity to small details, while lasso regression eliminates less important features from consideration.

Most measurements of regression quality have to do with how far off the predictions are from the actual values. The questions they answer are:

• Mean absolute error: On average, how far off are the predictions from the actual values?
• Mean squared error: How much do the errors grow when larger errors are more significant?
• Root mean squared error: How much do large errors cause predictions to deviate from actual values?
• R-squared: How well does the regression fit the data?

Applications of supervised learning

Supervised learning has a wide range of applications across various industries. Here are some common examples:

• Spam detection: Email services use binary classification to decide whether an email should hit your inbox or be routed to spam. They continually improve in response to people marking emails in the spam folder as not spam, and vice versa.
• Image recognition: Models are trained on labeled images to recognize and categorize objects. Examples include Apple’s Face ID feature, which unlocks your tablet or mobile device, optical character recognition (OCR) for turning printed words into digital text, and object detection for self-driving cars.
• Medical diagnosis: Supervised models can predict diseases and suggest potential diagnoses using patient data and medical records. For instance, models can be trained to recognize cancerous tumors in MRIs or develop diabetes management plans.
• Fraud detection: Financial institutions use supervised learning to identify fraudulent transactions by analyzing patterns in labeled transaction data.
• Sentiment analysis: Whether measuring positive or negative reactions or emotions such as happiness or disgust, manually tagged datasets inform models to interpret input such as social media posts, product reviews, or survey results.
• Predictive maintenance: Based on historical performance data and environmental factors, models can predict when machines are likely to fail so they can be repaired or replaced before they do.

Advantages of supervised learning

• Accurate and predictable. Assuming they’ve been given good data, supervised learning models tend to be more accurate than other machine learning methods. Simpler models are typically deterministic, meaning a given input will always produce the same output.
• Clear objective. Thanks to supervision, you know what your model is trying to accomplish. This is a clear contrast to unsupervised and self-supervised learning.
• Easy to evaluate. There are several quality measures at your disposal for judging the accuracy of both classification and regression models.
• Interpretable. Supervised models use techniques, such as regressions and decision trees, that are relatively straightforward for data scientists to understand. Interpretability improves decision-makers’ confidence, especially in high-impact settings and regulated industries.

Disadvantages of supervised learning

• Requires labeled data. Your data has to have clear inputs and labels. This is often a challenge for classification training, with many thousands (if not millions) of people employed to annotate data manually.
• Errors and inconsistent judgment in training data. With human labeling comes human fallacies, such as errors, typos, and different opinions. The latter is a particularly challenging aspect of sentiment analysis; high-quality sentiment training data typically requires multiple people to evaluate a given data point with a result recorded only if there’s agreement.
• Overfitting. Often a model will come up with calculations that work very well for the training data but poorly with data it hasn’t yet seen. A careful trainer will always look for overfitting and use techniques to reduce the impact.
• Restricted to known patterns. If your stock price prediction model is based only on data from a bull market, it won’t be very accurate once a bear market hits. Accordingly, be sensitive to the limitations of the data you’ve shown your model, and consider whether to find training data that will expose it to more circumstances or simply ignore its output.