Back to Projects
Unsupervised Learning

Customer Segmentation with Clustering

K-Means and hierarchical clustering of 68,300 retail customers, published as a learning exercise with its limitations stated.

Type
Unsupervised Learning
Domain
Marketing Analytics
Methods
K-Means, Hierarchical, PCA, t-SNE
Origin
Cambridge DS Career Accelerator coursework

The Challenge

K-Means and hierarchical clustering were tested on 68,300 retail customers; silhouette scores of about 0.25 showed the customer groups overlap heavily rather than forming distinct segments.

Marketing teams want customer groups they can target differently. The task was to build customer-level features from transaction data, choose a number of clusters and judge whether the resulting segments are distinct enough to act on.

Approach

01
Feature engineering
Aggregated 951,669 order lines to 68,300 customers across 47 countries: order frequency, recency, total profit, average unit cost and age.
02
Outlier removal
Isolation Forest removed 3,415 outlying customers (5%) before clustering.
03
Choosing k
Elbow method and silhouette scores for k = 2 to 10, then K-Means and agglomerative (hierarchical) clustering.
04
Visualisation
PCA and t-SNE projections to inspect how well the clusters separate.

Results

0.266
Best silhouette score (k = 2); k = 5 scored 0.255
61.0%
Variance explained by the first two principal components
66%
Share of customers in the largest K-Means cluster (k = 5)

The elbow plot suggested five clusters and the silhouette score preferred two, and neither shows clear separation. One cluster held about two-thirds of customers, so the five-cluster solution does not yet give segments a marketing team could target with confidence.

What I would fix: the profit ("CLV") feature was inflated by merging a customer-level value onto every order line before summing, and the final K-Means ran on unscaled features, so large-valued features dominated. Correcting both and re-running is the next step before naming any segments.

How it was tested

TEST REPORTTEST REPORTSEGMENTATIONSEGMENTATION
Checked
  • Elbow method and silhouette scores for k = 2 to 10, with K-Means and agglomerative clustering.
  • PCA and t-SNE projections used to inspect how well the clusters separate.
Found
  • The profit ("CLV") feature was inflated by merging a customer-level value onto every order line before summing.
  • The final K-Means ran on unscaled features, so large-valued features dominated.
Did not work
  • Silhouette scores of about 0.25 showed the groups overlap heavily; one cluster held about two-thirds of customers.
Limits
  • Both issues need correcting and a re-run before any segments are named.

Technology Stack

Python scikit-learn K-Means Agglomerative Clustering Isolation Forest PCA t-SNE pandas Matplotlib
Interested in this work or something similar?