Cluster Analysis on Real Customer Data: K-Means vs Hierarchical

A food wholesaler, 440 clients, and one year of spending per client across six product lines. There are no segments and no labels, and the job is to find the customer groups anyway.

The file also has a Channel column (hotel/restaurant/café vs retail) and a Region. Neither is used as an input. Channel is the answer key: we set it aside and bring it back at the end to check whether the clusters found something real.

Step 1 — Look before you cluster

Analyze ▸ Summary on the six spend columns shows the shape. Grocery averages 7,951 a year, but the median customer spends 4,756. The biggest delicatessen buyer spends ~50× the median. Skewness runs from 2.6 (fresh) to 11.2 (delicatessen).

Step 2 — The default k-means scores well and finds nothing useful

Analyze ▸ Data Mining ▸ Clustering, the six raw columns, K-Means, two clusters. Stratum standardizes by default and runs ten k-means++ restarts from a fixed seed, so the result is reproducible. The legend reports a mean silhouette ≈ 0.57, which by the usual rule of thumb is strong structure, and the Elbow view's silhouette peaks at k = 2 as well.

Then switch to View = Profile, which prints what kmeans() prints in R plus a silhouette per cluster. The sizes are ~390 vs ~40–50, and the small group's own Mean Silhouette is about 0.01 against 0.64 for the crowd. The headline 0.57 belongs to the crowd; the "segment" scores zero. In original units the profile says who it is: mean grocery spend 27,597 against 5,489, five times everyone else. These are the giant grocery accounts in the tail. The Fit section puts Between / Total at 26.0%.

With Standardize off it gets worse: fresh food carries 45% of the raw variance, so k-means splits on fresh and nothing else (65 vs 375).

Step 3 — Put spending on the right scale

Spending is multiplicative. Going from 1,000 to 2,000 is the same change in behavior as going from 10,000 to 20,000, and a log scale treats them that way. Data ▸ Add Column… with ln([Fresh]), and the same for the other five columns. The smallest value in the file is 3, so a plain natural log is exact and nobody drops out (with zeros you'd use log1p(); here it gives the identical partition). Grocery skewness goes from 3.59 to −0.74.

Re-run the clustering on the six log columns and switch to View = Elbow:

At k = 2 the groups are ~252 and ~188, two real segments instead of a whale group and a crowd. In the Profile view the overall silhouette fell from 0.57 to 0.29, but per cluster it reads 0.31 and 0.26: both segments hold together. And Between / Total rose from 26.0% to 30.0%. Both runs standardize six columns, so the total sum of squares is 2,634 in each and the ratios compare directly. The clusters explain more on the log scale with a lower silhouette, which is what overlapping groups of real businesses look like.

Step 4 — Ask a different algorithm: Ward hierarchical

On the same card, set Method to Hierarchical. Linkage defaults to Ward, which minimizes the same within-cluster variance as k-means but builds the groups bottom-up. The tallest gap in the dendrogram sits under the final merge. Cutting there gives 234 / 206. (For contrast, single linkage cut at two peels off just one customer.)

Hierarchical has the same Profile view: Ward's two groups show the same pattern as k-means, with Between / Total 27.1%, a little under k-means, which optimises exactly that ratio.

Write both assignments to the sheet with Add Cluster Column… and cross them in a contingency table. About 396 of 440 customers (90%) fall in the same group under both methods.

Step 5 — What separates the segments? PCA

Principal components on the six log columns: PC1 = 44.0% loads on grocery, detergents & paper and milk (a stock-the-shelves axis); PC2 = 27.1% loads on fresh, frozen and delicatessen (a cook-the-food axis). Together that's 71% of the variation.

Back on the clustering card, Profile with Units: Standardized gives the cluster centers in standard deviations of log spend, the same numbers as R's kmeans(scale(log(x)), 2)$centers:

In money terms (geometric means, computed outside Stratum), the kitchen segment spends about 8,600 on fresh and 290 on detergents & paper; the shelf segment about 12,100 on grocery and 4,000 on detergents & paper.

Step 6 — Validate against the label it never saw

Now bring back the answer key. Contingency table: cluster down the side, Channel across, row percentages.

The clusters match Channel for 86% of customers, against 68% for always guessing the bigger channel. Two comparisons back this up. The raw-scale clustering, with its 0.57 silhouette and a small cluster scoring zero, only reaches V ≈ 0.4: the best-looking score matched the business worst. And as a negative control, the same segments against Region give V = 0.05, p = 0.54. The segments reflect what customers buy, not where they are.

Step 7 — Two questions anyone can use

A segmentation is only useful if someone can assign a new customer to it. A decision tree with the cluster as response, the six original spend columns as predictors, Max Depth = 2 and pruning off, then View = Rules:

The first question alone (detergents & paper above 1,535 a year?) sorts ~93% of customers correctly. With Validation = 30% the tree regrows on the rest and lands in the low 90s on customers it never saw.

What we actually did

We saw the default clustering score well while finding only the whales (a cluster with a silhouette of zero), and fixed the scale with logs: Between / Total went up as the silhouette went down. Elbow and silhouette both chose two clusters, and Ward agreed with k-means for nine customers in ten. PCA and the standardized profile explained the split, a held-back label validated it at V ≈ 0.72 while Region showed nothing, and a two-question tree, printed as rules, turned it into something sales can use. That's a full cluster analysis on real data, no code.