Silhouette scores: useful signal and common misinterpretations

The silhouette coefficient compares how tightly a row sits in its assigned cluster with how close it is to the nearest other cluster. Values range from -1 to 1. The average is useful, but it can hide small, unstable or poorly assigned groups.

Inspect the distribution, not only the winner

We standardize three behaviour features for 360 synthetic customers and fit K-means for k=2 through k=6 with fixed initialization rules.

k=3 has the highest mean in this candidate grid. The two-cluster solution merges structure into a 239-row group, while larger values split existing groups and introduce negative rows.

The scikit-learn silhouette example recommends examining per-cluster thickness and below-average values along with the mean.

from unsupervised_cases import silhouette_case

r = silhouette_case()
assert r['highest_mean_k'] == 3
assert [row['k'] for row in r['candidates']] == [2, 3, 4, 5, 6]
k4 = next(row for row in r['candidates'] if row['k'] == 4)
assert k4['negative_rows'] == 2
assert sum(k4['cluster_sizes']) == 360
print(r)

Run it in the unsupervised lab. The score uses Euclidean distance in one standardized feature space; changing that space changes the result.

Know what silhouette cannot establish

A high silhouette does not prove that clusters are natural entities, stable over time or useful for decisions. Elongated, varying-density or nested structures can score poorly under a centroid-oriented metric even when they matter. A clean separation can also come from a proxy, data collection artifact or feature leakage.

Do not search dozens of representations and k values, choose the highest score and report it as if predeclared. That workflow overfits an internal diagnostic. Use a bounded candidate set motivated by use and check resampling stability, later-period assignment and profile coherence.

For large datasets, approximate silhouette calculations must document sample size and seed. For one cluster, the score is undefined. For singleton clusters, interpretations need care. Always pair the mean with cluster sizes and sample-level or per-cluster distributions.

Exercise: compute silhouette on raw features, standardized features and a duplicated-feature representation. Compare which k wins. Then write why an internal criterion cannot tell whether a proposed campaign causes better outcomes.

NeuraPath's Data Science course uses silhouette as one diagnostic in a broader segmentation review. A good analysis shows the distribution and the assumptions behind the distance.

Continue learning

This article is part of the Clustering, reduction and recommendations sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in K-means: choose features before choosing the number of clusters.
• Continue with DBSCAN: distinguish noise from a bad density setting.