Cosine similarity versus Euclidean distance after scaling

Euclidean distance measures separation in the chosen coordinate system. Cosine similarity compares directions relative to the origin. Both depend on how the features represent the problem, and cosine is not immune to changing individual feature units.

Use three original abstract vectors: A=(1,1), B=(2,1) and C=(1,3). Euclidean distance from A to B is one; from A to C it is two. B is the nearest of these two candidates under this representation.

Change one unit and watch the neighbour change

Multiply the first feature by ten for every vector. The transformed vectors are A=(10,1), B=(20,1) and C=(10,3). The distances become ten and two, so C is now nearest.

The underlying records have not changed. The coordinate weights have. This is why mixing a large-magnitude monetary feature with a small-scale count can make a distance-based method behave differently from what its user intended.

The example is deliberately unit-free. Before adapting it to a business dataset, define what one unit of difference in each feature should mean and whether missing values, outliers or categorical encodings make the distance meaningful.

Compare two different kinds of scaling

Multiplying an entire nonzero vector by a positive scalar preserves its direction and therefore its cosine similarity with another fixed vector. Multiplying just one coordinate across the dataset can change directions.

The original cosine of A and B is about 0.948683. After the first-feature rescaling it is about 0.998765. Scaling all of B by ten leaves its original cosine with A unchanged.

import numpy as np
from model_geometry import scaling_case

r = scaling_case()
assert r['original_distances']==[1.,2.]
assert r['rescaled_distances']==[10.,2.]
assert np.isclose(r['original_cosine_ab'],.9486832980505137)
assert np.isclose(r['rescaled_cosine_ab'],.9987646995976289)
assert np.isclose(r['original_cosine_ab'],r['whole_vector_scaled_cosine_ab'])
assert not np.isclose(r['original_cosine_ab'],r['rescaled_cosine_ab'])
print(r)

Run in the mathematics lab. The helper uses dot products and normalized vectors; NumPy's dot documentation supplies the underlying operation's reference.

Recognize the connection for unit vectors

For vectors individually normalized to Euclidean norm one, squared Euclidean distance equals 2-2*cosine_similarity. Consequently, ranking those normalized vectors by ascending Euclidean distance matches ranking by descending cosine similarity, apart from ties and numerical effects.

That identity does not mean every preprocessing pipeline is interchangeable. Centering changes the origin, standardization changes coordinate weights, and row normalization removes overall magnitude. Each operation encodes a different choice about what information the comparison should retain.

Zero vectors need an explicit policy because the ordinary cosine formula divides by their norms. The lab rejects them. Silently assigning an arbitrary similarity can hide empty or failed feature extraction.

Evaluate the representation with the task

If overall magnitude matters, normalization may discard useful information. If direction matters and length reflects irrelevant repetition, normalization may help. Neither statement selects a representation without a task-specific evaluation.

Fit learned preprocessing statistics on training data. Record feature names and order with the fitted transformation so that a later prediction cannot accidentally compare a different coordinate system.

Exercise: change the first-feature multiplier from ten to 0.1 and recompute the distances. Then normalize all original vectors to unit length and verify the distance–cosine identity. Explain which operation changes feature importance and which removes vector magnitude.

NeuraPath's Data Science course connects feature preparation with model behavior. A useful nearest-neighbour result includes the representation and scaling choices that produced it.

Continue learning

This article is part of the Mathematics and statistical foundations sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Entropy and information gain in a small decision tree.
• Continue with Covariance matrices: detect redundant features.