Changing units

Topic: Nearest neighbours · more questions on this topic
‹ Neighbours by hand Rank distance ›

The query is (0,0) and the training points are a=(1,10) and b=(3,1). Which is nearer in Euclidean distance? Divide the second coordinate by 10 and repeat. What does this say about scale invariance?

Worked answer (click to show)

In the original units,

d(0,a)=\sqrt{1+100}=\sqrt{101}=10.05, \qquad d(0,b)=\sqrt{9+1}=\sqrt{10}=3.16,

so b is nearer. After dividing the second coordinate by 10, a'=(1,1) and b'=(3,0.1), and

d(0,a')=\sqrt{2}=1.41, \qquad d(0,b')=\sqrt{9+0.01}=3.00,

so now a is nearer. The nearest neighbour, and hence the classification, depends on the units. Euclidean distance is not scale invariant.