Cosine Similarity

Cosine similarity measures how alike two vectors are by computing the cosine of the angle between them. It is defined as the dot product of the vectors divided by the product of their magnitudes, written cos(theta) = (A · B) / (||A|| ||B||). The result lies between -1 and 1, where 1 means the vectors point in exactly the same direction, 0 means they are orthogonal and therefore unrelated, and -1 means they point in opposite directions. For vectors whose components are all non-negative, which is common for many representations, the value ranges from 0 to 1.

Why direction rather than distance

The defining characteristic of cosine similarity is that it depends only on the orientation of the vectors, not their length. Two documents, one short and one long, that discuss the same topic in the same proportions will have similar direction even though their raw word counts differ greatly in magnitude. By normalising away magnitude, cosine similarity focuses on the pattern of features rather than their absolute scale, which is usually what matters when comparing meaning or composition.

This contrasts with Euclidean distance, which is sensitive to magnitude and can report two semantically similar but differently scaled vectors as far apart. When vectors are first normalised to unit length, cosine similarity and Euclidean distance become monotonically related, and ranking by one is equivalent to ranking by the other.

Role in embeddings and search

Cosine similarity is the workhorse comparison in modern systems built on embeddings, the dense numerical representations of text, images or other data produced by neural networks. In semantic search, a user query is converted into an embedding and compared against a collection of stored embeddings; the items with the highest cosine similarity are returned as the most relevant. The same principle underpins retrieval-augmented generation, recommendation systems, duplicate detection, and clustering.

Vector databases such as those used to power large-scale retrieval implement approximate nearest-neighbour search optimised for cosine similarity or the closely related inner product, allowing similarity queries over millions or billions of vectors in milliseconds. Because normalised dot product equals cosine similarity, many systems store unit-normalised vectors and compute a simple inner product for efficiency.

A worked intuition

Consider three short texts represented as term-frequency vectors. Two that share many of the same words, regardless of how long each text is, will have a small angle between their vectors and thus a cosine similarity near 1. A text on an unrelated subject will share few terms, giving a near-orthogonal vector and a similarity near 0. This simple geometry generalises directly to high-dimensional learned embeddings, where each of hundreds or thousands of dimensions encodes some latent feature.

| Measure | Sensitive to magnitude | Typical use | | --- | --- | --- | | Cosine similarity | No | Embedding and document comparison | | Dot product | Yes | Scoring with unnormalised vectors | | Euclidean distance | Yes | Geometric nearest-neighbour search |

Cosine similarity underlies many AI applications being built by Malaysian companies and institutions. E-commerce and ride-hailing platforms operating in the country, including Grab and regional marketplaces, use embedding-based recommendation and search in which cosine similarity ranks products, merchants or content for users. As these firms expand multilingual support for Bahasa Melayu, English and Chinese, embedding comparison helps match queries to relevant items across languages.

Homegrown language models such as ILMU and MaLLaM produce embeddings tuned for Malaysian languages and dialects, and downstream retrieval systems rely on cosine similarity to surface relevant Bahasa Melayu documents. This matters for public-sector knowledge bases and for banks such as Maybank and CIMB building internal semantic search over local policy and regulatory text.

Malaysia's growing data-centre and AI-cloud capacity, including investments by YTL and multinational providers, gives local organisations the infrastructure to run vector databases at scale. Training delivered through MDEC programmes and HRD Corp-funded courses commonly covers embeddings and similarity search, equipping Malaysian developers to build retrieval and recommendation features for regional markets across Southeast Asia.

1. Manning, C. D., Raghavan, P., and Schutze, H. (2008). Introduction to Information Retrieval. Cambridge University Press.
2. Mikolov, T. et al. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781.
3. Johnson, J., Douze, M., and Jegou, H. (2019). Billion-Scale Similarity Search with GPUs. IEEE Transactions on Big Data.