Key takeaways
- FAISS supports cosine similarity by normalising vectors and using an inner product index.
- FAISS offers exact flat search and ANN indexes such as IVF, PQ and HNSW.
- BM25 does not use cosine similarity; it scores term frequency, rarity and length.
- Cosine scores depend on how embeddings were trained, so treat them as rankings, not probabilities.
- In RAG, add hybrid search and reranking, and calibrate LangChain thresholds on real queries.
FAISS supports cosine similarity indirectly: normalise your vectors to unit length and use an inner product index, because the inner product of unit vectors equals cosine similarity. FAISS, Meta's library for fast similarity search, also offers approximate nearest neighbour indexes such as IVF, PQ and HNSW. RAG.NextUpgrad combines dense retrieval with BM25, so this page covers FAISS metrics and indexes, how search works, cosine versus BM25 and semantic search, when cosine misleads in RAG, and similarity thresholds in LangChain.
Does FAISS use cosine similarity?
Not directly by name. FAISS provides L2 distance and inner product indexes. To use cosine similarity, normalise all vectors with faiss.normalize_L2 and use an inner product index such as IndexFlatIP; scores then equal cosine similarity.
Normalise both the stored vectors and every query vector, otherwise inner product favours longer vectors. Many embedding models already output unit-length vectors, in which case inner product and cosine give the same ranking, and even L2 distance ranks identically, since for unit vectors squared distance equals two minus twice the cosine. In LangChain's FAISS wrapper, setting the distance strategy to inner product or enabling L2 normalisation achieves cosine behaviour. Check your model's documentation for normalisation.
What does FAISS use for search?
FAISS uses vector indexes and distance metrics: exact flat indexes that compare against every vector, and approximate indexes such as inverted files, product quantization and HNSW graphs, with L2 or inner product as the metric.
Also asked as: does faiss use ann
Indexes can be combined through the index factory; for example, a string such as IVF4096,PQ64 creates an inverted file with 4,096 clusters and product-quantized codes. FAISS is written in C++ with Python bindings, uses SIMD instructions on CPUs, and supports GPUs for very fast batch search. It is a library, not a database, so persistence, metadata filtering and updates are your responsibility, or are handled by frameworks that wrap it.
Does FAISS use approximate nearest neighbour search?
Yes, optionally. FAISS offers both exact search with flat indexes and approximate nearest neighbour search with IVF, PQ, HNSW and LSH indexes, which trade a small loss in recall for much faster queries and lower memory.
Exact flat search is ideal up to hundreds of thousands of vectors, especially on GPU. Beyond that, ANN indexes become necessary: IVF searches only the closest clusters, controlled by nprobe; PQ compresses vectors so billions fit in memory; HNSW navigates a graph for low-latency queries. Tune parameters by measuring recall against exact search on a sample of queries, choosing the cheapest setting that meets your recall target.
Does FAISS use HNSW?
Yes. FAISS includes HNSW indexes, such as IndexHNSWFlat, which build a hierarchical navigable small world graph over vectors for fast approximate search, with M controlling graph connectivity and efSearch controlling search breadth.
HNSW in FAISS offers excellent speed and recall without a training step, but uses more memory than compressed indexes and does not support removing vectors easily. Increase efSearch for better recall at the cost of latency, and choose M around 16 to 64 depending on accuracy needs. For very large collections, combining IVF with PQ is more memory-efficient, while HNSW suits datasets that fit comfortably in RAM and need low latency. It is also the default choice in many vector databases.
How does FAISS similarity search work?
You add vectors to an index, then search with query vectors; FAISS returns the k nearest vectors and their distances. Flat indexes compute distances to every vector; approximate indexes narrow candidates first, using clusters, compressed codes or graphs.
In Python: create index = faiss.IndexFlatIP(dim), call index.add(vectors) with a float32 NumPy array, then distances, ids = index.search(queries, k). IDs map back to your documents through a list or dictionary you maintain. Approximate indexes such as IVF need index.train(sample) before adding vectors. Save indexes with faiss.write_index and load them with faiss.read_index. Batch queries together for speed, since FAISS is optimised for matrix operations.
Does BM25 use cosine similarity?
No. BM25 is a probabilistic ranking function that scores documents by term frequency, inverse document frequency and document length normalisation. It sums per-term scores rather than computing an angle between vectors.
Classic TF-IDF retrieval in the vector space model does use cosine similarity between sparse term vectors, which is where the confusion comes from. BM25 improved on it with term frequency saturation, so repeating a word has a diminishing effect, and length normalisation controlled by the parameters k1 and b, with typical defaults around 1.2 and 0.75. Cosine similarity on dense embeddings captures meaning, while BM25 captures exact term overlap, which is why hybrid retrieval combines both, often with reciprocal rank fusion.
Does semantic search use cosine similarity?
Usually, yes. Semantic search embeds the query and documents into vectors and ranks documents by similarity, most often cosine similarity or the equivalent dot product on normalised vectors. Some systems use Euclidean distance instead.
Semantic search and cosine similarity are not the same thing: semantic search is the overall task of finding results by meaning, while cosine similarity is one scoring function used inside it. The embedding model matters far more than the metric: a good model makes similar meanings point in similar directions. Use the metric your embedding model was trained with, which its documentation states, and keep it consistent between indexing and querying.
Is cosine similarity of embeddings really about similarity?
Not always. A 2024 paper with this title, from Netflix researchers, showed that for some learned embeddings, cosine similarity can produce arbitrary results, because training can rescale dimensions in ways that change cosine scores without changing the model's predictions.
Also asked as: is cosine-similarity of embeddings really about similarity
Their analysis focused on matrix factorisation models with certain regularisation, but the lesson generalises: cosine scores depend on how embeddings were trained, and a high score does not guarantee semantic relevance. Modern sentence embedding models are trained specifically so cosine similarity is meaningful, which makes them more reliable. Still, validate on your data: check whether top results are actually relevant, and treat raw similarity scores as relative rankings rather than calibrated probabilities.
Why does cosine similarity fail in RAG, and what should you use instead?
Cosine similarity on embeddings can rank passages that sound similar above passages that actually answer the question, miss exact terms such as codes and names, and give scores that are hard to threshold reliably across queries.
Also asked as: why cosine similarity fails in rag (and what to use instead
Fixes: combine dense retrieval with BM25 in hybrid search; rerank the top candidates with a cross-encoder that reads the question and passage together; use embedding models trained for retrieval with separate query and document modes; and add metadata filters. Base decisions on reranker scores rather than raw cosine values. RAG.NextUpgrad applies hybrid retrieval, fusion and reranking, then a confidence gate, so answers rely on evidence that actually supports them.
How do you set a similarity threshold in a LangChain FAISS retriever?
Create the retriever with search_type="similarity_score_threshold" and pass search_kwargs={"score_threshold": 0.5, "k": 5}. Only documents whose relevance score exceeds the threshold are returned, up to k.
Also asked as: how to specify similarity threshold in langchain faiss retriever
Relevance scores in LangChain are normalised to a zero-to-one range, but how they are computed depends on the distance strategy: FAISS defaults to L2 distance, converted into a relevance score, so pick thresholds by testing. Normalise embeddings and use inner product or cosine settings for more interpretable scores. Calibrate by running known relevant and irrelevant queries, then choose a threshold that keeps relevant results while filtering noise. Too strict a threshold returns nothing for valid questions, so log empty results.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
