What are text embeddings? How they work, creating them, models, clustering and decoding

Text embeddings are lists of numbers, vectors, that represent the meaning of a piece of text, a word, sentence, paragraph or document, so that texts with similar meanings have vectors close together. They let software search, cluster, classify and recommend text by meaning rather than exact words. Text embeddings drive retrieval in RAG.NextUpgrad and the OCR and Speech Workspace, so this page covers what they are, how they work and are created, converting text, models, clustering, embedding servers, decoding, and foundation models.

What are text embeddings?

Text embeddings are dense vectors, typically with hundreds to a few thousand dimensions, produced by a model from text. Each vector captures the text's meaning in a way that makes similarity measurable with cosine similarity or dot product.

Also asked as: what are text embeddings models

For example, "How do I reset my password?" and "Steps to recover my login" produce nearby vectors, even with few shared words, while "best biryani recipes" lands far away. Uses include semantic search, RAG retrieval, clustering support tickets, detecting duplicates, recommending articles and classifying text with little labelled data. Embeddings work across languages with multilingual models, so a question in Hindi can retrieve a relevant English passage. That cross-lingual ability is valuable in India, where users often mix languages in the same query.

How do text embeddings work?

An embedding model reads the text, converts it to tokens, processes them through transformer layers that capture context, then pools the token representations into one vector. The training makes geometry meaningful: similar meanings point in similar directions.

Comparison is simple arithmetic: cosine similarity measures the angle between two vectors, with higher values meaning more similar. Search systems embed all documents once, store the vectors in an index, then embed each query and find the nearest vectors. Because each dimension is not human-readable, embeddings work as a whole; you evaluate them by how well they retrieve or group text for your task.

How are text embeddings created?

Embedding models are trained with contrastive learning on large collections of related text pairs, such as questions and answers, titles and articles, or translations. The model learns to place each pair close together and push unrelated texts apart.

Training often starts from a pretrained language model, then fine-tunes on hundreds of millions of pairs, sometimes with hard negatives, texts that look similar but are not relevant. Sentence-BERT, in 2019, showed how to adapt BERT for efficient sentence embeddings. Modern models also use instructions or prefixes, such as "query:" and "passage:", to distinguish questions from documents.

How do I convert text to embeddings?

Use a library such as Sentence Transformers for local models, or an embedding API from providers such as OpenAI, Google or Cohere. Pass a list of texts and receive a list of vectors.

Also asked as: how to convert text to embeddings

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BAAI/bge-small-en-v1.5")
vecs = model.encode(["How do I reset my password?", "Steps to recover my login"],
                    normalize_embeddings=True)
print(vecs.shape, vecs[0] @ vecs[1])

Split long documents into chunks first, since models have input limits. Embed in batches for speed, store vectors with IDs and metadata, and always use the same model for documents and queries.

What are text embedding models?

Text embedding models are neural networks, usually transformer encoders, designed to output vectors rather than generate text. Examples include open models such as BGE, E5, GTE, Nomic Embed and Qwen3 Embedding, and hosted models from OpenAI, Google, Cohere and Voyage.

They differ in size, dimensions, maximum input length, languages supported and domain strengths. Small models run quickly on CPUs; larger ones are more accurate but slower. Some support Matryoshka truncation, letting you shorten vectors to save storage. Multilingual models matter for Indian languages and mixed-language content, common in Indian applications.

What is the best text embedding model?

There is no single best model; the best is the one that retrieves best on your data within your speed, cost and privacy limits. The MTEB leaderboard on Hugging Face compares many models across tasks and languages, useful for shortlisting.

Also asked as: best text embeddings model

How to choose:

• Shortlist three models from MTEB for your language and task.
• Build 50 to 100 real queries with known relevant passages.
• Measure recall at 5 or 10 for each model.
• Check speed, memory and cost at your scale.
• Prefer models with suitable licences for commercial use.

A smaller model with good chunking and hybrid search often beats a larger model used alone.

How do I cluster text embeddings?

Embed your texts, optionally reduce dimensions, then apply a clustering algorithm such as k-means, HDBSCAN or agglomerative clustering, and inspect example texts from each cluster to name the themes.

Also asked as: how to cluster text embeddings

Steps: normalise embeddings; try k-means with several values of k, or HDBSCAN, which finds the number of clusters automatically and marks outliers; use UMAP to reduce dimensions for visualisation and sometimes better clustering; and label clusters by reading samples or asking a language model to summarise them. Clustering support tickets, feedback or survey answers reveals common issues quickly. Re-run clustering periodically on new data, since themes shift over time, and track how cluster sizes change to spot emerging problems.

What is Text Embeddings Inference?

Text Embeddings Inference, TEI, is Hugging Face's open-source server for serving embedding and reranking models efficiently, with an HTTP API, dynamic batching, optimised kernels and support for many popular models on CPUs and GPUs.

Use TEI when you need a self-hosted embedding service for production: run it with Docker, point it at a model from the Hugging Face Hub, and call it from your application. It handles concurrency and batching better than calling a Python library per request. Similar options include embedding endpoints in vLLM, Ollama and llama.cpp, and managed embedding APIs.

Can you convert embeddings back to text?

Not exactly: embeddings are lossy summaries, and no standard function turns a vector back into its original text. However, research has shown that inversion models can reconstruct much of the original text from embeddings, especially for short inputs.

A 2023 study recovered a large share of short texts exactly from their embeddings using iterative correction, which means embeddings should be treated as sensitive data, not anonymised. Encrypt them, control access, and do not share embeddings of private documents with untrusted parties. For your own applications, store the original text alongside vectors, since you need it for answers and citations anyway.

What is a text embedding foundation model?

A text embedding foundation model is a large, general-purpose embedding model trained on broad data so it works across many tasks and domains without task-specific training, serving as a base for search, clustering, classification and RAG.

Also asked as: what are text to embeddings foundation model

Foundation embedding models from cloud providers and open-source projects support many languages and long inputs, and can be fine-tuned for specialised domains such as legal or medical text. Using one saves training effort; fine-tuning on your own query-passage pairs can then improve retrieval further. Evaluate both the general model and any fine-tuned version on the same test set.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail