Text embedding models: how they are trained, uses, best for RAG and Ollama, OpenAI facts

Text embedding models turn sentences and documents into vectors so software can compare meaning, powering semantic search, RAG, clustering and classification. They are trained mostly with contrastive learning on huge sets of related text pairs. RAG.NextUpgrad combines dense embeddings with BM25, so this page explains how embedding models are trained, what they are used for, strong choices for 2026, RAG and Ollama, whether OpenAI's models are open source or multilingual, which similarity metric to use, and joint text-image embeddings.

How are text embedding models trained?

Most start from a pretrained transformer, then train with contrastive learning: pairs of related texts, such as a question and its answer, are pulled together in vector space, while unrelated texts in the same batch are pushed apart.

Training usually has stages: weakly supervised pretraining on hundreds of millions of naturally paired texts, such as titles and bodies or questions and answers, then fine-tuning on high-quality labelled datasets for retrieval, similarity and classification, often with hard negatives that look relevant but are not. Recent models are built from decoder LLMs and trained on synthetic data generated by larger models. Techniques such as Matryoshka learning let embeddings be shortened without retraining.

What are text embedding models used for?

They are used for semantic search, retrieval for RAG, recommendation, clustering similar documents, classification with simple models on top, duplicate detection, anomaly detection and measuring text similarity in evaluation.

In RAG, embeddings let a system find passages that answer a question even when the wording differs. In support systems, they route tickets by topic and find similar past issues. In analytics, clustering embeddings of customer feedback reveals themes without manual labels. Embeddings also power memory for agents, storing and retrieving past interactions. Because they are cheap to compute compared with generation, embeddings are often the first, most scalable step in AI pipelines.

What are the best text embedding models in 2026?

Strong options include Qwen3-Embedding and other leaders on the MTEB leaderboard, Google's gemini-embedding-001, OpenAI's text-embedding-3-large, Cohere Embed, Voyage AI models, and open models such as BGE-M3, multilingual E5, Jina embeddings, Nomic Embed and EmbeddingGemma.

Also asked as: best text embedding models 2026

Rankings change often, so check the MTEB leaderboard for current results by task and language, then test two or three candidates on your own data. Consider dimension size and storage, maximum input length, multilingual needs, licence, latency and whether you need local hosting. A smaller model well matched to your domain can beat a larger general model. Re-evaluate yearly, since switching requires re-embedding your corpus.

What are the best text embedding models for RAG?

Choose models trained for retrieval, which handle queries and documents separately or accept instructions. Hosted options include OpenAI text-embedding-3, Voyage, Cohere and Gemini embeddings; open options include BGE-M3, E5, Qwen3-Embedding, Nomic and Jina models.

For RAG, retrieval benchmarks such as BEIR and the retrieval tasks in MTEB matter more than average scores. Test with your own questions and known relevant passages, measuring recall at k. Multilingual corpora need multilingual models such as BGE-M3 or Qwen3-Embedding. Pair any embedding model with BM25 for hybrid search and a reranker, since those often improve quality more than switching embedding models. Also check the maximum input length against your chunk size.

What are the best text embedding models on Ollama?

Popular Ollama embedding models include nomic-embed-text, mxbai-embed-large, bge-m3, snowflake-arctic-embed, all-minilm and embeddinggemma, plus newer options such as Qwen3 embedding models. Pull one and call Ollama's embeddings endpoint.

Also asked as: best text embedding models ollama

Choose by needs: all-minilm is tiny and fast for prototypes; nomic-embed-text and mxbai-embed-large are strong general English choices; bge-m3 handles many languages and long inputs; embeddinggemma offers good quality in a small size. With LangChain or LlamaIndex, use their Ollama embedding classes, and make sure documents and queries use the same model. Local embeddings keep data private and avoid API costs, which suits sensitive RAG systems. Benchmark speed on your hardware, since large models slow indexing.

Is text-embedding-3-large open source?

No. OpenAI's text-embedding-3-large and text-embedding-3-small are proprietary models available only through OpenAI's API and Azure OpenAI. Their weights are not released, so you cannot run them locally or fine-tune them.

Also asked as: is text embedding 3 large open source

If you need open weights, for privacy, offline use or fine-tuning, choose open embedding models such as BGE-M3, E5, Qwen3-Embedding, Nomic Embed, Jina embeddings or EmbeddingGemma, checking each licence for commercial terms. Using the API means sending text to OpenAI, so review data policies for sensitive content. Vectors from text-embedding-3 models are tied to that service: if you switch to an open model later, you must re-embed your documents.

Are text-embedding-3 models multilingual?

Yes. Both text-embedding-3-large and text-embedding-3-small support many languages. OpenAI reported large gains on the multilingual MIRACL benchmark, with the large model scoring about 54.9 and the small model about 44.0.

Also asked as: is text embedding 3 large multilingual

Quality varies by language: widely used languages perform well, while lower-resource languages, including some Indic languages, may be weaker. Cross-lingual retrieval, such as an English query finding a Hindi passage, often works but should be tested. For multilingual RAG, compare text-embedding-3-large with open multilingual models like BGE-M3 and Qwen3-Embedding on your languages. Keep language metadata on chunks, so you can filter results and report recall per language.

Is text-embedding-ada-002 multilingual?

Partly. ada-002 handles many languages, but its multilingual retrieval quality is much weaker than OpenAI's newer models: OpenAI reported about 31.4 on MIRACL for ada-002, compared with 44.0 and 54.9 for text-embedding-3-small and large.

Also asked as: is text embedding ada 002 multilingual

For English-heavy applications, ada-002 still works, but for any multilingual or cross-lingual system, migrating improves results substantially. Migration means re-embedding all documents with the new model and rebuilding your index, because vectors from different models are incompatible. Run a comparison on your own queries before switching production traffic. If your users write in several languages, include examples of each in your evaluation set, since averages hide weak languages.

Which metric should you use for semantic similarity with text embeddings?

Use cosine similarity by default, or dot product when embeddings are normalised, since they then rank identically. Most sentence embedding models are trained and evaluated with cosine similarity, and their documentation states the intended metric.

Also asked as: which metric i should use in general for semantic similarity in text embedding

Some models are trained for dot product without normalisation, where vector length carries meaning, so cosine would discard useful information; follow the model card. Euclidean distance gives the same ranking as cosine for normalised vectors. Whatever you choose, use the same metric in your vector index and evaluation. For decisions such as duplicate detection, calibrate thresholds on labelled pairs from your own data, since raw similarity values differ between models.

Can you create a single embedding for both text and an image?

Yes. Multimodal embedding models such as CLIP and SigLIP map text and images into a shared space, and some newer models accept interleaved text and image input to produce one combined embedding for a pair, such as a product photo with its description.

Also asked as: is it possible to create one single embedding for both text and a corresponding image

Options: use a model that natively embeds mixed inputs, as offered by some commercial multimodal embedding APIs; or embed text and image separately with a shared-space model like CLIP and combine the vectors by averaging or concatenation, then normalise. Concatenation preserves both signals but doubles dimensions. Evaluate on your retrieval task, since the right weighting between text and image depends on whether users search mainly by words or by visual similarity.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail