Key takeaways
- Embedding models turn text or images into vectors so similar meanings are close together.
- They are neural networks, usually transformers, trained with contrastive learning on related pairs.
- OpenAI's text-embedding-3-small returns 1,536 dimensions and text-embedding-3-large 3,072, both shortenable.
- OpenAI embeddings are paid per token; open models via Ollama, Hugging Face or llama.cpp run free locally.
- Batch by sending lists of inputs per request, or use the Batch API for large asynchronous jobs.
Embedding models are neural networks that convert text, images or other data into vectors, lists of numbers, so that items with similar meaning end up close together. They power semantic search, RAG retrieval, recommendations, clustering and deduplication. Choosing and running the right one is a core part of building RAG systems like RAG.NextUpgrad. This page explains what embedding models are and how they work, why AI uses embeddings, OpenAI's embeddings, dimensions, cost and batching, and running open models with Ollama, Hugging Face and llama.cpp.
What are embedding models?
Embedding models are models whose output is a vector representation of their input rather than generated text. Given a sentence, they return a fixed-length vector capturing its meaning, which can be compared with other vectors by cosine similarity.
Also asked as: what are embeddings models
There are text embedding models for search and RAG, multimodal models such as CLIP that embed images and text into the same space, code embedding models, and specialised models for faces, audio and products. Options include hosted APIs from OpenAI, Google, Cohere and others, and open models such as BGE, E5, GTE, Nomic and Qwen3 Embedding, which you can run yourself. Leaderboards such as MTEB compare them across tasks.
How do embedding models work?
Most text embedding models are transformer encoders. They tokenise the text, process tokens through attention layers, then pool the token representations, for example by averaging or taking a special token, into one vector, often normalised to unit length.
Also asked as: how do embeddings models work
They learn through contrastive training: the model sees pairs that should match, such as a question and its answer or a title and its article, and pairs that should not, and adjusts so matching pairs score higher. Training on huge and varied pairs gives general-purpose embeddings; fine-tuning on domain pairs specialises them. Some models expect prefixes such as "query:" and "passage:" to distinguish questions from documents, so read each model card.
Why do AI systems use embeddings?
AI systems use embeddings because they turn messy inputs into numbers that capture meaning, letting algorithms compare, search and learn efficiently. Computers cannot compare the meaning of two sentences directly, but they can compare vectors in microseconds.
Inside language models, every token becomes an embedding before processing. Outside them, embeddings enable retrieval for RAG, so models can use your documents; recommendation systems, which match users to items; and classification with little labelled data. Embeddings are also reusable: one embedding pass over your data supports search, clustering and analytics, which makes them one of the most practical tools in applied AI.
How do OpenAI embeddings work?
You send text to OpenAI's embeddings endpoint with a model name, such as text-embedding-3-small, and receive a vector for each input. Store the vectors, embed queries with the same model, and compare with cosine similarity.
Also asked as: how openai embeddings work
from openai import OpenAI
client = OpenAI()
resp = client.embeddings.create(model="text-embedding-3-small",
input=["How do I reset my password?"])
vector = resp.data[0].embedding
OpenAI's vectors are normalised, so dot product equals cosine similarity. Inputs have a maximum token length, so chunk long documents first. Keep your API key in an environment variable, never in code.
How many dimensions do OpenAI embeddings have?
text-embedding-3-small returns 1,536 dimensions and text-embedding-3-large returns 3,072 by default. The older text-embedding-ada-002 also returns 1,536. The newer models accept a dimensions parameter to return shorter vectors.
Also asked as: how many dimensions openai embeddings
Shortening, for example to 512 or 256 dimensions, cuts storage and speeds search with modest quality loss, because these models were trained so that the leading dimensions carry the most information. Choose a size by testing retrieval recall on your own questions. Whatever size you pick, your vector index must be configured with the same dimension, and all stored vectors must come from the same model and size.
Are OpenAI embeddings free?
No. OpenAI charges for embeddings per token processed, though embedding costs are low compared with text generation, and the Batch API offers a discount for asynchronous jobs. New accounts may receive limited trial credits depending on current policies.
Also asked as: is openai embeddings free
If you need zero-cost embeddings, run open models locally with Ollama, Hugging Face Sentence Transformers or llama.cpp; they cost only your own compute. Many open models perform competitively on retrieval benchmarks. For production, compare total cost, including servers for self-hosting, against API costs and quality. Check OpenAI's pricing page for current rates before estimating.
How do I batch embeddings with OpenAI's API?
Send a list of texts in the input field of one request, up to the documented limits on inputs and tokens per request, and process the returned vectors in order. For very large jobs, use the Batch API, which runs asynchronously within a time window at lower cost.
Also asked as: how can i use batch embeddings using openai's api
Tips:
- Group chunks into batches of a few hundred, staying under token limits.
- Retry failed batches with exponential backoff on rate-limit errors.
- Store each vector with its chunk ID so order mistakes cannot mix them up.
- For the Batch API, upload a JSONL file of requests, then download results when complete.
- Cache embeddings so unchanged chunks are never re-embedded.
How do I use Ollama embeddings?
Pull an embedding model, such as ollama pull nomic-embed-text, then call Ollama's embed endpoint or use its client libraries to get vectors locally, with no API key and no data leaving your machine.
Also asked as: why do ais use embeddings · how to use ollama embeddings · how to use huggingface embeddings
import ollama
resp = ollama.embed(model="nomic-embed-text", input=["What is RAG?"])
vector = resp["embeddings"][0]
Frameworks such as LangChain and LlamaIndex have Ollama embedding integrations. Local embeddings suit private documents and offline apps; speed depends on your hardware, and a GPU helps for large collections. Use the same Ollama model for indexing and querying.
How do I use Hugging Face embeddings?
The easiest way is the Sentence Transformers library: install it, load a model from the Hugging Face Hub by name, and call encode. Models download once and run locally on CPU or GPU.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BAAI/bge-small-en-v1.5")
vectors = model.encode(["What is RAG?"], normalize_embeddings=True)
Choose models from the MTEB leaderboard filtered by your language and size limits; multilingual models exist for Hindi and other Indian languages. For serving at scale, Hugging Face's Text Embeddings Inference server provides an efficient API. Check each model's licence for commercial use.
Can llama.cpp use Qwen3 embeddings?
Yes. llama.cpp can run embedding models in GGUF format, including Qwen3 Embedding models, by starting its server with embeddings enabled and the pooling method the model expects, then calling the embeddings endpoint.
Also asked as: can llamacpp use qwen3 embeddings
Download a GGUF version of the Qwen3 Embedding model, run llama-server with the --embedding flag and the pooling option recommended on the model card, and send text to the OpenAI-compatible embeddings endpoint. Qwen3 Embedding models are multilingual and perform strongly on retrieval benchmarks. As with any model, verify the output dimension and test retrieval quality on your own data before switching production systems.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
