Key takeaways
- Generating embeddings converts data into vectors where similar meanings sit close together.
- Use embedding APIs or sentence-transformers, and the same model for documents and queries.
- Qdrant can embed through FastEmbed, while pgvector only stores and searches vectors.
- Embeddings can be partly inverted, so protect them like the original text.
- Embeddings are one component of RAG, which also needs chunking, keyword search and reranking.
Embeddings turn text, images or code into vectors of numbers that capture meaning, so similar items sit close together. They power semantic search, RAG, clustering and recommendations. RAG.NextUpgrad uses dense embeddings alongside BM25, and FaceVision compares face embeddings, so this page covers what generating embeddings means, how to create text and word embeddings, LLM embeddings, Qdrant, pgvector and Cursor, whether embeddings can be reversed, and how they differ from RAG and encoding.
What does generating embeddings mean?
Generating embeddings means passing data, such as a sentence, document chunk, image or code snippet, through an embedding model that outputs a fixed-length vector of numbers representing its meaning.
Similar inputs produce vectors that are close together, measured by cosine similarity or dot product, so a question about resetting passwords lands near an article about account recovery. Vectors typically have hundreds to a few thousand dimensions. You generate embeddings for every item in a collection once, store them in an index or vector database, then embed each query at search time and compare. The same model must embed both documents and queries for results to make sense.
How do you generate embeddings?
Call an embedding API, such as OpenAI's text-embedding-3 models, Cohere Embed, Voyage AI or Gemini embeddings, or run an open model locally with the sentence-transformers library, passing your text and receiving vectors back.
Also asked as: what does generate embeddings mean · how to generate embeddings
In Python, sentence-transformers loads a model such as BGE or E5 and encodes a list of sentences in one call, using a GPU if available. Batch requests to reduce overhead, and chunk long documents to fit the model's input limit. Normalise vectors if your similarity measure expects it. Store each vector with its text and metadata. Choose models by language support, domain, dimension size and benchmark results on the MTEB leaderboard, then verify on your own queries.
How do you create word embeddings?
Train classic word embeddings with Word2Vec, GloVe or FastText on a large text corpus, using libraries such as Gensim, or download pretrained vectors. Each word receives one fixed vector learned from the contexts it appears in.
Also asked as: how to create word embeddings
Word2Vec, introduced by Mikolov and colleagues in 2013, learns vectors by predicting words from their neighbours, producing famous analogies such as king minus man plus woman approximating queen. FastText adds subword information, helping with rare words and languages with rich morphology, including many Indian languages. Modern systems usually use contextual embeddings from transformer models instead, where the same word gets different vectors in different sentences. Classic word vectors remain useful for lightweight tasks and teaching.
Can an LLM generate embeddings?
Yes, but general chat LLMs are not ideal embedding models. You can extract hidden states from an LLM and pool them into a vector, but models trained specifically for embeddings, including LLM-based ones, perform much better for search.
Also asked as: can llm generate embeddings
Many top embedding models are actually language models fine-tuned with contrastive learning, such as E5-Mistral, NV-Embed and Qwen3 Embedding, which learn to place related texts close together. Chat APIs generally do not expose embeddings directly; providers offer separate embedding endpoints. For RAG, use a dedicated embedding model for retrieval and a chat model for answering. Larger embedding models improve quality but cost more to run, so test whether the gain justifies it.
Does Qdrant create embeddings?
Qdrant primarily stores and searches vectors that you provide. However, its client libraries integrate FastEmbed, which generates embeddings locally when you add or query text, and Qdrant Cloud offers built-in inference for generating embeddings server-side.
With FastEmbed, you can pass plain text to the client, which embeds it with lightweight models running on the CPU, then stores the vectors. This simplifies prototypes. For production, many teams generate embeddings in their own pipeline, giving full control over models, batching and versioning, then upsert vectors into Qdrant. Whichever approach you use, keep a record of the model and version used, since mixing embeddings from different models breaks similarity search.
Can pgvector generate embeddings?
No. pgvector stores vectors and performs similarity search inside PostgreSQL, but it does not run embedding models. You generate embeddings in your application or through other extensions and services, then store them in a vector column.
Extensions such as pgai and PostgresML can call embedding models from SQL, automating embedding generation when rows change, which keeps vectors in sync with source data. Otherwise, a background worker reads new rows, calls an embedding API or local model, and writes vectors back. Track which model produced each vector, and re-embed everything when you change models. Combine pgvector with PostgreSQL full-text search for hybrid retrieval in one database.
Where does Cursor create embeddings?
Cursor's codebase indexing splits your code into chunks and computes embeddings on Cursor's servers, storing them in a remote vector database so the editor can find relevant code for AI requests.
Cursor's security documentation explains how indexing works, including that file paths are obfuscated and how privacy mode affects whether code is stored. Files listed in a cursorignore file are excluded from indexing. If your organisation has strict policies, review these settings before indexing proprietary code, or disable indexing. Other AI coding tools take different approaches, such as searching files directly with grep-style tools rather than relying on embeddings, which avoids sending code for indexing.
Can embeddings be reverse engineered?
Partly, yes. A 2023 paper by Morris and colleagues introduced Vec2Text, which reconstructed many short texts exactly from their embeddings, so embeddings can leak the content they represent and should be treated as sensitive data.
Their iterative inversion recovered most short inputs from a widely used embedding model, and similar attacks work on other models with enough access. Defences include access controls, encryption at rest, not exposing raw vectors through APIs, and adding noise for some use cases at a cost to accuracy. Never assume that storing embeddings instead of text makes data anonymous. Apply the same privacy protections to vector databases as to the original documents.
What is the difference between embeddings and RAG?
Embeddings are vector representations of data. RAG, retrieval-augmented generation, is a full technique that retrieves relevant information and gives it to a language model to answer. RAG often uses embeddings for the retrieval step.
In a typical RAG pipeline, documents are chunked and embedded, the question is embedded, similar chunks are retrieved, optionally reranked, and the model answers using them with citations. Embeddings are one component; RAG also involves chunking, keyword search, reranking, prompting and evaluation. RAG.NextUpgrad combines dense embeddings with BM25 through reciprocal rank fusion before reranking, showing that embeddings alone are rarely the whole retrieval story. Embeddings also serve other uses, such as clustering and recommendations.
What is the difference between embeddings and encoding?
Encoding is any conversion of data into another format, such as tokenizing text, one-hot encoding categories or UTF-8 encoding characters. Embeddings are a specific kind of encoding: learned, dense vectors that capture meaning and similarity.
One-hot encoding represents each word as a long vector with a single one, so all words are equally distant and meaning is lost. Embeddings compress meaning into fewer dimensions where related items are close. In transformer terminology, an encoder model, such as BERT, produces embeddings, and positional encoding adds order information. The terms overlap in casual use, but embedding usually implies a learned representation suited to similarity search and machine learning inputs.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
