Key takeaways
- Embeddings include word, sentence, image, audio, multimodal, graph and user or item types, dense or sparse.
- Every embedding is a vector, but embeddings are vectors learned so that similar things sit close together.
- GPT-3 used 12,288-dimensional token embeddings; OpenAI's embedding APIs output 1,536 or 3,072 dimensions.
- Vector databases come as purpose-built systems, database extensions, search engines and plain libraries.
- Sentence Transformers, LangChain and Hugging Face make embeddings easy to use for search, RAG and recommenders.
Embeddings come in several types, word, sentence, document, image, multimodal, graph and user or item embeddings, and they can be dense or sparse. All are vectors, lists of numbers, learned so that similar things sit close together. RAG.NextUpgrad combines dense embeddings with BM25 sparse retrieval, so this page covers types of embeddings, embeddings versus vectors, embedding sizes in GPT models, types of LLMs and vector databases, tutorials for LangChain and Hugging Face, and recommender systems.
How many types of embeddings are there?
There is no fixed number, but common types are word embeddings, sentence and document embeddings, image embeddings, audio embeddings, multimodal embeddings, graph or node embeddings, and user and item embeddings in recommenders. Each can be dense or sparse.
Also asked as: how many types of llm are there
By form, dense embeddings are short vectors of real numbers from neural models, while sparse embeddings, such as SPLADE or BM25-style term weights, have mostly zero values tied to vocabulary terms. Multi-vector embeddings, as in ColBERT, keep one vector per token for fine-grained matching. Embeddings also differ by training: static ones like word2vec give one vector per word, while contextual ones from transformers change with the surrounding text.
What are embeddings and vectors?
A vector is an ordered list of numbers, such as (0.2, -1.3, 0.8). An embedding is a vector produced by a model to represent something, such as a word, image or user, so that meaningful relationships become distances between vectors.
Also asked as: what is embeddings and vectors
Vectors are the mathematical container; embeddings are vectors with learned meaning. A colour as RGB values is a vector but not usually called an embedding, because the numbers were not learned to capture similarity. Embeddings typically have hundreds or thousands of dimensions, and their individual numbers are not interpretable; meaning lies in the overall pattern. In AI applications, embeddings are stored in vector databases and compared with cosine similarity or dot product.
Are embeddings and vectors the same thing?
Every embedding is a vector, but not every vector is an embedding. The word embedding implies the vector was learned to represent an object's meaning or properties, so that similar objects have similar vectors.
In conversation, people use the terms loosely: vector database, vector search and embeddings often refer to the same workflow. The precise distinction helps when debugging: a feature vector of hand-crafted values, such as price and size, behaves differently from a learned embedding, because distances may not reflect similarity unless features are scaled and weighted. Embeddings from different models also live in different spaces, so their vectors cannot be compared directly, even with equal dimensions.
How many embeddings does ChatGPT have?
OpenAI has not published full architecture details for current ChatGPT models. For reference, GPT-3's largest version used an embedding dimension of 12,288, with a vocabulary of about 50,000 tokens, meaning one 12,288-number vector per token.
Inside the model, every token is converted to an embedding vector, then transformed through many layers. Newer GPT models use larger vocabularies, such as the roughly 200,000-token o200k tokenizer used by GPT-4o, but exact embedding sizes are not disclosed. Separately, OpenAI offers embedding API models for developers: text-embedding-3-small outputs 1,536 dimensions and text-embedding-3-large 3,072. Those are the embeddings you can actually use in your own applications.
How many types of LLMs are there?
LLMs can be grouped by architecture, training and access: decoder-only models like GPT and Llama, encoder models like BERT, and encoder-decoder models like T5; base, instruction-tuned and reasoning models; and open-weight versus proprietary models.
Other useful categories: by size, from small models of one to a few billion parameters to frontier models; by modality, text-only versus multimodal models that handle images, audio and video; by architecture variant, dense versus mixture-of-experts; and by purpose, general assistants, coding models, embedding models and domain models for medicine or law. Embedding models are technically language models too, but they output vectors rather than text, which is why RAG systems use both kinds together.
How many types of vector databases are there?
Broadly three types: purpose-built vector databases such as Pinecone, Qdrant, Weaviate and Milvus; general databases with vector extensions such as PostgreSQL with pgvector, MongoDB Atlas and Redis; and search engines with vector support such as Elasticsearch and OpenSearch.
Libraries such as FAISS, Annoy and hnswlib form a fourth category: vector indexes without database features like persistence, filtering and replication. Deployment options add another dimension: managed cloud services, self-hosted open source, and embedded databases such as Chroma or LanceDB that run inside your application. Choose based on scale, hosting preferences, filtering and hybrid search needs, and whether you want vectors stored alongside existing relational or document data.
What is a good vector embeddings tutorial?
Start by generating embeddings for a few sentences with Sentence Transformers, computing cosine similarities between them, and seeing which pairs score highest. Then build a tiny semantic search over a list of documents, and visualise embeddings in two dimensions.
Also asked as: huggingface embeddings tutorial
Step by step: install sentence-transformers, load a small model such as all-MiniLM-L6-v2, call model.encode(sentences), and compare with util.cos_sim. Next, embed 100 short documents, embed a query, and print the top five matches. Plot embeddings with UMAP or PCA to see clusters. Finally, store them in FAISS or Chroma and add metadata. This sequence teaches the core ideas behind semantic search and RAG in an afternoon.
How do you use embeddings in LangChain?
LangChain provides embedding classes with a common interface: embed_documents for lists of texts and embed_query for a single query. Choose a provider class, such as OpenAIEmbeddings, HuggingFaceEmbeddings or OllamaEmbeddings, then pass it to a vector store.
Also asked as: langchain embeddings tutorial
A typical flow: load documents, split them with a text splitter, create an embeddings object, and build a vector store with FAISS.from_documents(chunks, embeddings) or a Chroma equivalent. Then create a retriever with as_retriever() and use it in a chain. Use the same embedding class for indexing and querying. Add caching with a cache-backed embeddings wrapper to avoid recomputing vectors for unchanged documents, which saves time and API costs during development.
How do you use Hugging Face embeddings?
The simplest route is Sentence Transformers: SentenceTransformer("all-MiniLM-L6-v2").encode(texts) returns embeddings for a list of texts. You can also call models through Hugging Face's Inference API or a Text Embeddings Inference server for production.
Choose a model from the MTEB leaderboard suited to your language and task, such as the BGE, E5 or GTE families, and read its model card, since some require prefixes like query: and passage: for best retrieval. Normalise embeddings for cosine similarity. For speed, batch inputs and use a GPU, or deploy Text Embeddings Inference, which serves models efficiently. LangChain and LlamaIndex wrap these models, so you can plug them into RAG pipelines directly.
Can you use embeddings for a recommender system?
Yes, embeddings are central to modern recommenders. Collaborative filtering learns latent vectors for users and items from interactions, and recommends items whose vectors score highly with a user's vector. Content embeddings from text or images handle new items.
Also asked as: can we use embeddings or latent vectors for a recommender system
Common architectures include matrix factorisation, two-tower models that embed users and items separately for fast retrieval, and sequence models that embed browsing history. At scale, item embeddings go into a vector index, the system retrieves hundreds of candidates by similarity, and a ranking model orders them using more features. Text embeddings of product descriptions help with cold start, when items have no interaction history yet. Evaluate with offline metrics and online A/B tests.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
