Key takeaways
- Chroma and ChromaDB are the same open-source vector database; the Python package is chromadb.
- In LLM apps, Chroma stores document chunks with their embeddings and metadata and returns the most similar ones for a query.
- Use a persistent client and stable IDs so you never re-embed unchanged documents.
- In-memory storage is fine for tests and notebooks, but production data needs persistence and backups.
- 'Illegal instruction' crashes usually mean a CPU or architecture mismatch with compiled dependencies.
Chroma, often called ChromaDB, is an open-source vector database for AI applications: it stores text chunks, their embeddings and metadata, and returns the most similar chunks for a query, which is the retrieval step in retrieval-augmented generation (RAG). It is popular for prototypes because it runs inside Python with a few lines of code. This page covers the name, its role in LLM apps, Python usage and examples, avoiding repeated embedding costs, in-memory versus persistent storage, a common Docker crash, testing RAG with Chroma, and the unrelated colour term.
What is the difference between Chroma and ChromaDB?
There is no difference: Chroma is the project and company name, and ChromaDB is the common name for the database and its Python package, installed with pip install chromadb.
Also asked as: chroma vs chromadb
Chroma runs in several modes: embedded inside your Python process, as a client-server setup with a separate Chroma server, or as a hosted cloud service. Tutorials use "Chroma", "ChromaDB" and "Chroma DB" interchangeably. In LangChain and LlamaIndex, the integration is usually called Chroma. If you see "Chroma" in a photography or colour context, that is an unrelated meaning covered at the end of this page.
What is ChromaDB in LLM applications?
In LLM applications, ChromaDB is the vector store that holds your knowledge: document chunks with their embeddings. When a user asks a question, the app searches Chroma for similar chunks and passes them to the LLM so it can answer from your documents.
Chroma provides collections, which are like tables, with documents, embeddings, metadata and IDs. It can compute embeddings for you with a default or chosen embedding function, or accept embeddings you compute yourself. Metadata filters let you restrict search, for example to one course or date range. It uses an approximate nearest-neighbour index, HNSW, for fast search. For many student projects and internal tools, Chroma is all the vector database you need.
What does Chroma do in Python?
In Python, Chroma lets you create collections, add documents with IDs and metadata, and query them by text or embedding to get the most similar items, all from a simple API that can run in-process without a separate server.
Also asked as: what does chroma do python · how to use chroma db in python
Typical operations:
client.get_or_create_collection(name)to create or open a collection.collection.add(ids=..., documents=..., metadatas=...)to index chunks.collection.query(query_texts=[...], n_results=5, where={...})to search with filters.collection.update,upsertanddeleteto keep data current.
Results include IDs, documents, metadata and distances, which you pass to your LLM prompt with IDs for citations.
How do I use ChromaDB in Python?
Install chromadb, create a persistent client, create a collection, add documents with unique IDs, and query it.
import chromadb
client = chromadb.PersistentClient(path="./chroma_store")
col = client.get_or_create_collection("notes")
col.upsert(
ids=["os-1", "os-2"],
documents=["A process is a program in execution.", "Paging divides memory into fixed-size pages."],
metadatas=[{"subject": "os", "page": 3}, {"subject": "os", "page": 7}],
)
res = col.query(query_texts=["what is paging?"], n_results=1, where={"subject": "os"})
print(res["documents"][0], res["metadatas"][0])
The default embedding function downloads a small model on first use; for production, choose your embedding model explicitly and use it consistently.
What are some Chroma examples?
Common Chroma examples are a chatbot over your PDFs, semantic search over notes or support tickets, a FAQ assistant for a college or product, and memory for an AI agent that stores past conversation summaries.
Project ideas to practise:
- Study assistant: index lecture notes by page and answer questions with page citations.
- Support search: index past tickets and suggest similar resolved issues.
- Code search: embed function docstrings and search your codebase in plain language.
- Agent memory: store summaries of past tasks and retrieve them when relevant.
Pair Chroma with keyword search for exact terms such as error codes, which embeddings often miss, and add a reranker when results look plausible but wrong.
How do I avoid recomputing embeddings with Chroma?
Use a persistent client so embeddings survive restarts, give each chunk a stable ID, and upsert only new or changed chunks, tracking a hash of each document to detect changes. Never rebuild the whole collection when one file changes.
Also asked as: how to avoid re computation of embeddings with chroma
A simple pattern: compute an ID such as docname#chunkindex and a content hash; before indexing, check which IDs already exist with the same hash; embed and upsert only those that are new or changed; delete IDs for removed content. If you compute embeddings yourself, cache them on disk keyed by content hash. Remember that changing the embedding model requires re-embedding everything, because vectors from different models are not comparable.
Is it a problem to store a vector database in memory?
For prototypes, tests and small datasets, in-memory storage is fine and fast. For anything users depend on, it is a problem: data disappears when the process restarts, memory limits cap your dataset, and multiple app instances cannot share it.
Also asked as: is it a problem to store your vector database in memory
Use in-memory clients for unit tests and notebooks, and persistent storage in development and production, with backups. If you run several app instances, use a client-server Chroma or a database such as PostgreSQL with pgvector, so all instances see the same data. Also keep your source documents as the true record: the vector index should be rebuildable from them, which makes recovery and model changes straightforward.
Why does ChromaDB crash with "illegal instruction" in a Docker container?
"Illegal instruction" usually means a compiled dependency, such as the HNSW index library or an ML runtime, was built for CPU instructions your machine or emulator does not support. It often happens when running an image built for a different CPU architecture, such as amd64 under emulation on an ARM Mac.
Also asked as: why does chroma db crash with "illegal instruction" in this python:3.10-slim container
Fixes to try:
- Run an image that matches your machine's architecture, or build the image natively on it.
- Avoid emulation for this workload where possible.
- Use the official Chroma server image and connect from your app as a client.
- Rebuild the dependency from source without CPU-specific optimisations, following Chroma's troubleshooting docs.
- Pin versions known to work on your platform.
Check Chroma's GitHub issues for your exact error and platform; others usually hit it first.
How do you test RAG memory with ChromaDB?
Treat retrieval as testable code: create a fresh in-memory or temporary Chroma collection in each test, load a fixed set of documents, run known queries and assert that the expected IDs come back in the top results.
Also asked as: how automating rag memory tests with chromadb quadrupled our bug discovery rate
What to test:
- Retrieval correctness: expected chunk IDs appear in the top k for each query.
- Updates: changed documents replace old chunks; deleted ones disappear.
- Filters: metadata filters exclude what they should.
- Agent memory: stored summaries are retrieved in later turns and old ones expire as designed.
- Regressions: run the suite on every change to chunking or embeddings.
Automated retrieval tests catch silent failures, such as duplicate chunks or broken updates, long before users notice wrong answers.
What is colour chroma?
In colour theory, chroma is the colourfulness or intensity of a colour relative to a neutral grey of the same brightness: high chroma colours are vivid, low chroma colours are muted or greyish. It is unrelated to the Chroma vector database.
Chroma is one of the three dimensions in colour systems such as Munsell, alongside hue and value or lightness. "High chroma vs low chroma" comparisons appear in painting, design, photography and fashion; for example, a bright red is high chroma, while a dusty rose is low chroma. In photography, "chroma noise" refers to random colour speckles in images. If you came here for databases, the sections above cover ChromaDB.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
