Vector databases explained: history, how they work with LLMs, SQL vs NoSQL, AWS options

A vector database stores embeddings, numerical vectors that represent meaning, and finds the most similar ones to a query vector quickly, even among billions, using approximate nearest neighbour indexes. Vector databases became central to AI applications with RAG and semantic search. RAG.NextUpgrad pairs dense vector retrieval with BM25, so this page covers their history, how they work with LLMs, why they matter, whether they are dead, NoSQL or SQL, whether they are trained, examples and AWS options.

When were vector databases invented?

The underlying ideas are decades old: nearest neighbour search research dates to the 1970s, locality-sensitive hashing to 1998, product quantization to 2011, and the HNSW graph index to 2016. Dedicated vector databases emerged around 2017 to 2019.

Facebook released the FAISS library in 2017, Milvus was open-sourced in 2019, and companies such as Pinecone, Weaviate, Qdrant and Chroma launched between 2019 and 2022. The category exploded in 2023, when RAG made vector search essential for LLM applications. At the same time, established databases added vector support, such as PostgreSQL's pgvector, Elasticsearch, OpenSearch, MongoDB and Redis, so today vector search is both a product category and a common database feature.

How do vector databases work with LLMs?

In a RAG system, documents are split into chunks, converted into embeddings and stored in a vector database. When a user asks a question, the question is embedded, the database returns the most similar chunks, and the LLM answers using them.

Also asked as: how vector databases work with llm

The database handles the heavy lifting: indexing millions of vectors with structures such as HNSW, filtering results by metadata such as user permissions or dates, and returning results in milliseconds. Many also support hybrid search, combining keyword scores with vector similarity. Vector databases also give LLM applications memory, storing past conversations or facts as embeddings to retrieve later. The LLM itself never reads the whole database; it only sees the retrieved passages.

Why are vector databases important for AI?

AI models represent meaning as vectors, and vector databases make those representations searchable at scale. They power semantic search, RAG, recommendations, image and audio search, deduplication and anomaly detection, where finding similar items quickly is the core operation.

Without an index, finding the nearest neighbours means comparing a query with every stored vector, which is too slow for millions of items. Approximate nearest neighbour indexes trade a tiny loss in recall for searches thousands of times faster. Vector databases add production features around this: persistence, updates, filtering, replication, access control and monitoring. That combination lets AI applications ground answers in private, current data rather than relying only on what a model memorised.

Why use a vector database?

Use a vector database when you need fast similarity search over many embeddings with filters, updates and reliability. For small collections, a simple in-memory array or a library like FAISS may be enough.

Signs you need one: more than a few hundred thousand vectors, frequent inserts and deletes, metadata filtering by user, tenant or date, multiple concurrent users, or requirements for backups and high availability. If you already run PostgreSQL, pgvector may cover your needs without a new system. Choose a dedicated vector database for very large collections, heavy filtering or advanced features such as quantization and multi-tenancy. Let measured scale and operational needs, not hype, drive the decision.

Are vector databases dead?

No, but the market has changed. Vector search is now a feature in many general databases, long context windows reduce some retrieval needs, and some coding agents rely on keyword and file search instead of embeddings, which led to dead-or-alive debates.

For large knowledge bases, retrieval remains essential: even million-token contexts are slower and costlier than retrieving relevant passages, and models can miss details in very long inputs. What changed is that vector search is often one component of hybrid retrieval, alongside keyword search, reranking and structured queries, and it frequently lives inside an existing database. Dedicated vector databases remain relevant for scale and advanced features, while many teams use pgvector or search engines instead.

Are vector databases NoSQL?

Dedicated vector databases are usually classed as NoSQL, since they are non-relational systems built around collections of vectors with JSON-like metadata, queried through APIs rather than SQL. But vector search also exists inside relational databases.

Pinecone, Qdrant, Weaviate, Milvus and Chroma use their own APIs and data models, resembling document stores with vector indexes. Document databases such as MongoDB and key-value stores such as Redis add vector search to NoSQL foundations. Relational databases, including PostgreSQL with pgvector, Oracle and SQL Server, add vector types to SQL tables. So the label depends on the product; what matters is how you query, filter and scale your data.

Do vector databases use SQL?

Some do. PostgreSQL with pgvector, SingleStore, ClickHouse, DuckDB with vector extensions, Oracle and SQL Server support vector search through SQL. Most dedicated vector databases use their own APIs with filter expressions instead of SQL.

Also asked as: why use vector databases

With pgvector, a similarity query is ordinary SQL: order rows by the distance operator between the embedding column and a query vector, add WHERE clauses for filters, and limit the results. That lets you join vectors with users, permissions and other tables in one query. Dedicated databases offer client libraries where you pass a vector and a filter object. SQL support is valuable when vectors must live alongside relational data and transactions.

Are vector databases trained?

Generally no. The embedding models that create vectors are trained; the database stores and indexes those vectors. However, some index types do require a training step on sample data, such as clustering for IVF indexes or codebooks for product quantization.

For example, a FAISS IVF index first runs k-means on a sample of vectors to create clusters, then assigns all vectors to them; product quantization learns compression codebooks the same way. HNSW indexes need no training: they build a graph as vectors are inserted. If your data distribution changes greatly, retraining or rebuilding such indexes can restore accuracy. None of this changes the embeddings themselves, which come from your embedding model.

What are some examples of vector databases?

Dedicated examples include Pinecone, Milvus and Zilliz Cloud, Weaviate, Qdrant, Chroma, LanceDB and Vespa. Databases with vector support include PostgreSQL with pgvector, Elasticsearch, OpenSearch, MongoDB Atlas, Redis, DataStax Astra DB and SQLite with sqlite-vec.

Also asked as: what are vector databases examples

Libraries such as FAISS, Annoy, ScaNN and hnswlib provide vector indexes without database features, useful inside applications or for research. Choose based on hosting, managed or self-hosted; scale; filtering and hybrid search needs; ecosystem integrations; and your team's existing infrastructure. For a first RAG project, Chroma or pgvector is easy; for large production systems, compare Qdrant, Milvus, Weaviate, Pinecone or your search engine of choice on your own data.

What are vector databases in AWS?

AWS offers vector search in several services: Amazon OpenSearch Service and OpenSearch Serverless with k-NN search, Aurora PostgreSQL and RDS for PostgreSQL with pgvector, Amazon MemoryDB, DocumentDB, Neptune Analytics, and Amazon S3 Vectors for low-cost storage of large vector sets.

Amazon Bedrock Knowledge Bases builds managed RAG on top of these, handling chunking, embedding and retrieval with a choice of vector stores, including some third-party databases such as Pinecone. Choose OpenSearch for hybrid search at scale, Aurora with pgvector when data is relational, MemoryDB for very low latency, and S3 Vectors for large, infrequently queried collections. Third-party vector databases are also available through AWS Marketplace or self-hosted on EC2 and EKS.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail