RAG without a vector database, plus Postgres, SQLite, ClickHouse, Databricks and OpenAI

RAG does not strictly need a vector database: keyword search, SQL, graphs or plain in-memory indexes can retrieve context too. Vector databases help when you need semantic search at scale with filtering. In RAG.NextUpgrad I combine dense vector retrieval with BM25 through reciprocal rank fusion and reranking, so this page covers RAG without vector databases, what they are, how search and indexing work, choosing one, and options in Postgres, SQLite, ClickHouse, OpenAI and Databricks.

Can RAG work without a vector database?

Yes. RAG only needs a way to retrieve relevant context. Keyword search with BM25, SQL queries over structured data, knowledge graphs, file search with grep, or embeddings held in memory all work, depending on the data.

Also asked as: can rag work without vector database

For small collections of a few thousand chunks, store embeddings in a NumPy array or a FAISS index file and search in memory. For exact terms, codes and names, BM25 often beats vectors. Coding agents retrieve context by searching files directly. Long-context models can sometimes take whole documents without retrieval. A vector database becomes worthwhile when you need semantic search over large, changing collections with filters, concurrency and persistence. Hybrid retrieval, as in RAG.NextUpgrad, combines keyword and vector signals.

What is a vector database in simple terms?

A vector database stores items as lists of numbers, called embeddings, that capture their meaning, and quickly finds the items whose numbers are closest to a query's. It searches by meaning rather than exact words.

Also asked as: what is vector database in simple terms

For example, an embedding model turns the sentence how do I reset my password into a vector, and the database finds help articles with similar vectors, even if they say account recovery instead. Vector databases add features around this core: metadata filters, updates, scaling across machines, access control and backups. Examples include Qdrant, Weaviate, Milvus, Pinecone and Chroma, while general databases such as PostgreSQL add vector support through extensions.

How does vector database search work?

The query is converted into an embedding with the same model used for documents. The database then finds the nearest stored vectors by a similarity measure, such as cosine similarity, using an index that avoids comparing every vector.

Approximate nearest neighbour indexes, such as HNSW, explore a small part of the collection to find likely nearest neighbours quickly, trading a little accuracy for large speed gains. Metadata filters, such as document type or date, narrow results before or during search. The top results, with scores and stored text, return to the application, which often reranks them with a cross-encoder before sending them to the LLM. Search quality depends heavily on chunking and the embedding model.

How does vector database indexing work?

Indexing organises vectors so similar ones can be found without scanning everything. HNSW builds a layered graph linking each vector to its neighbours; IVF clusters vectors and searches only nearby clusters; quantization compresses vectors to save memory.

When you insert a vector into an HNSW index, the database connects it to nearby vectors at several layers, so searches can hop quickly from distant regions to the closest matches. Parameters control the trade-offs: more connections and wider search improve recall but use more memory and time. IVF indexes need training on sample data to define clusters. Updates and deletions affect index quality over time, so some systems rebuild or compact indexes periodically.

How do you choose a vector database for RAG?

Choose based on scale, existing stack, filtering needs, hybrid search support, operations effort and cost. For many projects, pgvector in an existing PostgreSQL database is enough, while dedicated databases suit large or demanding workloads.

Also asked as: how to choose vector database for rag

Questions to ask: how many vectors now and in a year; do you need fast filtering by tenant, date or permissions; do you need keyword and vector hybrid search; should it be managed or self-hosted; and what latency is acceptable. Benchmark candidates on your own data, measuring recall and latency with realistic filters. Remember that chunking, embeddings and reranking usually matter more for answer quality than the database choice itself. Avoid premature complexity.

Can you use Postgres as a vector database?

Yes. The pgvector extension adds a vector data type, distance operators and HNSW and IVFFlat indexes to PostgreSQL, letting you store embeddings beside relational data and query them with SQL.

Also asked as: can i use postgres as vector database

This suits many RAG applications: keep documents, users, permissions and embeddings in one database, filter with ordinary WHERE clauses, and combine vector similarity with PostgreSQL full-text search for hybrid retrieval in a single query. Managed services such as Supabase, Neon and major clouds support pgvector. Performance is strong up to millions of vectors with proper indexes and memory, and extensions such as pgvectorscale extend it further. Very large or high-throughput workloads may still benefit from dedicated vector databases.

Can you use SQLite as a vector database?

Yes, with extensions. sqlite-vec adds vector storage and similarity search to SQLite, making it practical for local apps, desktop tools, mobile devices and prototypes that need RAG without running a separate server.

Also asked as: can i use sqlite as a vector database

SQLite keeps everything in one file, which simplifies deployment and backups, and combines well with SQLite's FTS5 full-text search for hybrid retrieval. It suits collections from thousands to a few million vectors on a single machine, depending on hardware and query patterns. It is less suited to many concurrent writers or distributed scaling. For offline or privacy-sensitive applications, such as personal document assistants, SQLite with local embeddings keeps all data on the user's device.

Can ClickHouse replace a vector database?

For many workloads, yes. ClickHouse supports vector distance functions and approximate vector similarity indexes, so it can store embeddings alongside large analytical datasets and run similarity searches with SQL filters.

It fits teams already using ClickHouse for analytics, logs or events who want semantic search without another system, such as finding similar support tickets across millions of records with time and category filters. Dedicated vector databases may still offer better tooling for frequent updates, low-latency serving, hybrid search features and RAG integrations. Evaluate recall, latency and update patterns on your data. Choosing a database your team already operates well often beats adopting a specialised one.

What vector database does OpenAI use for ChatGPT?

OpenAI does not publicly detail the internal retrieval stack behind ChatGPT. It acquired Rockset, a real-time search and analytics database company, in 2024 to strengthen its retrieval infrastructure, and its API offers managed vector stores for file search.

Also asked as: what vector database does chatgpt use

For developers, the File Search tool lets you upload files to an OpenAI-managed vector store, which handles chunking, embedding and retrieval automatically, with configurable options. This is convenient but less flexible than running your own retrieval, where you control chunking, hybrid search, reranking and citations. Many teams use OpenAI embeddings with their own database, such as pgvector or Qdrant, to keep control over data and ranking, and to switch providers more easily.

Does Databricks have a vector database?

Yes. Databricks offers Mosaic AI Vector Search, a managed vector search service integrated with Unity Catalog and Delta tables, which can automatically sync embeddings when source data changes.

Also asked as: does databricks have vector database

It suits organisations whose data already lives in Databricks, since indexes stay governed by the same permissions and lineage as other data. You can compute embeddings with Databricks-hosted models or supply your own, and query indexes from RAG applications built with Databricks tools or external frameworks. Features include filtering and hybrid keyword and vector search. As with any managed option, compare cost, latency and flexibility against pgvector or dedicated databases for your workload.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail