RAG over a SQL database: why text-to-SQL is a different problem

Vector search doesn't help when the answer is 'run this aggregate query'. A practical pattern for letting an LLM answer questions against a real database safely.

In a packed college auditorium
In a packed college auditorium

'How many students scored above 80 last semester' is not a retrieval question — it's an aggregate query. Embedding the database rows and searching semantically will not answer it correctly.

The pattern: schema-grounded SQL generation

Give the model the table schema (not the data), a handful of example question-to-SQL pairs, and let it generate a query. Run the query against a read-only replica, then hand the result — not the raw SQL — back to the model to phrase as an answer.

  • Never let the generated SQL run against a writable connection — read-only credentials, always.
  • Validate the query touches only expected tables before executing it.
  • Show the user the generated SQL alongside the answer; it is the fastest way to catch a wrong query.

This is a genuinely different pipeline from document RAG — see RAG vs fine-tuning: when to use which for how to decide which problem you actually have.

RAGText-to-SQLDatabases
Pranjul Rathour
Pranjul Rathour
GenAI Engineer · Kanpur, Uttar Pradesh, India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.

Keep reading

More on RAG & retrieval

All RAG & retrieval posts →
Campus talks · hackathon judging · mentoring

Want this as a live session at your college?

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges.

Pranjul Rathour, GenAI engineer in Kanpur