A document set that updates weekly doesn't need the whole corpus re-embedded every time — most of it hasn't changed. Re-embedding everything anyway is the default in a lot of tutorials, and it's needlessly slow and costly at any real scale.
A practical incremental approach
- Hash each source document (or each chunk) and store the hash alongside its embedding.
- On an update run, compute new hashes and diff against stored ones — only re-embed documents whose hash changed.
- Delete embeddings for documents that were removed entirely, not just leave them stale in the index.
The trap to avoid
Chunk boundaries can shift even for a small edit near the start of a document, which changes every downstream chunk's hash. Chunk-level hashing catches real changes; document-level hashing is simpler but coarser — pick based on how often your documents change internally versus wholesale.
See metadata filtering in RAG pipelines for a related ingestion-time concern.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
