A free-tier hosting instance typically gives you 512 MB of memory. That number sounds tiny until you realise most of the "expensive" parts of an AI app are optional, not fundamental.
The biggest single cost: local embeddings
Loading torch plus sentence-transformers for local embedding generation alone needs roughly 500 MB — before your application code runs at all. Switching to a hosted embedding API removes that entire cost, at the price of a network call per embedding.
What actually fits
FAISS as a local vector store, a lightweight web framework, BM25 for keyword search, and hosted calls for embeddings and generation — that combination is how RAG.NextUpgrad runs in about 220 MB.
When to actually pay for compute
Once you need local embeddings for latency or cost-at-scale reasons, or you're running your own reranker model instead of a hosted one. Until then, the free tier teaches you more about your dependencies than a rented GPU ever will.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
