Half the projects at any student hackathon now include an AI component, and most panels have no reliable way to tell a real one from a wrapper around a prompt. Having built RAG, fine-tuning and vision systems for a living, these are the checks I run in three minutes at a judging table — and the ones I share with fellow judges beforehand.
Is the inference real?
- Ask for an input the team did not prepare. A hard-coded response will not adapt.
- Ask to see the request go out — a network tab, a log line, a token counter. Real calls leave traces.
- Ask what the model was given as context. Teams who built it can answer instantly.
Did they evaluate anything?
"We tested it on a few examples" is acceptable at a hackathon if they can show the examples and the outcomes. A team with ten test questions and a note on which three failed is showing engineering maturity; score it. A team that claims 95% accuracy with no test set is showing a slide.
How does it fail?
Ask what the system does when the model is wrong, slow or unavailable. A confidence threshold, a fallback, a clear error message or a human hand-off all count. "The AI handles it" does not. The human decisions around the model are what distinguish a product from a demo.
Where does the data go?
Any project touching faces, health, documents or children should be able to say what leaves the device, what is stored and for how long. A team that thought about this deserves credit even if the implementation is thin.
Is the AI the right tool?
Some of the best projects use AI for one narrow step and plain code for the rest. Some of the weakest bolt a chatbot onto a problem that needed a form. Score appropriateness, not quantity of AI.
Scoring it fairly
- Do not reward the team with the biggest model; reward the team with the clearest understanding.
- Do not penalise a scoped-down but real inference against a broad but mocked one.
- Write the AI-specific feedback down; it is the part teams most need and least often get.
Organisers running AI-themed hackathons: brief your judges with this list, or invite one who already uses it. I am open to judging student hackathons on-site or remotely; details on the campus invite page.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
