Building a support-ticket triage system with an LLM: a project walkthrough

Classifying, prioritising and routing incoming support tickets is a genuinely useful, well-scoped AI project with a clear before-and-after metric.

At an Integral Startup Foundation hackathon
At an Integral Startup Foundation hackathon

Automatically classifying, prioritising and routing incoming support tickets is a genuinely useful project — it has a clear success metric (faster resolution, correct routing) and forces real engineering around classification and evaluation.

The pipeline

  1. Classify the ticket by category (billing, technical, general) and urgency, using a prompt or a small fine-tuned classifier.
  2. Route to the appropriate team or queue based on the classification.
  3. Draft a suggested first response for the agent to review and send, rather than sending automatically — keep a human in the loop for anything customer-facing.

How to evaluate it honestly

Compare classification accuracy against how tickets were actually routed historically (if that data exists), and track how often agents edit or reject the suggested draft response — a high rejection rate signals the drafting step needs work, not just the classification step.

See fine-tuning for classification vs generation.

ShippingClassificationProject
Pranjul Rathour
Pranjul Rathour
GenAI Engineer · Kanpur, Uttar Pradesh, India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.

Keep reading

More on Shipping AI

All Shipping AI posts →
Campus talks · hackathon judging · mentoring

Want this as a live session at your college?

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges.

Pranjul Rathour, GenAI engineer in Kanpur