Key takeaways
- A system prompt is the instruction block that sets an AI model's role, rules and context before the conversation.
- User prompts are the requests; models are trained to give system or developer instructions higher priority.
- Write system prompts with role, goal, context, rules, output format and examples, then test them.
- Keep them as long as needed but no longer; caching makes long, stable prompts cheaper.
- System prompts can leak, so never put secrets in them and enforce real permissions in code.
A system prompt is the set of instructions given to an AI model before a conversation starts, defining its role, behaviour, rules, knowledge and output format. Users usually do not see it, but it shapes every answer. Every AI product I build, from RAG.NextUpgrad's cited answers to DocuLens AI's document checks, depends on carefully tested system prompts. This page explains system and user prompts, how they work and why they matter, how to write and size them, leaks, ChatGPT and Claude prompts, and LiteLLM.
What are system prompts?
System prompts are instructions sent to a language model in a special system role, separate from user messages, telling it who it is, what it should and should not do, and how to respond. Developers write them; end users typically never see them.
Also asked as: what are system prompts and user prompts · how to write system prompts · how big should system.prompts be
Examples: "You are a support assistant for a college admissions office. Answer only from the provided documents, cite sources, and say you do not know if the documents do not cover a question." A system prompt can include persona, task, domain knowledge, safety rules, tool instructions and format requirements. Some platforms call it the developer message. It is the main lever for customising a general model into a specific product.
What is the difference between system prompts and user prompts?
The system prompt sets standing instructions for the whole conversation and comes from the developer or platform. User prompts are individual requests from the person chatting. Models are trained to weigh system instructions above conflicting user requests.
For example, a system prompt might say "Reply only in English and never give medical diagnoses", while a user prompt asks "Explain my symptoms in Hindi". The model should follow the system's language rule and safety boundary while still helping. This priority is not absolute; clever user inputs can sometimes override instructions, which is why important restrictions belong in code and permissions too, not only in prompts.
How do system prompts work?
The system prompt is placed at the start of the model's input, marked with a system role, before user and assistant messages. The model reads it as context on every turn, and training teaches it to follow system instructions consistently.
Technically, the whole conversation, system prompt included, is converted into tokens with special markers for each role, and the model predicts its reply from all of it. Nothing is "installed" in the model; the instructions are re-sent with every request, which is why they consume tokens each time. Providers' training, such as instruction hierarchies, makes models treat system-role text as more authoritative than user text.
Why are system prompts important in AI?
System prompts turn a general-purpose model into a reliable, specific product. They set scope, tone, safety boundaries and output formats, and inject context such as policies or company facts, without retraining the model.
Also asked as: why is the system prompts important in ai
In practice, the system prompt is often the biggest single factor in an AI product's quality after the choice of model. A RAG assistant's prompt decides whether it cites sources and refuses when evidence is weak; an extraction tool's prompt decides whether output is valid JSON. Changing a few sentences can fix or break behaviour, so system prompts deserve version control, review and automated tests like any other code.
How do I write a good system prompt?
State the role and goal, give necessary context, list rules and boundaries, specify the output format, include one or two examples, and explain the reasons behind important rules. Then test against real and adversarial inputs and revise.
A structure that works:
- Role and goal: who the assistant is and what success looks like.
- Context: audience, domain facts, tools available.
- Rules: what to always and never do, with reasons.
- Format: length, structure, language, citations.
- Edge cases: what to do when information is missing or a request is out of scope.
- Examples: a good answer to a typical request.
Keep instructions clear and non-contradictory, and test every change on a fixed set of prompts.
How big should a system prompt be?
As long as needed to specify behaviour clearly, and no longer. Simple assistants need a few hundred words; complex products can use several thousand tokens. Production assistants from major AI companies use very long system prompts.
Trade-offs: longer prompts cost more tokens per request and add latency, and too many rules can conflict or dilute attention. Prompt caching, offered by several providers, makes long, unchanging system prompts much cheaper and faster after the first request. Put stable instructions first and variable context later so caching works. Remove rules that tests show are unnecessary, and move large reference material into retrieval rather than the prompt.
How are system prompts leaked?
System prompts leak mainly through prompt extraction: users ask the model to repeat its instructions, often disguised as role-play, translation or formatting tasks. Leaks also happen through logs, client-side code, or tools that echo context.
Many popular products' system prompts have been extracted and posted publicly. Treat system prompts as non-secret: never put API keys, passwords or confidential data in them, and enforce access control in code, not in instructions. You can reduce leaks by instructing the model not to reveal its instructions and filtering outputs, but determined users often succeed. OWASP lists system prompt leakage as a top risk for LLM applications.
What are ChatGPT's system prompts?
ChatGPT uses hidden system prompts written by OpenAI that set its identity, current date, capabilities, tool instructions and behaviour rules. OpenAI does not publish them in full, though versions extracted by users circulate online and may be outdated or inaccurate.
Also asked as: what are chatgpt system prompts
OpenAI publicly describes intended behaviour in its Model Spec, including a chain of command where platform and developer instructions outrank user instructions. In the API, developers write their own system or developer messages, and Custom GPTs let users add instructions. For your own applications, write instructions specific to your product rather than copying leaked prompts, which are designed for a different, general-purpose context.
What are system prompts in Claude?
In Claude's API, the system prompt is a separate system parameter that sets Claude's role and instructions for the conversation. For its own apps, Anthropic publishes the system prompts used in Claude's web and mobile apps in its documentation release notes.
Anthropic recommends using the system prompt to give Claude a role, such as a senior data analyst, and putting task-specific instructions and documents in user turns, often structured with XML tags. Published app prompts show techniques such as stating the current date, describing capabilities and explaining reasoning behind rules. When building with Claude, test prompts against your own examples.
Can LiteLLM inject system prompts?
Yes. LiteLLM's proxy supports hooks that run before each call, which can modify requests, including adding or changing the system message, so you can enforce a shared system prompt across applications and model providers.
This is useful when many apps call models through one gateway and you want consistent rules, such as safety instructions or formatting, without changing each app. Implement it with a pre-call hook in the proxy configuration and test that it merges correctly with any system message the app already sends. Also check LiteLLM's documentation for built-in prompt management features. Log which prompt version was applied to each request for debugging.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
