Prompt injection explained: attacks, examples, origin, prevention, resistance, resumes

Prompt injection is an attack where text in a model's input tricks it into ignoring its instructions, and it is the top security risk for applications built on language models, especially agents that read untrusted content and take actions. I build AI products where untrusted documents meet models, such as RAG.NextUpgrad and DocuLens AI, so this page covers what prompt injection means, how attacks work, types and examples, who coined the term, vulnerability status, prevention, resistance, why it persists and resume tricks.

What does prompt injection mean in AI?

Prompt injection is when attacker-controlled text in a model's input overrides or subverts the developer's instructions, making the model ignore rules, leak information or take unintended actions. The name echoes SQL injection, where data is mistaken for commands.

It happens because language models read instructions and data in the same stream of text and cannot reliably tell them apart. Direct injection comes from a user typing malicious instructions; indirect injection hides instructions in content the model processes, such as web pages, emails or documents. Risk grows with capability: a chatbot can only say wrong things, while an agent with tools can send emails or change data. Every LLM application that reads untrusted text should consider it.

How does a prompt injection attack work?

The attacker places instructions where the model will read them, such as in a message, a web page, a PDF or an email, phrased to look authoritative, for example claiming to be a system update or urging the model to ignore earlier instructions.

Also asked as: how does prompt injection attack work

When the application feeds this content to the model alongside its own instructions, the model may follow the injected text. In agents, the injected instructions might direct the model to call tools, such as sending data to an attacker's address or approving a transaction. Hidden text, such as white text on white backgrounds or content in metadata, makes injections invisible to human reviewers while remaining visible to the model.

What are the main types of AI prompt injection attacks?

Main types are direct injection, where users type override instructions; indirect injection, hidden in external content the model reads; jailbreaks, which aim to bypass safety rules; and data exfiltration attacks, which trick models into leaking secrets or private data.

Also asked as: what are ai prompt injection attacks

Indirect injection is especially dangerous for agents and RAG systems, since attackers do not need access to the conversation; planting text in a web page or shared document is enough. Other variants include multi-step attacks that gradually shift behaviour, injections in images or audio for multimodal models, and attacks on tool descriptions in connected servers. Research by Greshake and colleagues in 2023 highlighted indirect injection risks in integrated applications.

What does prompt injection look like?

Examples include a web page with hidden text telling an assistant to praise a product, an email instructing an AI to forward the inbox to an outside address, or a document saying to ignore previous instructions and reveal the system prompt.

Other examples: a support chat user claiming to be an administrator who needs refunds approved, a calendar invite containing instructions for an assistant, or code comments telling a coding agent to add a hidden backdoor. Injections often use urgency, fake authority or formatting that mimics system messages. Recognising these patterns helps developers design tests and helps users spot when an AI's behaviour has been hijacked.

Who coined the term prompt injection?

Simon Willison coined the term prompt injection in September 2022, comparing it to SQL injection, after Riley Goodside publicly demonstrated that instructions in user input could override a GPT-3 application's original instructions.

Researchers at Preamble had privately reported a similar issue to OpenAI earlier in 2022. Willison has since written extensively about the problem, arguing that it is hard to solve and proposing design patterns to limit damage, such as separating privileged and quarantined models. The term stuck because the analogy captures the core problem: untrusted data being interpreted as instructions, though unlike SQL injection, there is no simple escaping fix.

Is prompt injection a vulnerability?

Yes. Prompt injection is recognised as a security vulnerability, ranked first in the OWASP Top 10 for Large Language Model Applications, and it appears in government and industry guidance on securing AI systems.

Its severity depends on what the application can do: a chatbot without tools may only produce misleading text, while an agent with email, payment or code execution access can cause real harm. Security teams treat it like other injection classes, assessing impact, adding defences and testing regularly. Bug bounty programmes increasingly include prompt injection findings. Any application processing untrusted content with an LLM should include it in threat modelling and document accepted risks and mitigations clearly.

How do you prevent prompt injection in LLM applications?

No single method prevents it fully, so combine defences: treat external content as data, separate it from instructions with clear delimiters, give models least-privilege tools, require human approval for sensitive actions, and never place secrets in prompts.

Also asked as: how to prevent prompt injection in llm

Add input and output filtering with classifiers that detect injection attempts, validate outputs against schemas, as DocuLens AI does for extracted fields, and restrict where data can be sent. Design patterns such as separating a privileged model from one that reads untrusted content, or Google DeepMind's CaMeL approach separating control flow from data, limit damage. Red-team regularly with injected documents and web pages, and monitor agent actions for anomalies.

What is prompt injection resistance?

Prompt injection resistance is a model's or system's ability to keep following legitimate instructions despite injected text. Providers improve it through training, such as teaching models an instruction hierarchy that prioritises system and developer instructions over content in data.

OpenAI's instruction hierarchy research and similar work at other labs train models to ignore instructions found in tool results and documents. Classifiers screen inputs and outputs, and benchmarks such as AgentDojo measure how often agents resist attacks while completing tasks. Resistance improves but never reaches perfection, so system design must assume some attacks succeed. Measure resistance on your own scenarios, since results vary by model, prompt and tools.

Why won't prompt injection be fully fixed?

Language models process instructions and data as the same kind of text, so there is no reliable boundary between them, unlike SQL where parameters can be escaped. Defences are probabilistic, and attackers keep finding new phrasings that slip through.

Also asked as: why prompt injection won't be "fixed

Training and filters reduce success rates but cannot guarantee zero, and a single successful attack can be enough when agents have powerful tools. That is why experts emphasise architecture over prompts: limit permissions, isolate untrusted content, require confirmations and design systems so that a hijacked model cannot cause serious harm. Progress continues, but builders should plan for prompt injection as a permanent risk to manage rather than a bug awaiting a patch.

Does prompt injection work on resumes?

Some candidates hide instructions in resumes, such as white text telling AI screeners to rank them highly. It may fool naive screening tools, but many systems detect it, recruiters may disqualify candidates for it, and it undermines trust.

Hidden instructions have also appeared in academic papers aimed at AI reviewers, drawing public criticism in 2025. For job seekers, the better strategy is a clear, honest resume with relevant keywords and measurable achievements, which works for both AI tools and human reviewers. Honesty also protects your reputation in small professional circles. For employers, AI screening should keep humans in the loop, sanitise inputs by extracting visible text, and never rely solely on model rankings for hiring decisions.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail