In a growing number of organizations, AI assistants now read email, summarize documents, pull information from the web, and increasingly take action on employees’ behalf. The problem is that AI tools follow instructions written in plain language, and they have no reliable way to tell whether an instruction came from you or from an attacker who used a technique called prompt injection to hide it inside a message or document the AI was asked to read.
This article explains what prompt injection is, why the risk of encountering it in the wild grows as you connect AI to inboxes, files, and customer systems, and what you can do to protect yourself.
What Is Prompt Injection and How Does It Work?
A prompt injection attack happens when someone feeds an AI tool a piece of text designed to override its original instructions and make it follow the attacker’s instead.
The prompt injection text usually doesn’t include any malicious code. Instead, the entire attack is made of ordinary words, meaning it has more in common with social engineering scams than ransomware and other malware.
The reason prompt injection works comes down to how the large language models (LLMs) behind tools like ChatGPT and Microsoft Copilot process text. The UK’s National Cyber Security Centre (NCSC) explains that an AI model makes no real distinction between the content it reads and the commands it follows. The model simply predicts which words should come next based on everything in front of it. Your instructions, the attacker’s instructions, and the document being summarized all arrive as one stream of words.
Prompt injection attacks can be divided into two broad groups based on how they work:
- Direct prompt injection happens when the attacker types malicious instructions straight into the chat window. A good example of a direct prompt injection attack happened in December 2023. A user named Chris Bakke convinced a Chevrolet dealership’s chatbot to agree with anything he said, then got it to accept one dollar for a 2024 Chevy Tahoe.
- Indirect prompt injection involves an attacker hiding instructions inside content the AI reads on an employee’s behalf, such as an inbound email, a shared document, a web page, or a calendar invite. In 2025, researchers at Aim Security demonstrated an attack called EchoLeak, in which a single crafted email could quietly instruct Microsoft 365 Copilot to gather internal company data and send it to the attacker, with the victim never clicking a thing. In this case, Microsoft patched the flaw before any customers were affected.
Of the two, indirect prompt injection is the one that should worry organizations more since copying and pasting outside content into an AI chat window has become a routine part of office work. That said, direct prompt injection can’t be completely ignored either, whether it comes from strangers poking at your public-facing chatbot or from insider threats aimed at your internal AI tools. The good news is that you can defend against both.
How to Protect Your Organization from Prompt Injection Attacks
The security tools you likely already have in place were not built with prompt injection attacks in mind. Firewalls, antivirus software, and email filters hunt for malicious code, suspicious attachments, and known attack signatures, and a prompt injection carries none of those, so there’s nothing for a traditional scanner to catch.
Just like with social engineering attacks, the goal of your defenses should be to raise awareness of the threat while limiting the damage any single successful attempt can cause.
With that mindset, here are some actionable defenses that can effectively protect your organization against prompt injections (the first two build awareness, the rest limit the damage):
- Train your team and put your AI rules in writing: Employees who know prompt injection exists are more careful about what they paste into AI tools and quicker to notice when an assistant starts acting strangely. Make the threat part of your regular AI training, and back it up with an AI acceptable use policy that spells out which tools are approved and what data must never go into them.
- Take inventory of every AI tool in use: You can’t protect what you don’t know exists, and employees adopt AI tools on their own faster than most leaders realize. This unsanctioned shadow AI gets expensive when things go wrong, as IBM’s 2025 Cost of a Data Breach Report found that breaches involving heavy shadow AI use cost an extra $670,000 on average, an amount that can decide whether a smaller company survives the incident.
- Give each AI tool the minimum access it needs: An assistant that summarizes email has no business reaching your HR files or accounting system. The Open Worldwide Application Security Project (OWASP) recommends least-privilege access and read-only permissions wherever possible because tight permissions limit what an attacker gains even when a trick succeeds.
- Keep a human in the loop for consequential actions: Anything that involves sending data outside the company, moving money, or deleting records should require a person’s confirmation before the AI proceeds. Microsoft uses this exact pattern in its own defenses against prompt injection. For instance, Copilot can draft an email for you, but you must approve and send it yourself, because Microsoft doesn’t count on blocking every injection.
- Never let one AI tool hold the full “lethal trifecta”: Security researcher Simon Willison coined this term for the combination of access to private data, exposure to untrusted content, and the ability to communicate externally. An AI agent with all three can be tricked into stealing your data and delivering it straight to the attacker, so at least one leg of the trifecta should always be missing.
- Ask the right question before buying AI tools: The NCSC specifically cautions against vendors who claim they can stop these attacks outright, because at the time of writing, nobody can. The trustworthy ones talk about reducing the risk, not eliminating it.
This whole topic deserves special attention from government contractors handling Controlled Unclassified Information (CUI). Under the Cybersecurity Maturity Model Certification (CMMC), any AI tool that touches CUI counts as a cloud service and must meet FedRAMP requirements (the federal security standard for cloud services), which rules out the consumer versions of ChatGPT, Claude, Copilot, and Gemini for that work.
Instead, contractors need an AI assistant that runs inside an authorized government cloud. For organizations on Microsoft 365, that’s Copilot in GCC High (Government Community Cloud High), which became generally available in December 2025 and is built to satisfy FedRAMP High, CMMC, and the other frameworks contractors live under. It even ships with the ability to pull content from the open web turned off by default to keep data inside the compliance boundary. For a deeper look at how the major platforms handle regulated environments, see our comparison of Copilot, Claude, and Gemini.
Conclusion
Prompt injection attacks can make AI tools turn against you with nothing but a few well-placed words. There’s no malware to detect and no password being stolen, which means the protection can’t come from your existing scanners. It comes from employees who know the trick exists and from AI tools set up so that a fooled assistant has little to reach and nothing it can do alone.
Whether you’re still thinking about rolling out AI and want to do it safely from day one, or your team already relies on it and you want to close the gaps, we at OSIbeyond can help put every defense described in this article in place for you. To get started, schedule a meeting with our team.