A newsroom of AI agents spends its working day reading text written by strangers. Fourteen feeds supply it, each item free to say whatever its author wants, and a published article comes out the other end. That is the standard shape of an injection problem.
I would like to say I threat-modelled it before the first agent ran. I did not. The first version of the pitch stage treated feed text as plain input, and it was the security review of 1 October that turned it into untrusted data. The newsroom has since published seven articles, and this piece sets out the threat model I should have written first: what an attacker who controls one feed item can try, which controls stand in the way, what has been tested, and what I cannot prevent.
The short version is that prompt injection is not a prompt problem. No instruction I write inside a prompt will reliably stop a model from following instruction-shaped text it reads. What works is architecture: decide in advance what a fully successful injection could do, and make that very little.
This is the second piece in a series on the SSI newsroom. The first, Governing an Agentic Newsroom, introduced the six boundaries. This one goes deep on the second: untrusted input.
Where the trust boundaries sit
| Stage | Who controls the content | What it can do |
|---|---|---|
| Feeds and fetched source pages | Strangers | Supply text, nothing else |
| Input gate | Code | Sanitise, truncate and tag the text as untrusted data |
| Model calls | Can be fooled | Produce text. No call has tools |
| Output gate | Code | Validate schema, escape markup, verify quotes against the source |
| Pull request and merge | A person | Decide whether anything goes live |
Hostile text enters at the top and only ever reaches a model after a code-only gate. What the model emits passes a second code-only gate before it becomes a pull request, and a person decides whether it goes live. Outbound fetches made for fact-checking pass through a guarded HTTP client.
Six paths hostile content can take
I stopped asking whether the prompts were injection-proof and started enumerating where attacker-controlled text can travel. In this system there are six paths. Each one has a different failure mode, and only the first is about prompts.
| Path | What the attacker controls | Control | Evidence |
|---|---|---|---|
| Feed text to journalist prompt | Title and summary of a feed item | Text is sanitised and truncated before any model sees it, then wrapped in tagged blocks declared as untrusted data. No model call has tools. | Ingest test with an "ignore all previous instructions" item inside an event cluster |
| Feed text to source links | URLs embedded in an item | Links in a draft must appear verbatim in the event material. Anything else is replaced by a sentinel URL that fails review. | Verbatim-URL rule exercised in writer and fact-check tests |
| Model output to published page | Markup the model may emit, steered or not | PHP tags are rejected at schema level, HTML is escaped before citation markers are resolved, and source-flag fields are set by code and hidden from the model schema. | Tests with PHP and script payloads in drafts |
| Untrusted text to CI logs | Characters that look like workflow commands | Every untrusted string is collapsed to one line before it is logged, so a feed item cannot emit a runner command. | One-line confinement helper in the pipeline |
| Untrusted URL to outbound fetch | Where the fact-checker connects | A guarded HTTP client refuses internal and non-routable targets and redirects to them. A DNS rebinding window remains and is documented as residual risk. | Parametrised SSRF target tests |
| Fetched page to verifier | A source page that tries to instruct the claim checker | The page is treated as untrusted data, the checker must return a quote, and code, not a model, confirms the quote exists verbatim in the source. | Claim-check tests on quote verification |
The pattern is that the model is never the last line of defence on any of these paths. Where a model touches hostile text, a deterministic step sits either before it (sanitise, truncate, tag) or after it (schema, escape, string match). If the model is fully convinced by an attacker, the worst outcome is a bad draft that a human still has to merge.
Controls in four layers
The controls are easier to reason about as layers than as a list, because each layer assumes the one before it has failed.
The first layer is the input boundary. Feed text is treated as hostile from the moment it is parsed. It is sanitised, truncated and grouped into events before it is allowed near a model. This layer does not try to detect injection, because detection is a losing game against free text. It reduces what an attacker can say and how much room they have to say it.
The second layer is the model boundary. Every model call receives hostile material inside explicitly tagged blocks, with instructions that this content is data. More important than the wording is the capability budget: no model call in the pipeline has tools. A model that is persuaded to do something has nothing to do it with. It can only produce text, and text is handled by the third layer.
The third layer is the output boundary. Structured output is validated against a schema, markup that could execute is rejected, and everything else is escaped. Fields that carry trust decisions, such as whether a source was reachable or whether a quote was verified, are set by code and are not part of the schema the model sees. A model cannot claim a verification it did not earn.
The fourth layer is the blast radius. Drafts become pull requests from a bot with narrow permissions, authentication to the model provider uses short-lived federated credentials rather than a stored key, workflow actions are pinned to full commit hashes, and the main branch is protected so that only a human merge publishes. If layers one to three all fail, the result is a pull request that a reviewer reads before anything goes live.
What a review of my own design found
When I walked the code against this threat model, the first finding was a gap, not a confirmation. The ingest and claim-check stages tell the model explicitly that their input is untrusted data. The editor and writer prompts do not carry the same clause. They receive pitch fields that a journalist model produced from feed text, so they sit one hop downstream of the hostile content. The structural controls still apply to them: no tools, schema validation, escaping, human merge. But a prompt that is explicit in two stages and silent in two others is an inconsistency an auditor would flag, and it is a second-order injection path: hostile text shapes a pitch, and the pitch shapes the editor.
The fix is cheap, and I treat it as a defect to close. It is also a useful illustration of why I do not rely on prompts alone. The gap existed without any observable consequence, because the controls that matter were not in the prompt.
The second finding is about evidence. The test suite contains an item that says to ignore all previous instructions, and it passes. That proves the pipeline handles the textbook case. It says very little about a patient attacker who understands the pipeline, and I have no observed attack attempts to learn from. I read the absence of incidents as weak evidence, not as proof that the controls work.
Residual risk I am carrying
Three risks remain, and I would rather state them than let a reader find them.
The first is DNS rebinding in the outbound fetch guard. The guard validates the target before connecting, and a hostile resolver could in principle answer differently between the check and the connection. The fetch only reads public pages for fact-checking and runs on an ephemeral CI runner, which limits what a successful rebind could reach, but the window exists and is documented.
The second is subtle manipulation that survives every structural control. A hostile feed item cannot make the system do something, but it can make a draft slightly more favourable to a vendor or slightly more alarming than the facts justify. Schema validation does not catch bias, and a verbatim quote check confirms that a sentence exists in the source, not that it was used fairly. This is the risk the human reviewer exists for, and it is the one I would not claim to have automated away.
The third is the reviewer. A control that depends on a person reading carefully degrades with volume and fatigue. At the current output rate that is manageable. It becomes a design question the moment the rate rises, which is the subject of a later piece in this series.
What I would ask of any agentic system
◆ Key Takeaway
Prompt injection is not a prompt problem. Assume the model will be fooled, remove its tools, validate what it emits, and keep a human on the only action that matters.
If you are assessing a similar system in your own organisation, these are the questions I would put to the team that built it:
- Map the paths. Can you list every path by which external text reaches a model, including second-order paths through other models' output?
- Count the capabilities. Which model calls have tools, credentials or write access, and why does each one need them?
- Separate claims from facts. Which fields that carry a trust decision are set by code, and which can the model influence?
- Inspect the output path. What happens to model output before it is rendered, logged or executed, and is that step deterministic?
- Constrain egress. Where does outbound fetching happen, what does it refuse, and what residual risk is accepted in writing?
- Find the single gate. Which human action publishes, and what stops anyone, including the system itself, from bypassing it?
- Ask for evidence. What do you have beyond a passing test suite, and how will you notice an attack that does not look like one?
The next piece looks at the other half of this design: why the fact-checking stage runs deterministic checks first and uses a second model only for what code cannot decide, and what it means to build verification that does not trust another LLM.