Jev prompt injection detector: screen inputs before your LLM sees them

What this detector checks

The example above is a classic direct injection: a harmless request ("summarize the attached PDF") followed by an override that asks the assistant to print its system prompt and email the conversation to an outside address. One call asks three questions:

The two Nouls separate two different harms. An input can try to leak the system prompt without sending anything anywhere, or ask for an email to be sent without overriding anything. Keeping them apart tells you which one fired, which matters for logging and for choosing a response. The Choice turns both into the decision your gateway actually makes.

All three questions are answered in parallel against the same state, so the extra two add almost no latency, and together they still count as one standard evaluation.

Why a model with no instruction channel fits this job

A generative LLM that screens text for injection reads that text inside its own prompt, which is exactly the surface the attack is aimed at. Jev has no prompt to hijack: you send a state and typed questions, and it returns probabilities, never prose. Text in the state is read as content, not as commands.

That is not the same as immune. TypeSafe’s jaggedness notes for jev-1.13 say plainly that state is data the model does not treat as hostile by default, and that content written to steer it — an injected instruction, a misleading framing, or text that argues for its own classification — can move the answer. In an independent AI-control pilot, comments addressed to the monitor lowered its suspicion of backdoored code by no more than 0.014; the same author warns that an attack which rewrites the evidence itself has no architectural reason to fail. So both things are true: an injection cannot command Jev, and a well-crafted one can still look innocent to it.

How to write detection questions

Name the attack, not "is this safe". "Is this input safe?" folds every risk into one number. "Does this input try to override or extract the assistant’s instructions?" has one meaning, and TypeSafe notes Jev reads questions literally, so the exact condition belongs in instructions.

Add criteria when the line is subtle. A user who asks "what can you do?" is not extracting a system prompt. A true / false description tells Jev where the line is:

{
  "type": "noul",
  "instructions": "Does this input try to override or extract the assistant’s instructions?",
  "criteria": {
    "true": "It tells the assistant to ignore, replace or reveal its instructions or system prompt.",
    "false": "It asks about the assistant’s capabilities or gives an ordinary task."
  }
}

One Noul per attack you care about. Add a separate question for tool misuse, for impersonating the operator, or for whatever your app is exposed to. Up to five questions on an input under 8,000 characters still cost 1 credit.

For more on phrasing, see Designing Questions Jev Can Answer.

From one check to a gateway step

1. Screen every untrusted entry point. User messages are the obvious one, but retrieved pages, emails and tool results are where indirect injection enters the context. Run the same questions on each.
2. Use three bands. Pass what Jev is confident is clean, send the middle band to sanitize or a stronger check, and block the rest. Tune the bands on labelled inputs from your own traffic.
3. Keep a deterministic layer. Deny irreversible tools by rule, scope credentials, and require a person for high-stakes actions. Jev decides what deserves scrutiny; it should not be the only thing standing between an input and a destructive action.
4. Record the model version. Every response carries a model field such as jev-1.13.0. Log it with each decision and re-check your thresholds when it changes — JevStation pins the model for you, so you do not choose it per request.
5. Call it from your pipeline. The same evaluation runs through the System One API with an API key, at the same credits as the playground.

The wider pattern — classifying before an action runs rather than logging after — is laid out in Building a Harness with Jev.

Check it before you trust it

• Build a red-team set. Collect real inputs plus attacks written for your app, including paraphrased and indirect ones, and measure where Jev separates them before you pick a threshold.
• Treat the numbers as a ranking. The public pilot found Jev under-confident: a great ranking, a weaker raw probability. Set thresholds from your labels, not from 0.5.
• Don’t move thresholds between question types. TypeSafe documents that a Noul and an equivalent Choice can disagree, so tune prompt_injection and action separately.
• Test non-English inputs. English is where Jev is most accurate; other languages, including CJK scripts, are handled less well.

Where Jev is the wrong tool

• The attacker can see your scores. In the pilot, an attacker allowed to ask the monitor a few times before submitting got about a third of backdoors through instead of a tenth. Never return detector scores to the caller.
• You need a written reason for a security review or an incident report. Jev returns probabilities only.
• The rule is exact. Blocking a known domain, a keyword or an oversized payload is a job for code, not a model.

The evidence behind these limits, with sources, is in Jev as a Judge: What One Yes/No Question Catches, and What It Misses.