Prompt Injection Protection

Defense-in-depth against prompt injection attacks using character-level ASCII smuggling protection and LLM-based semantic detection across all entry points.

Why This Exists

External content processed by app skills (websites, emails, code, PDFs, video transcripts) may contain malicious instructions targeting the LLM. Attack vectors include direct instructions in text, hidden Unicode characters (ASCII smuggling), metadata injection via Open Graph tags, and obfuscated instructions in code comments.

How It Works

Layer 0: ASCII Smuggling Protection (Character-Level)

text_sanitization.py strips invisible Unicode characters that encode hidden instructions:

Category	Unicode Range	Danger
Unicode Tags	U+E0000-U+E007F	Critical – encodes hidden ASCII text
Variant Selectors	U+FE00-U+FE0F, U+E0100-U+E01EF	High – hidden data encoding
Zero-Width Characters	U+200B, U+200C, U+200D, U+2060, U+FEFF	High – binary data patterns
BiDi Controls	U+200E-U+200F, U+202A-U+202E, U+2066-U+2069	Medium – text reordering
ASCII Control	0x00-0x1F, 0x7F (except tab/newline/return)	Low

Entry points protected:

1. WebSocket handler – all user messages from web app
2. REST API endpoints – all programmatic requests
3. AI preprocessor – final safety check before LLM
4. App-skill output dispatch surfaces – REST/CLI/SDK, assistant tool execution, and Workflow app-skill nodes

Process: Detect and decode hidden ASCII content (logged as security alert), remove all invisible characters, normalize Unicode to NFC form.

Layer 1: Jev-first Prompt Injection Detection (Semantic-Level)

content_sanitization.py first runs a Jev yes/no decision over visible text from app-skill results. A low-risk result passes content unchanged; a high-confidence attack blocks the chunk. The middle confidence band and every Jev outage or invalid response use GPT-OSS Safeguard, because exact-span redaction requires generated evidence that Jev cannot return.

Model configuration:

• Main backend primary: typesafe/jev-1.13 via OpenRouter, configured as prompt_injection_decision_model.
• Main backend ambiguity/outage fallback: openai/gpt-oss-safeguard-20b via Groq with OpenRouter server fallback, configured as content_sanitization_model.
• Preview server: llama-3.3-70b-versatile via Groq. Configured in config.py.

Detection system prompt and thresholds are defined in prompt_injection_detection.yml.

Jev confidence bands:

Jev probability	Action
<= 0.20	Pass ASCII-cleaned content without a GPT call
>= 0.90	Block the chunk without a GPT call
0.20-0.90 or Jev failure	Run GPT-OSS Safeguard for exact-span analysis

GPT fallback scoring and actions:

Score	Action
>= 7.0 (block threshold)	Block entire content (return empty string)
5.0-6.9 (review threshold)	Replace detected injection strings with [PROMPT INJECTION DETECTED & REMOVED]
< 5.0 with detected strings	Replace detected strings with placeholder
< 5.0, no strings	Pass through

Text chunking: Long text outputs are split into 50,000-token chunks, processed separately, then combined. content_sanitization.py implements word-boundary-aware splitting via _split_text_into_chunks().

Execution order: ASCII smuggling runs first, Jev evaluates the cleaned text, and GPT-OSS runs only for ambiguity or recovery. Semantic detection remains the last trust-boundary step before external data reaches main processing.

App-Skill Output Dispatch Boundary

app_skill_output_safety.py is the central protection layer for app-skill outputs across assistant, Workflow, REST API, CLI, npm SDK, and pip SDK calls.

Mandatory behavior:

• ASCII-smuggling removal always runs on app-skill outputs and cannot be disabled on any surface.
• External-data skill outputs run semantic GPT-OSS prompt-injection scanning by default.
• Semantic scanning uses one model call per nonempty external skill result after ASCII cleanup. The shared policy covers all existing selected text fields, including multiline website, transcript, mail, document and search content.
• The scanner distinguishes assistant-directed instructions from human-facing documentation, narration and quoted demonstrations. Exact, uniquely located evidence is required for injection decisions; only verified spans are replaced. Uncertain text is preserved.
• Scanner timeout, malformed output, unavailability, or input exceeding the 50,000-character serialized single-call bound returns ASCII-cleaned content and logs status=unscanned with a stable reason. There are no retries or user questions, and failed scans are never reported as safe. Units are at most 4,000 characters with 256 characters of neighboring context.
• Skill-local helpers defer semantic work only inside an active server-owned dispatch scope. Background helpers without that scope still scan before persistence. Request authorization, tool permissions and explicit REST/CLI opt-out eligibility are unchanged.
• Binary/media fields, encrypted blobs, hashes, keys, and base64 payloads are excluded from semantic scanning by the existing external-result sanitizer rules.

Opt-out contract:

Direct programmatic callers may disable semantic scanning only for REST/CLI/SDK app-skill calls by sending this request-body metadata:

{
  "security": {
    "prompt_injection_protection": "disabled"
  }
}

The CLI maps --disable-prompt-injection-protection to this field. The npm SDK maps { promptInjectionProtection: false }; the pip SDK maps prompt_injection_protection=False.

Assistant-triggered app-skill calls and Workflow app-skill nodes cannot disable semantic scanning. Their dispatch adapters strip or ignore security.prompt_injection_protection before skill validation and never treat it as an authorized programmatic opt-out.

Preview Server Sanitization

The preview server (backend/preview/) applies both layers to metadata fetched from external websites and YouTube videos.

Source	Fields Sanitized
Website	title, description, site_name
YouTube Video	title, description, channel_name
YouTube Channel	title, description

Preview metadata text is sanitized before the LLM sees it. Raw URLs, thumbnails, and favicons are not rewritten by the preview server; assistant-visible links are handled by the response URL safety layer below.

Graceful degradation: If Groq API is unavailable, ASCII smuggling protection still runs. LLM detection is skipped with a warning log.

Layer 2: Assistant URL Source and Safety Checks

Assistant responses can turn prompt-injected source text into malicious links, for example by encoding chat secrets in a URL path or query string. OpenMates therefore treats assistant-visible URLs as generated content that must be proven safe before the final response is persisted or streamed as the corrected final text.

url_normalizer.py and stream_consumer.py enforce two gates:

1. Exact source allowlist – every URL in the assistant response must already exist byte-for-byte in trusted source material available before generation.
2. Batched safeguard classification – URLs that pass the source allowlist are sent in one batch to openai/gpt-oss-safeguard-20b using function/tool calling. The model reports only malicious URLs via report_malicious_urls.

Source material for the allowlist:

• User-typed message content and relevant chat history
• Tool results and app skill outputs passed back to main processing
• Embed preview/result payloads such as web pages, web search results, code results, documents, emails, and transcripts

Removal rules:

• If an assistant URL is not present exactly in the source allowlist, remove it immediately without an LLM call.
• If the safeguard reports a source-backed URL as malicious, remove it.
• If the safeguard call fails, returns malformed tool arguments, references unknown URLs, or rewrites URLs, fail closed for the affected batch and remove those URLs.

Why this is not URL parameter stripping: parameters, fragments, and paths are preserved when they are legitimate source URLs. The safety question is whether the exact URL came from source content and whether the full URL contains secrets, personal data, encoded payloads, prompt-injection instructions, phishing/malware indicators, or credential-exfiltration patterns.

Additional Defenses

• Manual confirmation for sensitive skills: Certain app skills require user confirmation before execution.
• Assistant URL source allowlist: Generated responses may only keep URLs that came from user/tool/embed source content exactly, then pass batched safeguard classification.
• CLI terminal command blocking: The CLI prevents LLM from executing arbitrary shell commands, providing safe file-reading instead.

Programmatic API Override

For direct REST API, npm package, pip package, and CLI app-skill access, users can optionally disable semantic prompt-injection scanning at their own risk through security.prompt_injection_protection: disabled. Scanning is ON by default. The web interface, assistant tool calls, and Workflows always enforce protection. ASCII-smuggling removal is never disabled.

Edge Cases

• Jev outage: GPT-OSS Safeguard runs independently. If both semantic providers fail, the existing entry-point-specific fail-open/fail-closed behavior remains unchanged; downstream authorization must never depend on a successful semantic verdict.
• False positives: Content discussing AI systems or prompt engineering may trigger moderate scores (5.0-6.9). The review threshold allows these through with targeted string replacement rather than full blocking.
• Cached preview metadata: Sanitization happens at fetch time. Cached metadata is already sanitized.

Related Docs

• Hallucination Mitigation – related but distinct defense layer
• PII Protection – client-side PII detection