Prompt Injection Protection

Defense-in-depth against prompt injection attacks using character-level ASCII smuggling protection and LLM-based semantic detection across all entry points.

Why This Exists

External content processed by app skills (websites, emails, code, PDFs, video transcripts) may contain malicious instructions targeting the LLM. Attack vectors include direct instructions in text, hidden Unicode characters (ASCII smuggling), metadata injection via Open Graph tags, and obfuscated instructions in code comments.

How It Works

Layer 0: ASCII Smuggling Protection (Character-Level)

text_sanitization.py strips invisible Unicode characters that encode hidden instructions:

Category	Unicode Range	Danger
Unicode Tags	U+E0000-U+E007F	Critical – encodes hidden ASCII text
Variant Selectors	U+FE00-U+FE0F, U+E0100-U+E01EF	High – hidden data encoding
Zero-Width Characters	U+200B, U+200C, U+200D, U+2060, U+FEFF	High – binary data patterns
BiDi Controls	U+200E-U+200F, U+202A-U+202E, U+2066-U+2069	Medium – text reordering
ASCII Control	0x00-0x1F, 0x7F (except tab/newline/return)	Low

Entry points protected:

1. WebSocket handler – all user messages from web app
2. REST API endpoints – all programmatic requests
3. AI preprocessor – final safety check before LLM
4. App-skill output dispatch surfaces – REST/CLI/SDK, assistant tool execution, and Workflow app-skill nodes

Process: Detect and decode hidden ASCII content (logged as security alert), remove all invisible characters, normalize Unicode to NFC form.

Layer 1: LLM-Based Prompt Injection Detection (Semantic-Level)

content_sanitization.py runs a specialized LLM to detect malicious instructions in visible text from app skill results.

Model configuration:

• Main backend: openai/gpt-oss-safeguard-20b via Groq (primary) with OpenRouter fallback. Configured in app.yml as content_sanitization_model.
• Preview server: llama-3.3-70b-versatile via Groq. Configured in config.py.

Detection system prompt and thresholds are defined in prompt_injection_detection.yml.

Scoring and actions:

Score	Action
>= 7.0 (block threshold)	Block entire content (return empty string)
5.0-6.9 (review threshold)	Replace detected injection strings with [PROMPT INJECTION DETECTED & REMOVED]
< 5.0 with detected strings	Replace detected strings with placeholder
< 5.0, no strings	Pass through

Text chunking: Long text outputs are split into 50,000-token chunks, processed separately, then combined. content_sanitization.py implements word-boundary-aware splitting via _split_text_into_chunks().

Execution order: ASCII smuggling runs FIRST on external content, then LLM detection runs as the LAST step before app skill endpoints return data to main processing.

App-Skill Output Dispatch Boundary

app_skill_output_safety.py is the central protection layer for app-skill outputs across assistant, Workflow, REST API, CLI, npm SDK, and pip SDK calls.

Mandatory behavior:

• ASCII-smuggling removal always runs on app-skill outputs and cannot be disabled on any surface.
• External-data skill outputs run semantic GPT-OSS prompt-injection scanning by default.
• Semantic scanner failures fail closed while protection is enabled; unscanned external text is not returned as a fallback.
• Binary/media fields, encrypted blobs, hashes, keys, and base64 payloads are excluded from semantic scanning by the existing external-result sanitizer rules.

Opt-out contract:

Direct programmatic callers may disable semantic scanning only for REST/CLI/SDK app-skill calls by sending this request-body metadata:

{
  "security": {
    "prompt_injection_protection": "disabled"
  }
}

The CLI maps --disable-prompt-injection-protection to this field. The npm SDK maps { promptInjectionProtection: false }; the pip SDK maps prompt_injection_protection=False.

Assistant-triggered app-skill calls and Workflow app-skill nodes cannot disable semantic scanning. Their dispatch adapters strip or ignore security.prompt_injection_protection before skill validation and never treat it as an authorized programmatic opt-out.

Preview Server Sanitization

The preview server (backend/preview/) applies both layers to metadata fetched from external websites and YouTube videos.

Source	Fields Sanitized
Website	title, description, site_name
YouTube Video	title, description, channel_name
YouTube Channel	title, description

Preview metadata text is sanitized before the LLM sees it. Raw URLs, thumbnails, and favicons are not rewritten by the preview server; assistant-visible links are handled by the response URL safety layer below.

Graceful degradation: If Groq API is unavailable, ASCII smuggling protection still runs. LLM detection is skipped with a warning log.

Layer 2: Assistant URL Source and Safety Checks

Assistant responses can turn prompt-injected source text into malicious links, for example by encoding chat secrets in a URL path or query string. OpenMates therefore treats assistant-visible URLs as generated content that must be proven safe before the final response is persisted or streamed as the corrected final text.

url_normalizer.py and stream_consumer.py enforce two gates:

1. Exact source allowlist – every URL in the assistant response must already exist byte-for-byte in trusted source material available before generation.
2. Batched safeguard classification – URLs that pass the source allowlist are sent in one batch to openai/gpt-oss-safeguard-20b using function/tool calling. The model reports only malicious URLs via report_malicious_urls.

Source material for the allowlist:

• User-typed message content and relevant chat history
• Tool results and app skill outputs passed back to main processing
• Embed preview/result payloads such as web pages, web search results, code results, documents, emails, and transcripts

Removal rules:

• If an assistant URL is not present exactly in the source allowlist, remove it immediately without an LLM call.
• If the safeguard reports a source-backed URL as malicious, remove it.
• If the safeguard call fails, returns malformed tool arguments, references unknown URLs, or rewrites URLs, fail closed for the affected batch and remove those URLs.

Why this is not URL parameter stripping: parameters, fragments, and paths are preserved when they are legitimate source URLs. The safety question is whether the exact URL came from source content and whether the full URL contains secrets, personal data, encoded payloads, prompt-injection instructions, phishing/malware indicators, or credential-exfiltration patterns.

Additional Defenses

• Manual confirmation for sensitive skills: Certain app skills require user confirmation before execution.
• Assistant URL source allowlist: Generated responses may only keep URLs that came from user/tool/embed source content exactly, then pass batched safeguard classification.
• CLI terminal command blocking: The CLI prevents LLM from executing arbitrary shell commands, providing safe file-reading instead.

Programmatic API Override

For direct REST API, npm package, pip package, and CLI app-skill access, users can optionally disable semantic prompt-injection scanning at their own risk through security.prompt_injection_protection: disabled. Scanning is ON by default. The web interface, assistant tool calls, and Workflows always enforce protection. ASCII-smuggling removal is never disabled.

Edge Cases

• Safeguard API outage: App-skill external-data outputs fail closed while semantic protection is enabled. ASCII smuggling protection has no external dependency and still runs before failure.
• False positives: Content discussing AI systems or prompt engineering may trigger moderate scores (5.0-6.9). The review threshold allows these through with targeted string replacement rather than full blocking.
• Cached preview metadata: Sanitization happens at fetch time. Cached metadata is already sanitized.

Related Docs

• Hallucination Mitigation – related but distinct defense layer
• PII Protection – client-side PII detection