Sections in this lesson

Secure Output Handling

A model can return valid JSON containing an unauthorized resource, a dangerous URL or invented SQL.

Reviewed: · Content owner: Interview Preparator

What you will learn

  • Validate syntax and domain separately
  • Render text as text
  • Keep the negative contract visible

Before you start

The flow

  1. 1.Identify scope
  2. 2.Inspect evidence
  3. 3.Apply host control
  4. 4.Run negative cases
  5. 5.Compare benign behavior
  6. 6.Review remaining risks

Validate syntax and domain separately

A model can return valid JSON containing an unauthorized resource, a dangerous URL or invented SQL. Parse a narrow schema, reject extra fields and then validate business constraints and authorization. Never treat syntactic validity as permission.

Use parameterized database operations and fixed tool implementations. Passing model text into a shell, SQL concatenation or a generic executor changes data into code and expands the attack surface.

Render text as text

escapedText encodes HTML-sensitive characters for a plain-text HTML context. In React, ordinary string interpolation already escapes text. Do not use raw HTML insertion for generated content or treat this helper as a sanitizer for every output context.

URLs, attributes, CSS and JavaScript have different rules. A safe text encoder does not validate a link target, and a valid link does not prove that a remote image or redirect is safe.

Keep the negative contract visible

The fixtures include script, image, iframe, SVG and javascript-link strings. Tests verify encoded output rather than executing them. Add legitimate code samples so defensive rendering does not erase useful generic type syntax.

Existing question content is repository-authored and follows its own formatting helper. Do not reuse that trusted-content path for model output. Apply the correct sink-specific control at every boundary.

Try it

policy.mjs
javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import { createHash, timingSafeEqual } from 'node:crypto';
export function payloadDigest(call) {
  // Canonical ordered fields; extra or ambiguous fields cannot bypass payload binding.
  if (!call || Object.keys(call).sort().join(',') !== 'action,resource' || !['readPlan','completePlan'].includes(call.action)
    || !/^PLAN-[0-9]+$/.test(call.resource)) throw new Error('Invalid call');
  return createHash('sha256').update(JSON.stringify([call.action,call.resource])).digest('hex');
}
export function authorize(scope, call, resource, grant, now, epoch = 1) {
  const digest = payloadDigest(call);
  if (!Number.isFinite(now) || !Number.isSafeInteger(epoch)) return 'DENY';
  if (!scope || !resource || resource.id !== call.resource || resource.tenant !== scope.tenant || !resource.readers.includes(scope.user)) return 'DENY';
  if (call.action === 'readPlan') return 'ALLOW';
  // Grant comes from a trusted server-side approval store, never from model output.
  if (!grant || !Number.isFinite(grant.expiresAt) || grant.revoked || grant.epoch !== epoch || grant.expiresAt <= now || grant.tenant !== scope.tenant
    || grant.user !== scope.user || grant.runId !== scope.runId || grant.digest !== digest) return 'APPROVAL_REQUIRED';
  return 'ALLOW';
}
export function escapedText(value) {
  if (typeof value !== 'string' || value.length > 4000) throw new Error('Invalid output text');
  return value.replaceAll('&','&amp;').replaceAll('<','&lt;').replaceAll('>','&gt;').replaceAll('"','&quot;').replaceAll("'",'&#39;');
}
export function allowedUrl(value, hosts) {
  try { const url = new URL(value); return url.protocol === 'https:' && !url.username && !url.password && url.port === '' && hosts.includes(url.hostname); }
  catch { return false; }
}
export function trustedToken(claims, policy, now) {
  // Claims must already be cryptographically verified by an auth adapter.
  return claims?.issuer === policy.issuer && claims?.audience === policy.audience && claims?.expiresAt > now
    && claims?.tenant === policy.tenant && Array.isArray(claims?.scopes) && claims.scopes.includes(policy.scope);
}
export function approvedArtifact(bytes, expectedSha256) {
  const actual = createHash('sha256').update(bytes).digest();
  if (!/^[a-f0-9]{64}$/.test(expectedSha256 ?? '')) return false;
  return timingSafeEqual(actual,Buffer.from(expectedSha256,'hex'));
}

Run npm run examples:security and npm run eval:security. This source is a deterministic teaching fixture; it does not prove live model safety, NLI, production auth or OS/network isolation.

Interview questions

Does valid JSON authorize execution?

Use parameterized database operations and fixed tool implementations. Passing model text into a shell, SQL concatenation or a generic executor changes data into code and expands the attack surface.

Is HTML escaping safe in every output context?

URLs, attributes, CSS and JavaScript have different rules. A safe text encoder does not validate a link target, and a valid link does not prove that a remote image or redirect is safe.

May model output use the trusted repository HTML path?

Existing question content is repository-authored and follows its own formatting helper. Do not reuse that trusted-content path for model output. Apply the correct sink-specific control at every boundary.

Practice

Extend one negative scenario and its authorized positive counterexample for Secure Output Handling.

Check your solution

  • State the trusted input and effect boundary.
  • Assert the expected status and observed effect.
  • Report coverage and limitations without claiming production readiness.

Official references

Continue learning

Category sequence

AI Security

Lesson 4 / 14

Was this lesson useful?

Change notes: 2026-10-31: security/reliability curriculum and source-bound fixture.