Sections in this lesson

Monitoring & AI Incident Response

Capture sanitized event codes, scoped request identifiers, policy versions and effect outcomes.

Reviewed: · Content owner: Interview Preparator

What you will learn

  • Observe the effect boundary
  • Contain before replaying
  • Turn incidents into scoped regressions

Before you start

The flow

  1. 1.Identify scope
  2. 2.Inspect evidence
  3. 3.Apply host control
  4. 4.Run negative cases
  5. 5.Compare benign behavior
  6. 6.Review remaining risks

Observe the effect boundary

Capture sanitized event codes, scoped request identifiers, policy versions and effect outcomes. A model explanation is not an audit log. Avoid retaining whole prompts or documents merely because they make debugging convenient.

The host trace records status and counters without raw input. Production audit records need access control, retention, integrity and an explicit distinction between attempted and committed effects.

Contain before replaying

Stop affected capabilities, revoke grants or credentials, invalidate unsafe memory and identify potentially committed external writes. Preserve evidence before restoring a trusted corpus or previous artifact.

stopAndRevoke increments a policy epoch and removes the local grant. A subsequent write fails approval even though the old object once authorized a call. It is a synchronous teaching control, not distributed revocation propagation.

Turn incidents into scoped regressions

Record the initial boundary failure, the corrective control and a benign counterexample. Reproduce in an authorized isolated environment; do not replay against a real third-party target.

After recovery, verify authorization, evidence freshness and rollback compatibility. A green fixture suite alone cannot establish that a production incident is contained or that all downstream effects were found.

Try it

host.mjs
javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import { authorize, payloadDigest } from './policy.mjs';
// Teaching host: no shell, network, or arbitrary-file tools are exposed.
export class TeachingHost {
  constructor({ scope, resource, grant, maxCalls = 3, maxTokens = 100, deadline = 1000, epoch = 1 }) {
    if (!Number.isSafeInteger(maxCalls) || maxCalls < 1 || !Number.isSafeInteger(maxTokens) || maxTokens < 1 || !Number.isFinite(deadline)) throw new Error('Invalid budget');
    this.scope = Object.freeze({ ...scope }); this.resource = structuredClone(resource); this.grant = grant ? structuredClone(grant) : null;
    this.maxCalls = maxCalls; this.maxTokens = maxTokens; this.deadline = deadline; this.epoch = epoch;
    this.calls = 0; this.tokens = 0; this.mutations = 0; this.receipts = new Map(); this.trace = [];
  }
  execute(call, callId, reservedTokens, now) {
    if (!Number.isSafeInteger(reservedTokens) || reservedTokens < 1 || !Number.isFinite(now) || !/^[a-z0-9-]{1,80}$/.test(callId)) throw new Error('Invalid operation');
    if (now >= this.deadline) return this.record('TIMEOUT');
    const permission = authorize(this.scope,call,this.resource,this.grant,now,this.epoch);
    if (permission !== 'ALLOW') return this.record(permission);
    const digest = payloadDigest(call);
    const cached = this.receipts.get(callId);
    if (cached) return this.record(cached.digest === digest ? 'REPLAY' : 'PAYLOAD_MISMATCH');
    if (this.calls >= this.maxCalls || reservedTokens > this.maxTokens - this.tokens) return this.record('BUDGET_EXCEEDED');
    this.calls++; this.tokens += reservedTokens;
    if (call.action === 'completePlan') this.mutations++;
    this.receipts.set(callId,{ digest });
    return this.record('EXECUTED');
  }
  record(status) { this.trace.push({ status, calls:this.calls, tokens:this.tokens, mutations:this.mutations }); return status; }
  stopAndRevoke() { this.epoch++; this.grant = null; }
}
// Cache keys include authorization scope and corpus version; cache values still
// require current authorization at read time (revocation/deletion can change).
export function cacheKey(scope, corpusVersion, question) { return JSON.stringify([scope.tenant,scope.user,corpusVersion,question]); }
export function memoryRead(record, scope, version) {
  return record.tenant === scope.tenant && record.user === scope.user && record.conversation === scope.conversation
    && record.trusted === true && record.version === version ? record.text : null;
}

Run npm run examples:security and npm run eval:security. This source is a deterministic teaching fixture; it does not prove live model safety, NLI, production auth or OS/network isolation.

Interview questions

What boundary matters in Monitoring & AI Incident Response?

The host trace records status and counters without raw input. Production audit records need access control, retention, integrity and an explicit distinction between attempted and committed effects.

What does the executable fixture demonstrate?

stopAndRevoke increments a policy epoch and removes the local grant. A subsequent write fails approval even though the old object once authorized a call. It is a synchronous teaching control, not distributed revocation propagation.

What remains a production integration responsibility?

After recovery, verify authorization, evidence freshness and rollback compatibility. A green fixture suite alone cannot establish that a production incident is contained or that all downstream effects were found.

Practice

Extend one negative scenario and its authorized positive counterexample for Monitoring & AI Incident Response.

Check your solution

  • State the trusted input and effect boundary.
  • Assert the expected status and observed effect.
  • Report coverage and limitations without claiming production readiness.

Official references

Continue learning

Category sequence

AI Security

Lesson 13 / 14

Was this lesson useful?

Change notes: 2026-10-31: security/reliability curriculum and source-bound fixture.