Sections in this lesson

Memory & Context Poisoning

An untrusted instruction stored as memory can influence later runs after the original document is gone.

Reviewed: · Content owner: Interview Preparator

What you will learn

  • Persistence changes the threat
  • Reauthorize cached context
  • Recover without reinfection

Before you start

The flow

  1. 1.Identify scope
  2. 2.Inspect evidence
  3. 3.Apply host control
  4. 4.Run negative cases
  5. 5.Compare benign behavior
  6. 6.Review remaining risks

Persistence changes the threat

An untrusted instruction stored as memory can influence later runs after the original document is gone. Record origin, approval, version and ownership separately from the text. Conversation history and durable execution state are different assets.

The fixture memoryRead checks tenant, user, conversation, trusted state and version. The trusted flag is host-authored; accepting the same flag from model output would erase the boundary.

Reauthorize cached context

Scope cache keys by tenant, user and corpus version, but do not stop there. Deletion, permission revocation and incident invalidation can happen while a key still matches; recheck the current source before reuse.

Test cross-user conversations, stale versions, untrusted records and revoked evidence. A checkpoint must not restore a capability that the current policy has removed.

Recover without reinfection

Quarantine suspect memory, revoke affected grants and reconstruct context from authorized sources. Retain sanitized incident evidence without copying secret-bearing prompts into ordinary logs.

This example is an in-memory policy fixture, not a production memory database or automated poison detector. Durable cleanup and migration require integration and their own recovery tests.

Try it

host.mjs
javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import { authorize, payloadDigest } from './policy.mjs';
// Teaching host: no shell, network, or arbitrary-file tools are exposed.
export class TeachingHost {
  constructor({ scope, resource, grant, maxCalls = 3, maxTokens = 100, deadline = 1000, epoch = 1 }) {
    if (!Number.isSafeInteger(maxCalls) || maxCalls < 1 || !Number.isSafeInteger(maxTokens) || maxTokens < 1 || !Number.isFinite(deadline)) throw new Error('Invalid budget');
    this.scope = Object.freeze({ ...scope }); this.resource = structuredClone(resource); this.grant = grant ? structuredClone(grant) : null;
    this.maxCalls = maxCalls; this.maxTokens = maxTokens; this.deadline = deadline; this.epoch = epoch;
    this.calls = 0; this.tokens = 0; this.mutations = 0; this.receipts = new Map(); this.trace = [];
  }
  execute(call, callId, reservedTokens, now) {
    if (!Number.isSafeInteger(reservedTokens) || reservedTokens < 1 || !Number.isFinite(now) || !/^[a-z0-9-]{1,80}$/.test(callId)) throw new Error('Invalid operation');
    if (now >= this.deadline) return this.record('TIMEOUT');
    const permission = authorize(this.scope,call,this.resource,this.grant,now,this.epoch);
    if (permission !== 'ALLOW') return this.record(permission);
    const digest = payloadDigest(call);
    const cached = this.receipts.get(callId);
    if (cached) return this.record(cached.digest === digest ? 'REPLAY' : 'PAYLOAD_MISMATCH');
    if (this.calls >= this.maxCalls || reservedTokens > this.maxTokens - this.tokens) return this.record('BUDGET_EXCEEDED');
    this.calls++; this.tokens += reservedTokens;
    if (call.action === 'completePlan') this.mutations++;
    this.receipts.set(callId,{ digest });
    return this.record('EXECUTED');
  }
  record(status) { this.trace.push({ status, calls:this.calls, tokens:this.tokens, mutations:this.mutations }); return status; }
  stopAndRevoke() { this.epoch++; this.grant = null; }
}
// Cache keys include authorization scope and corpus version; cache values still
// require current authorization at read time (revocation/deletion can change).
export function cacheKey(scope, corpusVersion, question) { return JSON.stringify([scope.tenant,scope.user,corpusVersion,question]); }
export function memoryRead(record, scope, version) {
  return record.tenant === scope.tenant && record.user === scope.user && record.conversation === scope.conversation
    && record.trusted === true && record.version === version ? record.text : null;
}

Run npm run examples:security and npm run eval:security. This source is a deterministic teaching fixture; it does not prove live model safety, NLI, production auth or OS/network isolation.

Interview questions

What boundary matters in Memory & Context Poisoning?

The fixture memoryRead checks tenant, user, conversation, trusted state and version. The trusted flag is host-authored; accepting the same flag from model output would erase the boundary.

What does the executable fixture demonstrate?

Test cross-user conversations, stale versions, untrusted records and revoked evidence. A checkpoint must not restore a capability that the current policy has removed.

What remains a production integration responsibility?

This example is an in-memory policy fixture, not a production memory database or automated poison detector. Durable cleanup and migration require integration and their own recovery tests.

Practice

Extend one negative scenario and its authorized positive counterexample for Memory & Context Poisoning.

Check your solution

  • State the trusted input and effect boundary.
  • Assert the expected status and observed effect.
  • Report coverage and limitations without claiming production readiness.

Official references

Continue learning

Category sequence

AI Security

Lesson 10 / 14

Was this lesson useful?

Change notes: 2026-10-31: security/reliability curriculum and source-bound fixture.