Design an Agent Execution Platform

The flow

1. 1.Authenticate job
2. 2.Admit and enqueue
3. 3.Claim with lease
4. 4.Run bounded steps
5. 5.Commit tool receipt
6. 6.Checkpoint or reconcile

Architecture and boundaries

Design assumptions

These values are example sizing inputs, not measurements or capacity guarantees.

Define the control and execution planes

The control plane authenticates a caller, validates the task, assigns a trusted scope and enforces quotas. It stores the job and outbox event atomically, then publishes for workers. The execution plane claims a bounded job, loads its checkpoint and invokes only permitted tools.

Use at-least-once delivery as a normal design condition. A visibility timeout is not proof that an old worker stopped. Add lease/fencing tokens where concurrent workers could otherwise commit stale work. The Java reference demonstrates CAS checkpoints and target receipts, not a complete queue/lease orchestrator.

Estimate active work before choosing worker count

Assume five jobs/second and twelve seconds mean residence time. In steady state, the mean in-flight workload is 60 jobs. With four active slots per worker and 2× planning headroom, the example suggests 30 workers; headroom is a chosen assumption, not an SLO proof.

At three tool calls per job, average downstream demand is 15 calls/second. Retries and fan-out change that rate. Bound queue age, per-tenant concurrency, total tool attempts and model reservations; unbounded waiting can consume timeouts before execution even begins.

Protect the write/checkpoint gap

A worker can crash after a tool commits and before the next checkpoint. Retrying against a durable payload-bound target receipt returns the previous result. The local PostgreSQL tests exercise that gap and concurrent delivery; arbitrary external services still need their own idempotency/outbox contract.

Store approvals as trusted, expiring action grants. Persist a waiting state without occupying a worker slot, then revalidate ownership, policy and deadline when the grant arrives. Cancellation changes the run state but does not automatically roll back committed work.

Expose safe operational evidence

Trace run ID, policy/model version, tool attempts, reservation/actual usage and terminal reason. Redact sensitive arguments and restrict who can inspect state. Alert on forbidden actions, repeated timeouts, stale checkpoints and reconciliation backlog separately from ordinary task failure.

For untrusted execution, isolate tools in processes or sandboxes with narrow credentials and network policy. Java Future interruption in the teaching runner is cooperative and is not a hostile-code boundary.

Try it

// Sizing assumptions, not measurements or provider capacity guarantees. Decimal GB.
function positive(value) {
  if (!Number.isFinite(value) || value <= 0) throw new Error('Positive finite input required');
  return value;
}
export function ragCapacity({ documents, chunksPerDocument, dimensions, qps, inputTokens, outputTokens }) {
  [documents, chunksPerDocument, dimensions, qps, inputTokens, outputTokens].forEach(positive);
  const chunks = documents * chunksPerDocument;
  const rawVectorGB = chunks * dimensions * 4 / 1_000_000_000;
  return { chunks, rawVectorGB, inputTokensPerSecond: qps * inputTokens, outputTokensPerSecond: qps * outputTokens };
}
export function agentCapacity({ jobsPerSecond, meanSeconds, slotsPerWorker, toolCallsPerJob, headroom }) {
  [jobsPerSecond, meanSeconds, slotsPerWorker, toolCallsPerJob, headroom].forEach(positive);
  if (headroom < 1 || !Number.isInteger(slotsPerWorker)) throw new Error('Invalid worker configuration');
  const meanInFlight = jobsPerSecond * meanSeconds;
  return { meanInFlight, workers: Math.ceil(meanInFlight * headroom / slotsPerWorker), toolCallsPerSecond: jobsPerSecond * toolCallsPerJob };
}

examples/agentic-ai/capacity-planning.mjs · npm run examples:agents. Java examples target 21. Models and usage are fixtures; no live inference or billing guarantee is implied.

Interview questions

Official references

• PostgreSQL — transactions
• PostgreSQL — row locking

Continue learning

• Design a Multi-Tenant RAG Platform →
• Agent Evaluation & Reliability →