Sections in this lesson

LLM Application Fundamentals

Design an application around probabilistic output, finite context, latency and measurable quality.

Reviewed: · Content owner: Interview Preparator

What you will learn

  • Separate model capability from application reliability.
  • Reserve context for output and measure token usage.
  • Compare models on your own task dataset.

Before you start

The flow

  1. 1.Task contract
  2. 2.Model request
  3. 3.Validate
  4. 4.Measure

The application owns the contract

A language model generates likely continuations. A fluent response can still be unsupported or wrong. Define permitted inputs, expected output, failure handling and user-visible uncertainty before choosing a provider.

Use rules or a database query when the task is exact and deterministic. Introduce a model when language understanding or generation adds value; keep authorization and financial calculations in application code.

Budget context and cost

The input includes instructions, history, retrieved material and tool results. Reserve space for output; do not estimate production token counts from character counts alone. Tokenization depends on the model and language.

Log usage, time to first token, total latency and validation failures without retaining sensitive text by default. Price the measured input and output separately; retries multiply both latency and spend.

Choose by evidence

Evaluate candidates on representative, adversarial and unanswerable inputs. A larger model is not automatically the best choice once latency, deployment constraints and cost are included. Set a fallback and a maximum retry count.

Try it

llm-application-fundamentals.mjs
javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import assert from 'node:assert/strict';
// Fixture usage counts, not a tokenizer or a provider price quote.
function fitsContext({ inputTokens, reservedOutput, contextLimit }) {
  return inputTokens + reservedOutput <= contextLimit;
}
function estimateCost(inputTokens, outputTokens, pricePerMillion) {
  return (inputTokens * pricePerMillion.input +
    outputTokens * pricePerMillion.output) / 1_000_000;
}
assert.equal(fitsContext({ inputTokens: 700, reservedOutput: 400, contextLimit: 1000 }), false);
assert.equal(fitsContext({ inputTokens: 600, reservedOutput: 400, contextLimit: 1000 }), true);
const cost = estimateCost(1000, 200, { input: 2, output: 8 });
assert.ok(Math.abs(cost - 0.0036) < 1e-10);
console.log({ fits: true, estimatedCost: cost });

Run with Node.js 22: node examples/generative-ai/llm-application-fundamentals.mjs. This is a deterministic teaching example, not a live model call.

Interview questions

Does temperature zero guarantee identical results?

No. It can reduce sampling variation, but service implementation and model changes can still affect output. Test acceptable behavior, not byte-for-byte prose.

What belongs outside the model?

Authentication, authorization, exact calculations and side-effect execution must be enforced by application code.

How do you select a model?

Compare task success and failure modes on a fixed dataset, then include latency, cost and data handling requirements.

Practice

Add a request gate that rejects negative token counts and reserves an output budget before a call.

Check your solution

  • Boundary fits exactly; overflow is rejected.
  • Invalid counts fail before any request.
  • Price inputs are configuration, not hardcoded provider claims.

Official references

Continue learning

Category sequence

Generative AI

Lesson 1 / 8

Was this lesson useful?

Change notes: Initial bilingual lesson and executable example.