AI Engineering Glossary: LLM, RAG, Agent, and MCP Terms Explained

Plain-English definitions of the AI engineering terms used across this wiki, from tokens and embeddings to RAG, tool use, agents, MCP, and evals.

Beginner12 min readUpdated October 7, 2026

On this page· Where the terms fit

Why this matters

AI engineering has its own vocabulary, and much of it is used loosely online: "agent" means anything from a single prompt to an autonomous system, and "RAG" is used for any app that touches a database. Precise words make precise designs. These are the definitions this wiki uses. Each entry starts with a one-sentence summary, followed by the details that matter when you build with it.

Where the terms fit#

Most terms name a part of this architecture, or something that flows through it:

Hover or tap a component to see what it does.

Full-stack architecture of an AI application: a React frontend, a Node.js API layer that owns every model call, Postgres with pgvector for data and embeddings, Redis, a job queue with workers, observability, and third-party model APIs.

Show text description

The diagram shows a production AI web application as boxes inside two trust zones: your own infrastructure, and third-party model APIs. Solid arrows are requests, dashed arrows are asynchronous jobs and telemetry, and arrows with moving dots are streamed responses.

User: The person using your product in a browser or mobile app.
React UI: Renders the chat or feature UI, sends requests to your API, and renders streamed tokens as they arrive. Never holds model API keys.
API layer: The only place that talks to model providers. Validates input, checks auth and rate limits, assembles context, calls the model, and streams the answer back over Server-Sent Events.
Auth: Identifies the user and decides what data and tools they may access. Every model call and tool call inherits these permissions.
Redis: Fast key-value store for per-user rate limits, response caching, and short-lived session state.
Postgres: The system of record: users, conversations, and documents. The pgvector extension stores embeddings so the same database answers similarity searches for RAG.
Job queue: Holds slow work (document indexing, long agent runs, batch jobs) so API requests return quickly. Jobs survive restarts and can be retried.
Workers: Node.js processes that pull jobs from the queue: chunk documents, request embeddings, write vectors to Postgres, and run long tasks with retries.
Observability: Collects a trace for every request and job, with prompts, token counts, latency, and cost, so you can debug bad answers and run evals on real traffic.
Model APIs: Hosted models reached over HTTPS: an LLM for generation and tool use, and an embedding model for vector search. Stateless: every request must carry its full context.

Chat request. The UI posts the message to your API, which verifies the session, checks the rate limit in Redis, retrieves context from Postgres, calls the model, and streams tokens back to the browser over SSE.

Document indexing. An upload is acknowledged immediately and queued. A worker chunks the document, requests embeddings from the model API, and writes the vectors to pgvector.

Observability. The API and workers emit a trace for every request and job, with token counts, latency, and cost. Traces feed dashboards, alerts, and evals on real traffic.

Definitions#

Agent

A system where an LLM decides its own next steps in a loop, calling tools and reading their results until a goal is met.

An agent gives the model control over the sequence of steps: it chooses which tool to call, observes the result, and decides what to do next. This flexibility suits open-ended tasks but costs more, takes longer, and is harder to test than a fixed workflow, so agents need step limits, permissions, and evals.

Agent loop

The cycle of calling the model, executing any tools it requests, appending the results, and calling it again until it stops.

Your code runs the loop: send the conversation and tools, check stop_reason, execute requested tools, append tool_result blocks, and repeat. Production loops cap the number of iterations, time, and cost, and log every step so failures can be traced.

Chunking

Splitting documents into smaller passages so each one can be embedded and retrieved on its own.

Good chunks are self-contained: they follow headings or paragraphs, run a few hundred tokens, and carry metadata such as source URL, title, and section. Chunk size is a trade-off between precise retrieval (small chunks) and enough surrounding context (larger chunks).

Context engineering

Deciding exactly which instructions, data, tools, and history the model sees on each request.

Models only know what is in the request. Context engineering covers the system prompt, which documents to retrieve, how to summarize long histories, which tools to expose, and in what order to place it all so the request is accurate, cheap, and cache-friendly.

Context window

The maximum number of tokens a model can read and write in one request, including the system prompt, history, tools, and output.

Everything the model knows about your task must fit in the context window: instructions, tool definitions, conversation history, retrieved documents, and room for the answer. A larger window is not free: every token you send costs money and adds latency, so context engineering is about sending the right tokens, not the most.

Embedding

A list of numbers that represents the meaning of a piece of text, so that similar meanings end up close together.

An embedding model turns text into a vector with hundreds or thousands of dimensions. Measuring the distance between vectors (usually cosine similarity) finds passages that mean the same thing even when they share no keywords, which is the basis of semantic search and RAG.

Eval

A repeatable test that scores an AI feature's outputs on a dataset of real examples.

Evals are the unit tests of AI engineering. Each case pairs an input with a way to grade the output: exact checks in code, schema validation, or an LLM judge with a rubric. Running the suite before every prompt, model, or retrieval change turns guesswork into measurable progress and catches regressions.

Use the terms in your own docs#

Linking the first mention of each technical term in a README or design doc helps readers who are new to the field. This small function does it for any glossary you give it, skipping code and existing links:

lib/link-terms.ts

export interface TermLink {
  term: string; // as written in prose, such as "prompt caching"
  href: string;
}
 
/** Link the first mention of each term in Markdown prose. Code spans, code blocks, and existing links are left alone. */
export function linkFirstMentions(markdown: string, terms: TermLink[]): string {
  // Split into protected parts (fenced code, inline code, links) and prose.
  const parts = markdown.split(/(`{3}[\s\S]*?`{3}|`[^`\n]+`|\[[^\]]*\]\([^)]*\))/g);
  const linked = new Set<string>();
  const sorted = [...terms].sort((a, b) => b.term.length - a.term.length); // "prompt caching" before "prompt"
 
  return parts
    .map((part, i) => {
      if (i % 2 === 1) return part; // protected
      let out = part;
      for (const { term, href } of sorted) {
        if (linked.has(term)) continue;
        const re = new RegExp(`\\b(${term.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")})\\b`, "i");
        if (re.test(out)) {
          out = out.replace(re, `[$1](${href})`);
          linked.add(term);
        }
      }
      return out;
    })
    .join("");
}

link-terms.test.ts

import { test } from "node:test";
import assert from "node:assert/strict";
import { linkFirstMentions } from "./lib/link-terms.ts";
 
const terms = [
  { term: "RAG", href: "/reference/glossary#rag" },
  { term: "prompt caching", href: "/reference/glossary#prompt-caching" },
];
 
test("links only the first mention, case-insensitively", () => {
  const out = linkFirstMentions("We use rag here. RAG again, with Prompt caching.", terms);
  assert.equal(out, "We use [rag](/reference/glossary#rag) here. RAG again, with [Prompt caching](/reference/glossary#prompt-caching).");
});
 
test("leaves code and existing links alone", () => {
  const md = "Call `enableRAG()` and read [our RAG notes](https://example.com). Then RAG.";
  assert.equal(linkFirstMentions(md, terms), "Call `enableRAG()` and read [our RAG notes](https://example.com). Then [RAG](/reference/glossary#rag).");
});

Common mistakes#

Calling every LLM feature an agent. If code decides the steps, it is a workflow.
Calling any database lookup RAG. RAG means retrieving relevant passages to ground a model's answer.
Using "hallucination" for every wrong answer. Many wrong answers are retrieval failures or missing context; name the cause.
Confusing the context window with memory. The context window is per request; memory is something you build.

Try it

Explain it back

Pick five terms you use often and write a one-sentence definition of each without looking.
Compare with the entries above and note any differences.
Run linkFirstMentions on one of your project READMEs with links to the entries you used.

Check your understanding

Key takeaways

Precise vocabulary leads to precise designs: workflow versus agent, retrieval versus memory, context window versus state.
Hover any dotted term in the wiki for its definition, and return here for the details.
Link terms the first time you use them in your own docs.

Next steps#

Prompting cheat sheet

The terms in practice, on one page.

The complete roadmap

See where each concept is taught.

Finished this article? Track it to see your progress on the roadmap.

Where the terms fit#

Definitions#

Use the terms in your own docs#

Common mistakes#

Next steps#

Related articles