Production AI Engineering
Evals, observability, reliability, security, and cost engineering for AI systems in production.
5 articles · about 97 minutes of reading
- LLM Evals in TypeScript: Datasets, Graders, LLM-as-Judge, and CIBuild an eval suite for an LLM feature: real-case datasets, code graders, a calibrated LLM judge, repeated trials, baseline comparison, and a CI gate in TypeScript.Difficulty: Advanced21 min read
- LLM Observability: Tracing, Metrics, and Logs with OpenTelemetryInstrument an LLM app with OpenTelemetry in Next.js: GenAI spans with tokens, cost, cache hits, and time to first token, plus dashboards, alerts, and safe logging.Difficulty: Advanced18 min read
- Reliable LLM Apps: Timeouts, Retries, Fallbacks, and Circuit BreakersKeep LLM features working when providers are slow or down: SDK retries and timeouts, deadlines, fallback models, circuit breakers, and graceful degradation in TypeScript.Difficulty: Advanced18 min read
- Security for LLM Apps: Prompt Injection, Data Leaks, and Agent RisksThreat-model an AI feature, defend against direct and indirect prompt injection, block exfiltration and SSRF, scope tools, and protect secrets and personal data.Difficulty: Advanced21 min read
- Cost Engineering for LLM Apps: Unit Economics, Routing, and BudgetsModel and cut LLM costs in TypeScript: unit economics, a cost model for levers like caching and batching, model routing with escalation, and per-feature budgets.Difficulty: Advanced19 min read