AI Risk, Security, and Regulation
A supplier’s email told the assistant to change the bank account, and it did. The approval screen showed twelve matched invoices, a total, and a button.
-
-
Evaluating AI Systems Before and After Release
A fictional claims assistant earns a higher judge score but misses required facts. Evaluation needs independent evidence, validated ratings, and outcome checks.
-
Shared AI Platform for Language Models
A fictional company shares model quotas across teams. Its failures show why capacity, versions, fallback, data handling, and costs need platform ownership.
-
AI Agent Tool Execution and Reliability
A fictional support agent repeats a refund after a timeout. The case shows why model-driven actions need stable operation IDs and enforced execution rules.
-
RAG Architecture and Answer Quality
A fictional HR assistant cites an outdated policy. Tracing its evidence reveals how ingestion, retrieval, permissions, and generation can fail together.
-
LLM Capabilities, Limits, and System Design
A fictional invoice pilot scores 97%, yet production exposes missing evidence, invented dates, and weak cost estimates despite schema-valid JSON.
-
Continuous Model Management and MLOps
An illustrative retention program retrains weekly and passes its checks, yet lacks evidence of its continuing business effect.
-
Model Serving Architecture
A fictional checkout failure shows why stored scores, online features, fallback capacity, and recovery compatibility need explicit contracts.
-
Model Versioning, Lineage, and Promotion
In a fictional credit-model investigation, the weights survived but training and request-time records did not. Rebuilding and replaying need different evidence.
-
Model Evaluation Reliability
A fictional claims model scores 0.97 offline and 0.71 on resolved production cases. Leakage, selection, and uncertain outcomes complicate both numbers.
-
Machine Learning Problem Framing
A fictional churn project shows why predicting who will leave is not enough to decide whom to call. Evaluate incremental benefit, costs, and constraints.