Reproducible Test Sets for AI Evaluation
A 27-versus-25 score on 30 pasted questions proves little. Build a test set from a failure taxonomy, sample it by stratum, score with checks and validated judges, and compare releases pairwise.
-
-
Designing Context and Prompt Interfaces
A prompt is an interface that merges trusted instructions with untrusted text. Write testable instructions, pack evidence within a budget, version releases, and test behavior.
-
Model API Integration and Failure Handling
A model API call can fail in transport, be rejected, return unusable text, or return valid JSON that is wrong. Define what each outcome does to your application.
-
Framing an AI Application
Before integrating a model, decide who consumes its output, where the task ends, what simple baseline it must beat, and which measured criteria, priced by error cost, allow it to ship.