LLM-as-a-Judge
A model can grade thousands of answers overnight. It can also favor longer answers or the one shown first. Check the judge before trusting its scores.
-
-
Evaluating AI Systems Before and After Release
A fictional claims assistant earns a higher judge score but misses required facts. Evaluation needs independent evidence, validated ratings, and outcome checks.
-
Reproducible Test Sets for AI Evaluation
A 27-versus-25 score on 30 pasted questions proves little. Build a test set from a failure taxonomy, sample it by stratum, score with checks and validated judges, and compare releases pairwise.