Release Gate

A release gate is a decision point where a candidate must meet specified requirements before progressing toward release. Requirements can include task quality, compatibility, access controls, operational readiness, and an identified owner. A gate can combine automated checks with human review.

A better total can still fail

Consider a fictional assistant evaluation with 80 routine and 20 critical cases. The accepted version passes 70 routine and all 20 critical cases: 90 in total. A candidate passes 78 routine and 17 critical cases: 95 in total.

If the agreed rule requires every critical case to pass, the candidate fails the gate despite its higher total. Preserve the evidence and investigate the three failures. Do not change the rule after seeing the scores merely to admit the preferred candidate.

Approval must identify its scope

Record which candidate, evidence, and environment the decision covers. Approval for a limited trial is not automatically approval for unrestricted production use. A changed prompt or dependency can invalidate the evidence for a previous candidate.

Passing a finite evaluation does not prove zero future failures. The gate establishes eligibility under stated rules; rollout monitoring and recovery arrangements still matter.

Reference: MLflow: LLM and Agent Evaluation.