A worked, fictional rollout of an AI coding assistant. Scoped credentials, a gated execution environment, a staged CI gate, and a randomized pilot are built and run locally, and a pre-agreed gate turns the evidence into a decision.
Write down what will be measured, what counts as enough, and what happens in each outcome before the pilot starts. The rule then limits how the result can be read.
In a randomized offer, only some people take it up, and those who do are not typical. Intention-to-treat compares everyone as assigned, which keeps the randomization and estimates the effect of the offer.
Dashboards can credit an AI coding tool for gaps that predate it. This guide covers pilot design, paired speed and quality measures, review cost, adoption, power, and gates.