Reading a Result Honestly: The Statistical Judgment Behind Every Number
The question is not whether a result is significant. It is what the sample represents, how wide the uncertainty is, and whether the difference matters.
-
-
Practical Significance
A big enough sample makes a 0.1% difference significant. Whether 0.1% is worth the engineering is a different question, decided in advance.
-
What Kind of Question Is It? Matching Analytics Work to the Answer You Need
“Why did revenue drop” and “what will revenue be next quarter” look like neighbouring questions. They need different methods and different evidence.
-
Decision Question
“What result would change your mind?” is the most valuable question at intake and the least often asked, because the answer is sometimes none.
-
Prescriptive Analytics
A system that recommends without a declared objective has one anyway — chosen by whoever wrote the code, and invisible to everyone else.
-
Advice Process
Anyone may decide, provided they first consult those affected and those with expertise. It removes the need for agreement — not anyone’s budget or risk mandate.
-
Risk-Based Decisions
Expected value is the right tool when you get many draws. When the bad outcome ends the game, averaging over it is the wrong question.