Type I Error
You choose a nominal significance level. Actual Type I error and power depend on the test, assumptions, sample size, and effect.
-
-
Critical Value
The significance level translated onto the statistic’s scale. How 1.645 and 1.96 arise, why two-sided tests use alpha over two, and the mistakes it invites.
-
Statistical Significance
Statistical significance says how incompatible the data are with a null model at a chosen threshold. It does not say how large or how important an effect is.
-
Significance Level (α)
α is a tolerance you choose, not a measure of evidence. And the risk it trades against goes up as you push it down.
-
Hypothesis Testing
A test rejects or fails to reject — it never confirms. And “fail to reject” covers both no effect and not enough data to tell.
-
Statistically Significant
“Unlikely due to chance” is the one reading a p-value cannot support. What it measures is incompatibility with a model, assumptions included.
-
Confidence Level
95% is a property of the method, not of the interval in front of you. A higher level targets greater coverage across samples and usually widens the interval; it does not assign a posterior probability to the observed interval.
-
Confidence Intervals (CIs)
95% describes the procedure across repeated samples, not your confidence in this particular interval. The distinction changes how you report it.