Low-pass Filtering
Group delay at 5 Hz: 9.5 samples for a 20-point average, 32.63 for a Butterworth cutting at 8 Hz.
-
-
Signal Processing
Move a tone half a bin and its power spreads over seven of them. A window changes the smear, not the truth.
-
Time Series
Observations indexed by time, where the sequence itself is data. Trend, seasonality, cycles and noise — and the distinction most summaries get wrong.
-
Bootstrap Confidence Intervals (CIs)
Sort 500 resampled values and read off two percentiles. That is the whole method, and it works for statistics that have no convenient formula.
-
Probability
Count the eleven totals and every one gets 0.0909. The true answers run from 0.0278 to 0.1667.
-
Mann–Whitney U Test
AUC = U/(n1 n2) holds for the positive group’s U. Using min(U1, U2) turns 0.9600 into 0.0400.
-
Bayesian Decision Theory (BDT)
Same posterior, P(p > 0.6) = 0.703716 throughout. The decision flips between a cost ratio of 3 and 4.
-
Bayesian Time Series
One-step coverage: plug-in 0.9444, posterior predictive 0.9513, known coefficient 0.9521.
-
Posterior probability of uplift
A direct answer to how likely it is that the treatment helped — and how much less comfortable that answer becomes once a threshold that matters is attached.
-
Variational Inference (VI)
At a correlation of 0.99, mean-field reports a standard deviation of 0.141067 where the truth is 1.0.
-
MCMC (Markov Chain Monte Carlo)
Four proposal widths, all landing near the right answer, worth 19, 984, 7,913 and 1,893 effective draws.
-
Sequential Settings
A bandit sends 91% of traffic to one arm and estimates the difference 1.6 times less precisely.
-
Frequentist
Unequal sizes break the pooled t-test both ways: 0.0006 one way, 0.2918 the other. Welch holds.
-
Beta Distribution
Beta(0.8, 0.2), Beta(4, 1) and Beta(400, 100) all have mean 0.8000. P(0.7 < p < 0.9) is 0.1512, 0.4160 and 1.0000.
-
Binomial Likelihood
Two designs, identical data, likelihoods differing by a constant factor of 4. The posterior is Beta(10, 4) either way.
-
Posterior belief
Posterior beliefs depend on stated assumptions. Coherence, interval coverage, prior sensitivity, and new evidence test different aspects of the claim.
-
Marginal Likelihood
Against a fixed-rate model the Bayes factor reads 0.7758 under one prior and 1.0519 under another. The direction flips.
-
Posterior
A posterior is a distribution, not just an estimate. Summaries, credible intervals, future predictions, and sequential updates use it in different ways.
-
Prior Belief (or Prior Probability)
A prior specifies assumptions before the analyzed data. Its concentration, parameterization, predictive implications, and sensitivity all affect the posterior.
-
Parameter(s) of Interest
Ten segments with no effect anywhere: pick the best-looking one and the reported effect averages 0.0146.
-
Bayes’ Theorem
Bayes’ theorem combines prior probabilities and likelihoods. A counting example shows why reversing conditional probabilities gives the wrong answer.
-
Conversion Rate Uplift
The change in conversion rate caused by a variant, on two scales that differ by a factor of the baseline. The relative figure needs its own interval formula.
-
Bayesian Stopping Rules
Futility at 0.20 abandons 59.51% of no-effect experiments; the median sample across all runs is 500. The simulated power falls by about 15 points with a real lift.
-
Google Experiments
Google Optimize closed on 30 September 2023 and was not replaced; Google points to third-party tools.
-
Optimizely
Twenty metrics, five true effects: Bonferroni finds 2.45 of five true effects, Benjamini-Hochberg 3.26.
-
Online Experimentation Platforms
At a pre/post correlation of 0.7, variance reduction buys the same precision with half the traffic.
-
Stopping Rules
When an experiment ends must be decided before the results are seen. The kinds of rule in use, which error rate each protects, and what looking early costs.
-
Treatment Effect
The outcome with the treatment minus the outcome without it, for a unit or on average. Which average, on which scale, for whom: the estimand an estimate needs.
-
Posterior Probability
Posterior probability measures a specified event under a prior and likelihood. It differs from a p-value and needs a decision-relevant effect threshold.
-
Bayesian Sequential Testing
One look at the end calls B better 5.03% of the time on identical variants. Looking every twenty users: 27.96%.
-
Likelihood Ratio (LR)
At ten tosses, -2 log lambda exceeds its 95th percentile 0.1084 of the time under a true null.
-
Sequential Probability Ratio Test (SPRT)
A fixed test needs 69 observations; the SPRT averages 34.3 under the null. Its 99.9th percentile is nine times its median.
-
Pocock Method
A Pocock design with four looks judges its final analysis at p = 0.01823. A p of 0.03 at the end is not significant.
-
O’Brien–Fleming (OBF) Method
At 25% information the example needs z = 4.049. Under its target effect, 0.39% of simulated trials stop at that look.
-
Group Sequential Testing
Pocock uses 71.8% of the data and gives up eight points of power. O’Brien-Fleming gives up one for a smaller saving.
-
Type I Error
You choose a nominal significance level. Actual Type I error and power depend on the test, assumptions, sample size, and effect.
-
Traditional A/B Test (Fixed-Horizon A/B Test)
A fixed-horizon A/B test needs a defined population, assignment, outcome, and stopping rule. Logging, dependence, and time variation affect interpretation.
-
Fixed-Horizon Testing
One test at the horizon gives 0.0497. Twenty unplanned looks give 0.2477 with the same statistic and the same 1.96.
-
True Conversion Rate
The unobservable rate behind the number in the dashboard. What defines it, and why the textbook interval around it is not as safe as it looks.
-
Standard Error (SE)
Standard error describes sampling variability; a margin of error also uses a confidence level and reference distribution. A normal 95% interval uses about 1.96 times the SE.
-
True Mean (Population Mean, μ)
The population average an experiment is trying to learn: fixed, unobserved, and defined only once the population itself is.
-
Margin of Error (MoE)
A margin of error needs a confidence level to exist. Quote one without it and you are off by the multiplier — about a factor of two at 95%.
-
Critical Value
The significance level translated onto the statistic’s scale. How 1.645 and 1.96 arise, why two-sided tests use alpha over two, and the mistakes it invites.
-
Sample Standard Deviation
Spread around the sample mean, dividing by n minus 1. Not the average deviation, not the standard error, and the sigma a sample-size calculation needs.
-
Sample Mean
The average of what you measured, standing in for the average you cannot. Unbiased, sensitive to outliers, and slower to behave than n over 30 suggests.
-
Proportion
Enumerated exactly, not simulated: the Wald interval delivers 0.8775 where it promises 0.95.
-
True Population Parameter
Split the same outcomes by what was observed and coverage runs from 1.0000 to 0.5979.
-
Compromise Power Analysis
The power calculation for a sample you cannot change: fix the ratio of the two error rates, and let the critical value and both rates follow.
-
Post Hoc Power Analysis
Observed power and the p-value are two readings of the same statistic. A near-miss result will always come back as low power.
-
A Priori Power Analysis
Before collecting data, a priori power analysis supports design decisions under stated effect and variability assumptions. With a fixed sample, sensitivity calculations answer a different planning question.