Step 1: Odds

  • Odds represent the ratio of the probability of success ($p$) to the probability of failure ($1-p$).

$\text{Odds} = \frac{p}{1-p}$

Example: If $p = 0.8$, $\text{Odds} = \frac{0.8}{0.2} = 4$

Meaning success is 4 times as likely as failure.

Step 2: Log-Odds (Logit)

  • Log-odds are simply the natural logarithm of the odds.

$\text{Log-Odds} = \ln\left(\frac{p}{1-p}\right)$

  • Also called the logit function.
  • Maps probabilities ($p \in (0,1)$) to the entire real line ($-\infty, +\infty$).

Why Log-Odds Are Useful

  1. Linearization:
    • Probabilities are bounded between 0 and 1, but log-odds can take any real value.This allows us to use linear models (like logistic regression).
    • Logistic regression model:
      • $\ln\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 x_1 + \cdots + \beta_n x_n$
  2. Interpretability:
    • Each coefficient $\beta$ represents the change in log-odds for a one-unit change in the predictor.
    • Exponentiating gives odds ratios, which are more interpretable.

Example

Suppose probability of purchase $9p = 0.9$.

  1. Odds:

$\frac{0.9}{0.1} = 9$

Customer is 9x more likely to purchase than not.

  1. Log-odds:

$\ln(9) \approx 2.20$

If $p = 0.5$, odds = 1, log-odds = 0.
If $p < 0.5$, log-odds < 0.
If $p > 0.5$, log-odds > 0.

Applications

  • Logistic Regression: Predicts log-odds of binary outcomes.
  • Bayesian statistics: Often use log-odds for priors/posteriors.
  • Machine learning classification: Many models output logits (raw log-odds) before converting with sigmoid/softmax.

In short:

  • Odds = ratio of success to failure.
  • Log-odds = log of odds.
  • It’s a convenient way to transform probabilities into a scale where linear models work naturally.