1) Definition

For binary classification:

$\text{logit}(p) = \ln \left(\frac{p}{1-p}\right)$

Where:

  • $p$ = predicted probability of the positive class
  • logit(p) maps $p \in (0,1)$ to $(-\infty, +\infty)$.

2) Intuition

  • The model computes a linear combination of inputs:
    • $z = w^\top x + b$
    • This $z$ is the logit.
  • Then it converts the logit into a probability using:
    • Sigmoid: $\sigma(z) = \frac{1}{1+e^{-z}}$
    • Softmax (multiclass): $P(y=i) = \frac{e^{z_i}}{\sum_j e^{z_j}}$

So logits are the raw “scores” that get transformed into probabilities.

3) Example

Suppose logistic regression gives $z = 2.2$.

  • This is the logit.
  • Convert to probability:
    • $p = \sigma(2.2) = \frac{1}{1+e^{-2.2}} \approx 0.90$

Interpretation: The model is about 90% confident the instance is positive.

4) Why use logits instead of probabilities?

  1. Numerical stability: Working in logit space avoids underflow when probabilities are very close to 0 or 1.
  2. Better for loss functions:
    • Binary cross-entropy is more stable if you pass logits instead of probabilities.
    • Many ML libraries (TensorFlow, PyTorch, scikit-learn) have *_with_logits versions of loss functions for this reason.
  3. Linear modeling convenience: Logits are linear in weights $w^\top x$, probabilities are not.

5) Applications

  • Logistic Regression: logit is the link function connecting linear predictors to probability.
  • Neural Networks: the last dense layer often outputs logits; then activation (sigmoid/softmax) converts them to probabilities.
  • Interpretability: logit scale shows how much the model “leans” toward a class (positive logits → more likely positive, negative logits → more likely negative).

Summary:

  • Logits = raw model scores before probability transformation.
  • For binary classification, it’s the log-odds.
  • Probabilities are derived by applying sigmoid (binary) or softmax (multiclass).
  • Using logits is numerically stable and aligns better with how models are trained.