Definition

Model weights are the trainable parameters of a machine learning model that determine how input features are transformed into predictions.

  • They represent the strength or importance of each input feature in the model.
  • During training, weights are adjusted (optimized) to minimize loss and improve predictions.

Examples by Model Type

  1. Linear Regression

$y = w_1x_1 + w_2x_2 + b$

  • $w_1, w_2$​ = model weights.
  • They tell how much each feature $x_1, x_2$​ contributes to prediction $y$.
  • Example: If $w_1 = 200$, every extra square foot adds $200 to house price.
  1. Neural Networks
  • Each connection between neurons has a weight.
  • Forward pass = multiply inputs by weights, apply activation.
  • Training = backpropagation adjusts weights to reduce error.
  • Example: Image classification → weights in CNN filters learn patterns (edges, textures).
  1. Logistic Regression
  • Weights define the log-odds contribution of each feature.
  • Positive weight → feature increases probability of positive class.
  • Negative weight → feature decreases probability.

How Weights Are Learned

  • Start: initialized randomly or with heuristics.
  • Training loop:
    1. Forward pass → compute prediction.
    2. Loss function → compare with true label.
    3. Backpropagation → compute gradients of loss w.r.t. weights.
    4. Optimizer (SGD, Adam) → update weights.

Why Weights Matter

  • They store the knowledge the model has learned.
  • Saving/loading a model = saving/loading its weights.
  • In transfer learning, we often freeze weights in the encoder and only fine-tune the last layers.

Example

  • Spam email classifier:
    • Weight for keyword “free” = +2.5 (strongly increases spam probability).
    • Weight for “invoice” = -1.0 (decreases spam probability).

Summary
Model weights = the parameters that a model learns during training.
They control how features influence predictions, are updated via optimization, and ultimately capture the model’s learned knowledge.