Regression Coefficient

1. Definition

• A regression coefficient is the number that represents the relationship between an independent variable (predictor) and the dependent variable (outcome) in a regression model.
• It tells us how much the dependent variable changes when the independent variable increases by one unit, holding all other variables constant.

---

2. Types of Regression Coefficients

• Simple Linear Regression (one predictor): $Y = β_0 + β_1 X + ε$
  • $β_0$ = intercept (value of Y when X = 0)
  • $β_1$​ = slope coefficient (change in Y for each 1-unit change in X)
• Multiple Linear Regression (many predictors):
  • $Y = β_0 + β_1 X_1 + β_2 X_2 + \dots + β_k X_k + ε$
    • Each $β_i$​ = partial regression coefficient (effect of $X_i$​ controlling for other variables).

---

3. Interpretation

• Sign (+ or –):
  • Positive coefficient → as X increases, Y increases.
  • Negative coefficient → as X increases, Y decreases.
• Magnitude (size):
  • Large |β| → stronger effect.
  • Small |β| → weaker effect.
• Statistical significance (p-value):
  • Checks if the coefficient is significantly different from 0.
  • If not significant → that predictor may not contribute meaningfully.

---

4. Examples

Example 1 – Simple Regression

• Model: Salary = 30,000 + 2,000 × (Years of Experience)
• Interpretation: Each extra year of experience increases salary by $2,000 on average.

Example 2 – Multiple Regression

• Model: House Price = 50,000 + 100 × (Square Feet) + 20,000 × (Garage)
• Interpretation:
  • Each additional square foot adds $100, controlling for garage.
  • Having a garage increases price by $20,000, controlling for size.

---

5. Standardized vs Unstandardized Coefficients

• Unstandardized (raw β): Expressed in original units (e.g., dollars, cm).
• Standardized (β*): Expressed in standard deviation units → allows comparison of effect sizes across variables with different scales.

---

6. In Logistic Regression

• Coefficients are in log-odds units.
• Exponentiating $β$ gives odds ratio → easier to interpret.

Example:

• Logistic regression coefficient $β = 0.7$.
• Odds ratio = $e^{0.7} ≈ 2.0$.
• Interpretation: 1-unit increase in predictor doubles the odds of the event.

---

7. Key Takeaways

• Regression coefficients quantify the direction and size of the effect of predictors on an outcome.
• Always interpret coefficients in context and alongside p-values, confidence intervals, and effect sizes.
• In multiple regression, coefficients show partial effects (controlling for other predictors).

---

In short:
A regression coefficient (β) shows how much the dependent variable (Y) changes when an independent variable (X) increases by one unit, keeping all other predictors constant.