1. Purpose of Conditional Modeling

Conditional modeling focuses on understanding how a response variable $y$ changes as a function of one or more explanatory variables $x$.

The goal is to model the conditional distribution: $p(y \mid x, \theta)$

under the assumption that the observed data pairs $(x_i, y_i)$ for $i=1,\dots,n$ are exchangeable.

This approach is used widely in scientific and social science research whenever the relationship between variables is the primary interest.

---

2. Notation and Key Components

Response variable

• $y$: The outcome we want to explain or predict.
• Typically assumed to be continuous in the linear regression context.

Explanatory variables

• $x = (x_1, \dots, x_k)$: Predictors, may be continuous or categorical.
• One variable may be designated as the treatment variable.
• Remaining variables act as control variables.

Data structure

• $y$: an $n \times 1$ vector of outcomes.
• $X$: an $n \times k$ matrix of predictors.
• Indices:
  • $i$ indexes units (subjects)
  • $j$ indexes variables

---

3. The Normal Linear Model

The most used conditional model is the normal linear regression model, where:

$E(y_i \mid X, \beta) = \beta_1 x_{i1} + \dots + \beta_k x_{ik}$

Often, $x_{i1} = 1$ for all $i$, so $\beta_1$​ acts as the intercept.

Assumptions for ordinary linear regression

1. Equal variances $\operatorname{var}(y_i \mid X, \theta) = \sigma^2.$
2. Conditional independence
   • $y_1, \dots, y_n$ are independent given $X, \theta = (\beta, \sigma)$.

Thus, the full parameter vector is:

$\theta = (\beta_1, \dots, \beta_k, \sigma)$

---

4. Key Modeling Tasks

Two major tasks define successful use of linear regression:

(1) Choosing and transforming variables

• Select the right $x$ variables.
• Transform $X$ or $y$ so that:
  • The conditional expectation is approximately linear in $X$.
  • The residuals behave approximately normally.

(2) Setting a prior distribution

The prior for $\beta$ and $\sigma$ must:

• Reflect real prior knowledge,
• Still allow the data to influence posterior inference,
• Avoid being unrealistically strong.

---

5. Inference Goal

Bayesian inference aims to compute:

$p(\theta \mid X, y)$

which involves combining:

• The likelihood $p(y \mid X, \theta)$,
• The prior $p(\theta)$.

Posterior inference yields estimates for:

• Regression coefficients $\beta_j$,
• Noise standard deviation $\sigma$.

---

6. Flexibility of the Linear Model

Because:

• Any number of predictors may be included,
• Predictors and the response can be transformed arbitrarily,

the linear model becomes a highly flexible tool for describing relationships among variables.

Generalized linear models extend the same predictor structure to non-normal distributions.

---

7. Bayesian Justification for Conditional Models

Why conditional modeling is logically valid

A full Bayesian model must include a distribution for the predictors $X$:

$p(X \mid \psi)$

where $\psi$ contains parameters governing $X$.

Then the full generative model is:

$p(X, y \mid \psi, \theta)$

With priors:

$p(\psi, \theta) = p(\psi)\, p(\theta)$

assuming independence of $\psi$ and $\theta$ a priori.

Posterior factorization

Under prior independence:

$p(\psi, \theta \mid X, y) = p(\psi \mid X)\, p(\theta \mid X, y)$

Thus, the posterior for $\theta$ is:

$p(\theta \mid X, y) \propto p(\theta)\, p(y \mid X, \theta)$

Key insight

The parameters governing X, $\psi$, do not influence the posterior of $\theta$.

Therefore, ordinary regression analysis is fully justified, even though it ignores the distribution of $X$.

---

8. Designed experiments

When researchers choose $X$ by design (e.g., treatment assignment), the distribution of $X$ is not random and has no unknown parameters.

Thus, regression becomes simpler and fully justified without needing to model $p(X)$.

---

9. Practical advantage

Specifying the conditional distribution $p(y \mid X, \theta)$ is much easier than specifying the joint distribution of all variables:

$p(y, X)$

This is why regression is such a powerful and widely used modeling framework.