Hamiltonian Monte Carlo (HMC) is powerful but hard to program and tune manually:

• You need to write code for log posterior densities,
• Derive and code gradients,
• Choose good mass matrices,
• Tune step sizes,
• Select the number of leapfrog steps,
• And diagnose convergence.

For realistic Bayesian models—especially high-dimensional or hierarchical ones—this is both complicated and error-prone.

To address this, the authors and collaborators created Stan, which automates nearly all the difficult parts of HMC.

Stan stands for:

Sampling Through Adaptive Neighborhoods

Its core mission is to take a Bayesian model and automatically:

1. Parse the data and model definition,
2. Compute the log posterior density,
3. Compute gradients automatically,
4. Apply the No-U-Turn Sampler (NUTS), an adaptive version of HMC,
5. Tune hyperparameters during a warm-up phase,
6. Run efficient inference,
7. Monitor convergence and summarize results.

Below is how each step works in detail.

---

1. Entering the Data and Model

A Stan model is written using a declarative syntax consisting of:

• data block
• parameters block
• model block
• (optionally) transformed data, transformed parameters, generated quantities, etc.

Every line of the Stan model contributes to defining the log posterior density.

Stan code can include:

• looping structures,
• conditional statements,
• intermediate computations,
• vector/matrix operations,
• standard probability distributions,
• built-in mathematical functions (exp, logit, etc.).

Everything you write is ultimately compiled into efficient C++ code.

Built-in probability distributions

Stan has optimized implementations of:

• Normal
• Gamma
• Binomial
• Poisson
• etc.

You can also write custom distributions by directly programming the log density.

---

2. Automatic Differentiation (Autodiff)

HMC requires gradients of the log posterior density.

Instead of requiring the user to derive gradients manually, Stan uses automatic analytical differentiation (C++ autodiff).

How autodiff works:

1. Stan parses the C++ expression tree for each computation.
2. It keeps track of:
   • operations,
   • intermediate variables,
   • dependencies.
3. During the reverse pass, it walks backward along the expression tree and applies the chain rule to compute gradients.

Optimizations:

• Many common statistical expressions have their gradients pre-programmed for higher speed.
• Dynamic programming ensures calculations are not repeated unnecessarily.

Outcome:

Stan’s gradients are dramatically faster and more accurate than numerical finite-difference approximations.

---

3. Additional Run Specifications

When calling Stan, you must supply:

• Number of chains
• Iterations per chain
• Warm-up iterations (tuning phase)
• Optional:
  • Initial values
  • Control parameters (adaptation intensity, max treedepth, etc.)

If the user does not provide starting values, Stan automatically generates reasonable random initials.

---

4. Warm-up Phase (Tuning HMC Parameters)

Warm-up is the most important part of Stan’s workflow.

Even though NUTS eliminates the need to choose L (number of leapfrog steps), Stan still needs to tune:

• the mass matrix $M$, and
• the step size $ϵ$.

Stan adapts these during warm-up using algorithms inspired by stochastic optimization.

What Stan does during warm-up:

• It estimates posterior scales and correlations to build a good mass matrix.
• It adjusts the step size to target a good acceptance rate.
• It learns the geometry of the posterior.

This adaptation:

• Does not always succeed,
• Particularly when the posterior curvature varies dramatically across regions,
• But often gives dramatically better performance than fixed HMC.

If chains mix poorly:

• You can inspect the tuned values of $M$ and $ϵ$ for each chain.
• You might reparameterize the model.

Stan is actively incorporating Riemannian adaptation, which uses a position-dependent metric to adapt more flexibly.

---

5. No-U-Turn Sampler (NUTS)

Stan implements HMC via NUTS, which:

• Automatically determines the appropriate path length (instead of fixed L),
• Uses reversibility and detailed balance to choose among candidate states,
• Avoids “U-turns,” i.e., useless trajectory doubling-backs,
• Makes HMC fully automatic and self-tuning.

Stan also:

• Automatically transforms bounded parameters to unconstrained spaces, when possible:
  • Positive → log transform
  • Bounded (0,1) → logit transform
• For more complex constraints (e.g., ordered parameters, simplex), Stan performs more sophisticated transformations or requires user reparameterization.

Stan continuously tracks:

• acceptance probabilities,
• divergences,
• step sizes,
• energy diagnostics.

These help users diagnose mixing problems.

---

6. Inference and Postprocessing

After warm-up:

• Stan discards warm-up samples (but keeps them for diagnostic inspection).
• Stan outputs multiple Markov chain sequences.
• Stan computes:
  • Gelman–Rubin $\hat{R}$,
  • Effective sample size $n_{\text{eff}}$​,
  • Monte Carlo standard errors,
  • Posterior means, medians, intervals,
  • Traceplots and other diagnostics (if using the R or Python interface).

Warm-up samples are not used for inference, but:

• can show whether adaptation failed,
• reveal divergences,
• indicate poor reparameterization.

---

7. Summary of Stan’s Contribution

Stan provides:

1. Automatic gradient calculation
   Eliminates manual derivatives.
2. Adaptive HMC/NUTS
   Automatically tunes step sizes and path lengths.
3. Automatic parameter transformations
   Allows HMC to run on unconstrained spaces.
4. High-speed C++ implementation
   Fast and scalable to high-dimensional problems.
5. Convergence monitoring
   Provides diagnostics and summaries automatically.
6. Ease of use
   User only needs to write the model; Stan handles the rest.

The result:

Stan makes advanced HMC technology accessible, reliable, and efficient—transforming complex Bayesian models into practical, scalable inference tasks.