Part III: Discovery Through Data and Models
Chapter 32: Bayesian Discovery and Uncertainty

Bayesian Discovery and Uncertainty

"I started with strong opinions about the rate constant. Then the data arrived and, to my credit, I updated. Slightly."

A Bayesian Prior With Strong Opinions

Overview

Science advances by proposing hypotheses and testing them against evidence. The Bayesian framework makes this cycle explicit and quantitative: a prior encodes what we believe before seeing data, a likelihood connects the hypothesis to observable quantities, and the posterior tells us what we should believe after the evidence arrives. This chapter builds the complete Bayesian discovery toolkit, from specifying models that encode competing scientific hypotheses, through the computational machinery for computing posteriors (MCMC, variational inference, normalizing flows), to the decision-theoretic tools that tell us which experiment to run next.

What makes Bayesian methods uniquely valuable for discovery is their treatment of uncertainty as a first-class object. A point estimate tells you where the answer probably is; a posterior distribution tells you how much you should trust that answer, where the remaining ambiguity lies, and what observation would most efficiently resolve it. For scientific applications, this distinction is critical: reporting a drug effect size of 0.3 is very different from reporting a posterior that places 90% of its mass between 0.1 and 0.5 versus one that places 90% between 0.29 and 0.31.

The ideas here build on the probability foundations from Appendix A and the data modeling concepts from Chapter 5. They connect backward to the causal inference methods of Chapter 31 (Bayesian networks are a special case of causal graphs) and forward to the scientific machine learning of Chapter 33 (where Bayesian calibration meets physics-informed models). The automated experiment design of Chapter 46 relies heavily on the value-of-information calculations we develop here.

Prerequisites

Readers should be comfortable with probability distributions, conditional probability, and Bayes' theorem from Appendix A: Mathematical Foundations. Familiarity with gradient-based optimization from Chapter 26: Representation Learning will help with the variational inference material. The Chapter 5 treatment of statistical models provides useful context. No prior exposure to MCMC or PyMC is assumed.

Learning Outcomes

Sections

32.1 The Bayesian Workflow

Bayesian models as scientific hypotheses. Choosing priors: uninformative, weakly informative, and domain-informed. Likelihoods for common data types. Prior predictive checks. The iterative model-building cycle with PyMC and ArviZ.

32.2 Posterior Inference Methods

MCMC fundamentals: Metropolis-Hastings, Hamiltonian Monte Carlo, and the NUTS sampler. The geometry behind NUTS. Variational inference with ADVI. Normalizing flows for amortized inference. Diagnostics: R-hat, ESS, divergences.

32.3 Model Comparison and Decision Making

WAIC, LOO-CV, and Bayes factors for comparing competing scientific hypotheses. Calibration: does your model's uncertainty match reality? Value of information: deciding which experiment to run next.

32.4 Building a Bayesian Discovery Model

Recipe: two competing hypotheses about enzyme kinetics, full PyMC inference, LOO-CV comparison, expected information gain calculation, and integration with the Discovery Workbench.

What's Next

Bayesian methods give us calibrated uncertainty about model parameters and predictions. Chapter 33: Scientific Machine Learning takes the next step by embedding physical laws directly into the model structure, using physics-informed neural networks, neural ODEs, and operator learning. The Bayesian calibration techniques from this chapter combine naturally with physics-informed constraints: the prior encodes physical law, the likelihood connects to observations, and the posterior quantifies what remains uncertain even after the physics and the data have both spoken.

Bibliography

Foundational Papers

Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. arXiv:1701.02434.

The definitive tutorial on HMC and NUTS geometry, explaining why Hamiltonian dynamics make MCMC efficient in high dimensions. Essential reading for Section 32.2.

Hoffman, M. D. & Gelman, A. (2014). The No-U-Turn Sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. JMLR, 15(47), 1593-1623.

The NUTS paper that made HMC practical by eliminating manual tuning of path length, now the default sampler in PyMC and Stan.

Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. JASA, 112(518), 859-877.

Comprehensive review connecting variational inference to classical statistics, covering ADVI and its relationship to MCMC.

Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5), 1413-1432.

The foundational paper on PSIS-LOO-CV for Bayesian model comparison, the method implemented in ArviZ and used throughout Section 32.3.

Books

Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian Data Analysis, 3rd edition. CRC Press.

The standard graduate reference for Bayesian statistics, covering theory, computation, and applications. Chapter 7 on model checking is particularly relevant.

Martin, O. A., Kumar, R., & Lao, J. (2022). Bayesian Modeling and Computation in Python. CRC Press.

A practical, code-first introduction to Bayesian modeling with PyMC, ArviZ, and related tools. Excellent companion to this chapter.

Tools & Libraries

Abril-Pla, O., Andreani, V., Carroll, C., et al. (2023). PyMC: A modern and comprehensive probabilistic programming framework in Python. PeerJ Computer Science, 9, e1516.

The primary probabilistic programming framework used in this chapter, built on PyTensor with NUTS sampling and ADVI.

Phan, D., Pradhan, N., & Jankowiak, M. (2019). Composable effects for flexible and accelerated probabilistic programming in NumPyro. arXiv:1912.11554.

JAX-based probabilistic programming with hardware acceleration. Used in Section 32.2 for GPU-accelerated MCMC and in Section 32.4 for the full recipe.

Kumar, R., Carroll, C., Hartikainen, A., & Martin, O. (2019). ArviZ: A unified library for exploratory analysis of Bayesian models in Python. JOSS, 4(33), 1143.

The visualization and diagnostics library for Bayesian inference, providing R-hat, ESS, LOO-CV, and publication-quality plots.

Wong, K. W. K., Gabrie, M., & Bhardwaj, U. (2023). flowMC: Normalizing-flow enhanced sampling package for probabilistic inference. JOSS, 8(83), 5021.

Combines normalizing flows with MCMC for efficient sampling of multimodal posteriors, particularly useful in astrophysics and gravitational-wave inference.

Tutorials & Surveys

Gelman, A., Vehtari, A., Simpson, D., et al. (2020). Bayesian workflow. arXiv:2011.01808.

The authoritative guide to iterative Bayesian model building: prior predictive checks, fake-data simulation, posterior predictive checks, and model comparison. The workflow that structures Section 32.1.

Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., & Lakshminarayanan, B. (2021). Normalizing flows for probabilistic modeling and inference. JMLR, 22(57), 1-64.

Comprehensive survey of normalizing flows, covering architectures, training, and applications to variational inference and density estimation.

Cranmer, K., Brehmer, J., & Louppe, G. (2020). The frontier of simulation-based inference. PNAS, 117(48), 30055-30062.

Survey of simulation-based inference connecting Bayesian methods to scientific simulators, bridging this chapter to Chapter 43 on scientific simulation.