Chapter 50: Discovery AI for Physics and Engineering

Overview

Physics has always been the science of discovering compact mathematical laws from empirical observation. Kepler distilled years of planetary data into three elegant equations. Newton compressed all of mechanics into $F = ma$. Einstein rewrote gravity as spacetime curvature. In each case, the discoverer brought powerful priors: dimensional consistency, symmetry, conservation laws, and the conviction that nature's equations should be simple. The question animating this chapter is: can AI systems leverage the same priors to accelerate the discovery of physical laws and engineering models?

The answer involves three complementary strategies. Symbolic regression searches the space of mathematical expressions for equations that fit data while respecting dimensional analysis and known symmetries. Physics-informed neural networks (PINNs) embed partial differential equations directly into training losses, turning inverse problems (inferring unknown parameters from observations) into optimization problems that neural networks solve naturally. Neural operators learn mappings between function spaces, enabling surrogate models that generalize across initial conditions and boundary parameters without retraining.

These methods differ fundamentally from the black-box regression of Chapter 25. There, the goal was to find patterns in data. Here, the goal is to find physical laws: equations that generalize beyond the training distribution, obey conservation principles, and provide interpretable explanations for why a system behaves as it does. The difference is not merely aesthetic. A neural network that fits a drag coefficient curve cannot predict what happens when the Reynolds number exceeds its training range. A symbolic expression with the correct dimensional structure can.

This chapter builds on the scientific machine learning foundations of Chapter 33, the symbolic regression techniques of Chapter 35, and the differentiable programming framework of Chapter 42. It applies those tools to physics and engineering problems at graduate depth, with working code you can run on your own data. By the end, you will have a complete pipeline that discovers symbolic laws from data, solves inverse PDE problems, and compares symbolic and neural approaches on extrapolation tasks.

Prerequisites

This chapter assumes familiarity with Chapter 33: Scientific Machine Learning (PINNs and neural operators at the conceptual level), Chapter 35: Symbolic Regression and Equation Discovery (genetic programming and expression trees), and Chapter 42: Differentiable Programming for Discovery (JAX fundamentals and automatic differentiation). Readers should be comfortable with partial differential equations at the level of the heat equation and wave equation; Appendix A covers the necessary calculus. Experience with SymPy for symbolic computation is helpful but not required.

Learning Outcomes

Configure PySR with dimensional constraints and custom operators to discover physically meaningful equations from noisy data.
Formulate inverse PDE problems as PINN training objectives and solve them with DeepXDE, recovering unknown coefficients from sparse observations.
Implement a Fourier Neural Operator (FNO) for parametric PDE solving and explain why neural operators generalize across initial conditions.
Build a Bayesian optimization loop with BoTorch for engineering design problems with expensive simulations.
Compare symbolic regression, PINNs, and standard neural networks on extrapolation benchmarks, quantifying when each method wins.
Integrate Noether's theorem as a structural prior for conservation-law discovery.

Sections

50.1 Symbolic Regression for Physics

Dimensional analysis as a hard constraint. PySR with custom operators and unit-aware search. Rediscovering Kepler's third law and the drag equation. SymPy verification and simplification. When symbolic regression outperforms neural networks on extrapolation.

50.2 Physics-Informed Learning

PDE residual losses for forward and inverse problems. Solving the inverse heat equation: recovering thermal diffusivity from sparse temperature measurements. Noether's theorem as a conservation-law prior. Simulation-based inference with normalizing flows. DeepXDE and JAX implementations.

50.3 Neural Operators for PDEs

The universal approximation theorem for operators. Fourier Neural Operator architecture and spectral convolutions. Learning parameterized Darcy flow and Navier-Stokes surrogate models. Comparing FNO, DeepONet, and classical solvers on accuracy, speed, and generalization.

50.4 Building a Physics Discovery Pipeline

Recipe: PySR law discovery from simulation data, PINN inverse problem to recover unknown parameters, neural operator surrogate for rapid evaluation, and BoTorch optimization over the surrogate. End-to-end pipeline on a heat transfer design problem with Discovery Workbench integration.

What's Next

Physics and engineering discovery relies on compact mathematical laws, conservation principles, and dimensional consistency. Chapter 51: Discovery AI for Climate and Earth Science shifts to a domain where the governing equations are known but the systems are so complex, chaotic, and multi-scale that analytical solutions are impossible. There, AI discovery focuses on learning subgrid parameterizations, downscaling climate projections, and detecting regime shifts in Earth system data. The neural operator and surrogate modeling techniques from this chapter transfer directly, but the evaluation criteria change: in climate science, the question is not "did we find the right equation?" but "does this surrogate reproduce the statistics of the full simulation over decades?"

Bibliography

Symbolic Regression for Physics

Cranmer, M. (2023). Interpretable machine learning for science with PySR and SymbolicRegression.jl. arXiv:2305.01582.

The definitive reference for PySR, covering multi-objective symbolic regression with dimensional constraints, custom operators, and integration with Python scientific workflows.

Udrescu, S.-M. & Tegmark, M. (2020). AI Feynman: A physics-inspired method for symbolic regression. Science Advances, 6(16), eaay2631.

Introduces dimensional analysis, symmetry detection, and separability testing as preprocessing steps that dramatically narrow the symbolic search space for physics equations.

Tenachi, W., Ibata, R., & Diakogiannis, F. I. (2023). Deep symbolic regression for physics guided by dimensional analysis. Nature Computational Science, 3, 1036-1045.

PhySO: a deep reinforcement learning approach to symbolic regression that uses dimensional analysis to constrain the search space, achieving state-of-the-art results on physics benchmarks.

Physics-Informed Neural Networks

Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686-707.

The foundational PINN paper establishing PDE residual losses for both forward (solving PDEs) and inverse (parameter recovery) problems.

Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021). DeepXDE: A deep learning library for solving differential equations. SIAM Review, 63(1), 208-228.

The DeepXDE library paper, providing a high-level API for PINNs with support for multiple backends, complex geometries, and inverse problems.

Neural Operators

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhatt, K., Stuart, A., & Anandkumar, A. (2021). Fourier Neural Operator for parametric partial differential equations. Proc. ICLR.

The FNO paper: spectral convolutions in Fourier space that learn resolution-invariant PDE solution operators, with 1000x speedups over classical solvers.

Lu, L., Jin, P., Pang, G., Zhang, Z., & Karniadakis, G. E. (2021). Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3, 218-229.

DeepONet formalizes operator learning with the branch-trunk architecture, grounded in the universal approximation theorem for operators (Chen & Chen, 1995).

Chen, T. & Chen, H. (1995). Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, 6(4), 911-917.

The theoretical foundation proving that neural networks can approximate any continuous nonlinear operator, the theoretical backbone of DeepONet and all neural operator methods.

Engineering Optimization

Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., & Bakshy, E. (2020). BoTorch: A framework for efficient Monte-Carlo Bayesian optimization. NeurIPS.

BoTorch provides composable Bayesian optimization primitives on top of GPyTorch, enabling multi-objective optimization over expensive engineering simulations.

Cranmer, K., Brehmer, J., & Louppe, G. (2020). The frontier of simulation-based inference. PNAS, 117(48), 30055-30062.

A comprehensive review of simulation-based inference methods, covering neural ratio estimation, neural posterior estimation, and sequential approaches for likelihood-free problems in physics.

Tools & Libraries

PySR: High-performance symbolic regression in Python.

The PySR library used throughout this chapter for dimensionally-constrained equation discovery, backed by the SymbolicRegression.jl Julia engine.

SymPy: Symbolic mathematics library for Python.

Used for expression simplification, dimensional verification, and converting discovered equations into LaTeX and code.

NeuralOperator: Neural operator architectures in PyTorch.

Reference implementations of FNO, DeepONet, and related architectures for operator learning on PDE problems.