Overview
Classical scientific computing solves partial differential equations on grids. The finite element method, spectral methods, and molecular dynamics simulations have powered physics, chemistry, and engineering for half a century. But they share a common bottleneck: computational cost scales steeply with accuracy. A high-fidelity turbulence simulation can consume millions of CPU hours. A single density functional theory (DFT) calculation for a moderate protein takes days. The question driving this chapter is: can neural networks learn the structure of physical laws well enough to accelerate these simulations by orders of magnitude, without sacrificing scientific validity?
Scientific machine learning (SciML) answers with a qualified yes. The qualification matters. Naive neural networks ignore conservation laws, symmetries, and boundary conditions. They extrapolate poorly outside their training distribution. And they produce predictions without uncertainty estimates. The methods in this chapter address each of these failures by building physical knowledge directly into network architectures and training procedures.
We begin with physics-informed neural networks (PINNs), which encode partial differential equations as training losses so the network learns solutions that satisfy known physics. We then move to neural operators (DeepONet, Fourier Neural Operator) that learn mappings between function spaces, enabling zero-shot generalization to new initial conditions and forcings. Next, we cover equivariant neural networks that respect the symmetry groups (E(3), SE(3), O(3)) governing molecular and physical systems, with architectures like MACE, NequIP, GemNet, and EquiformerV2. Finally, we put everything together by training a universal force field on molecular data, benchmarking against DFT calculations.
These ideas connect directly to the representation learning foundations of Chapter 26, the scientific foundation models of Chapter 27, and the differentiable programming framework of Chapter 42. They also feed forward into the domain-specific applications in Chapter 49: Chemistry and Materials and Chapter 50: Physics and Engineering.
Prerequisites
Readers should be comfortable with Chapter 26: Representation Learning for neural network fundamentals and Chapter 5: Discovery Through Data, Models, and Simulation for the simulation concepts we accelerate here. Basic familiarity with partial differential equations (the heat equation, wave equation) and molecular energy concepts (potential energy surfaces, forces as gradients) is helpful; Appendix A covers the necessary calculus and linear algebra. No prior experience with computational physics software is assumed.
Learning Outcomes
- Formulate PDE residual losses and train physics-informed neural networks with DeepXDE and JAX.
- Explain how DeepONet and FNO learn operator mappings between function spaces, and implement both architectures.
- Describe E(3), SE(3), and O(3) equivariance and why physical symmetries must be hard-coded into architectures for molecular and materials modeling.
- Compare MACE, NequIP, GemNet, and EquiformerV2 architectures for interatomic potentials.
- Train a MACE model on a molecular dataset, evaluate against DFT reference calculations, and interpret the results.
- Use JAX, DeepXDE, NeuralOperator, MACE, and torchdiffeq as production tools for scientific ML tasks.
Sections
33.1 Physics-Informed Neural Networks
PDE residual losses, boundary condition enforcement, and the PINN training loop. Solving the heat equation, Burgers' equation, and Navier-Stokes with DeepXDE and JAX. Failure modes: spectral bias, causal training, and when PINNs struggle.
33.2 Neural Operators
Learning maps between function spaces with DeepONet (branch-trunk architecture) and the Fourier Neural Operator (spectral convolutions in frequency space). Zero-shot generalization to new initial conditions. The NeuralOperator library.
33.3 Equivariant Neural Networks
E(3), SE(3), and O(3) symmetry groups for molecular and physical systems. Spherical harmonics and irreducible representations. MACE, NequIP, GemNet, and EquiformerV2 architectures. GNoME and FairChem for materials discovery.
33.4 Building a Universal Force Field
Recipe: train a MACE model on molecular energy and force data, benchmark against DFT calculations, and deploy for molecular dynamics. Differentiable simulation with torchdiffeq. Integration with the Discovery Workbench.
Bibliography
Foundational Papers
The seminal PINN paper that established PDE residual losses as a training objective, launching the modern SciML field.
DeepONet: the branch-trunk architecture for learning operator mappings between infinite-dimensional function spaces.
The Fourier Neural Operator that performs convolutions in spectral space, achieving resolution-invariant PDE solving.
MACE: the multi-body equivariant architecture that achieves state-of-the-art accuracy for interatomic potentials.
GNoME: Google DeepMind's graph network that predicted 2.2 million stable crystal structures, validated by experimental synthesis.
Equivariant Architectures
NequIP: the first E(3)-equivariant GNN to achieve DFT-level accuracy with remarkably small training sets.
GemNet: directional message passing with two-hop interactions for molecular property prediction.
EquiformerV2: an equivariant Transformer that scales to high-degree spherical harmonics for molecular modeling.
Tools & Libraries
The most mature PINN library, supporting forward and inverse problems with multiple backends (TensorFlow, PyTorch, JAX).
Reference implementations of FNO, DeepONet, and related operator learning architectures.
The production MACE implementation used in Section 33.4 for training universal force fields.
The torchdiffeq library for differentiable ODE solvers, enabling gradient-based optimization through simulation.
JAX provides automatic differentiation and JIT compilation, making it the natural substrate for differentiable scientific computing.
Datasets & Benchmarks
Large-scale molecular datasets (OC20, OC22) with DFT-computed energies and forces for training and evaluating interatomic potentials.