Emilien Seiler

arXiv preprint

Stiff Transfer Learning for Physics-Informed Neural Networks

Emilien Seiler, Wanzhou Lei, Pavlos Protopapas

STL-PINN pipeline: a multi-head PINN is trained in the low-stiff regime (α < α₀); its latent space H is then reused to compute one-shot solutions in the high-stiff regime (α > α₀).
STL-PINN pipeline: a multi-head PINN is trained in the low-stiff regime (α < α₀); its latent space H is then reused to compute one-shot solutions in the high-stiff regime (α > α₀).

TL;DR Vanilla PINNs fail on stiff differential equations. STL-PINNs train a multi-head PINN in a mildly stiff regime, then transfer to highly stiff regimes in one shot, without retraining.

This work comes from my Master's thesis at Harvard University with Prof. Pavlos Protopapas.

Abstract

Stiff differential equations are prevalent in various scientific domains, posing significant challenges due to the disparate time scales of their components. As computational power grows, physics-informed neural networks (PINNs) have led to significant improvements in modeling physical processes described by differential equations. Despite their promising outcomes, vanilla PINNs face limitations when dealing with stiff systems, known as failure modes. In response, we propose a novel approach, stiff transfer learning for physics-informed neural networks (STL-PINNs), to effectively tackle stiff ordinary differential equations (ODEs) and partial differential equations (PDEs). Our methodology involves training a Multi-Head-PINN in a low-stiff regime, and obtaining the final solution in a high stiff regime by transfer learning. This addresses the failure modes related to stiffness in PINNs while maintaining computational efficiency by computing "one-shot" solutions. The proposed approach demonstrates superior accuracy and speed compared to PINNs-based methods, as well as comparable computational efficiency with implicit numerical methods in solving stiff-parameterized linear and polynomial nonlinear ODEs and PDEs under stiff conditions. Furthermore, we demonstrate the scalability of such an approach and the superior speed it offers for simulations involving initial conditions and forcing function reparametrization.

Results on stiff ODEs (OHO, NCFF, Duffing) and on the AR PDE: solutions learned in the training regime (top) and transferred to much stiffer regimes (bottom) match the Radau reference solver.
Results on stiff ODEs (OHO, NCFF, Duffing) and on the AR PDE: solutions learned in the training regime (top) and transferred to much stiffer regimes (bottom) match the Radau reference solver.

Citation

@misc{seiler2025stiff,
  title  = {Stiff Transfer Learning for Physics-Informed Neural Networks},
  author = {Seiler, Emilien and Lei, Wanzhou and Protopapas, Pavlos},
  year   = {2025},
  url    = {https://arxiv.org/abs/2501.17281}
}