Derivative Enhanced Gaussian Processes

Gaussian process (GP) surrogate modeling has proven invaluable for approximating complex, computationally expensive simulations. While advancements have been made in incorporating first- and second-order derivatives into derivative-enhanced GPs, higher-order derivatives are seldom employed due to their prohibitive computational expense. However, recent progress in automatic differentiation techniques, particularly Hypercomplex automatic differentiation (HYPAD), has enabled the computation of accurate higher-order derivatives for complex computational models such as finite element analyses. This study investigates the application of higher-order derivatives in three distinct derivative-enhanced kriging strategies. The first approach, indirect derivative-enhanced kriging, generates synthetic training points via Taylor series extrapolation around existing samples. The second approach, weighted derivative-enhanced kriging, constructs a series of smaller kriging models and aggregates them with appropriate weight coefficients to manage large correlation matrices. The third approach, partial derivative-enhanced kriging, incorporates sensitivity information only along influential dimensions to mitigate computational overhead. Numerical experiments demonstrate that these higher-order derivative-informed methods alleviate issues such as ill-conditioning in the indirect approach and mitigate excessive matrix growth when compared to traditional derivative-enhanced kriging, while also improving predictive accuracy. The discussion concludes with an examination of the relative trade-offs among the three strategies and potential directions for further refinement of derivative-enhanced GP modeling.