Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation

Gaussian process regression has proven very powerful in statistics, machine learning and inverse problems. A crucial aspect of the success this methodology, a wide range applications to complex real-world problems, is hierarchical modeling hyperparameters. The purpose paper study two paradigms parameters: one from probabilistic Bayesian perspective, particular, empirical Bayes approach that been largely used statistics; other deterministic approximation theoretic view, particular kernel flow algorithm was proposed recently literature. Analysis their consistency large data limit, as well explicit identification implicit bias parameter learning, are established for Mat\'ern-like model on torus. technical challenge we overcome regularity field, which results have scarce spatial statistics Moreover, conduct extensive numerical experiments beyond model, comparing algorithms further. These demonstrate parameters, such amplitude lengthscale; they also illustrate setting misspecification could show superior performance more traditional approach.