This talk will address various aspects of the application of Bayesian Gaussian process (GP) regression in model calibration. Focusing on the weight space view, I will outline GP utility for model calibration in the context of quantum chemical density functional theory. With this and other model problems, I will discuss matters of orthonormal basis choice and expansion order, and their impact on the solution. I will also discuss the a-posteriori application of constraints on the GP via projection, and will highlight the utilization of GPs for embedded model error estimation in physical models.