1 Proof of Theorem 2 Here we proof the result that we can estimate the expected log likelihood and its gradients using expectations over low-dimensional Gaussians, that is that Theorem. For the structured gp model defined in our paper the expected log likelihood over the given variational distribution and its gradients can be estimated using expectations over Tn-dimensional Gaussians and |V|-di...

where π denotes a prior distribution on w, which we set as zero-mean Gaussian, π(w) ∝ exp(− 12 ||w|| ). We choose Qw as a Gaussian centered at αw, Qw(w̄) ∝ exp(− 12 ||w̄ − αw|| ). Then, the KL divergence between Qw and π is just α||w||/2. Analyzing the sample risk L(Qw, S) can be done for each training instance due to i.i.d. sampling. We denote by Ȳ i the predicted output for x with respect to w̄....