1 Gaussian Process regression and Bayes statistics

In the following Video 1, we explain the concepts behind Bayes statistic, and how this relates to GP regression. We also introduce type-II MLE, later discussed here.

Video 1: Bayes statistic & Gaussian Process regression. The metaphor of the detective is borrowed from (Kruschke and Liddell (2018)).

2 Posterior updates

In this other short Video 2, we simply illustrate how the posterior gets updated when new data points are collected.

Video 2: Updating the posterior distribution with data collection. We start with a very simple GP prior, centred in 0 and with a constant 95% confidence region along the x-axis. As data are observed (data collection), the likelihood changes, hence updating the posterior distribution of the function via Bayes rule. The animation is a screen recording from this website (Görtler, Kehlbeck, and Deussen (2019)).

3 To go further

For interested and advanced users, we refer to the following website (Görtler, Kehlbeck, and Deussen (2019)) for a nice and visual introduction to GPs, and to the following fundamental book (Williams and Rasmussen (2006)).

References

Görtler, Jochen, Rebecca Kehlbeck, and Oliver Deussen. 2019. “A Visual Exploration of Gaussian Processes.” Distill. https://doi.org/10.23915/distill.00017.

Kruschke, John K., and Torrin M. Liddell. 2018. “The Bayesian New Statistics: Hypothesis Testing, Estimation, Meta-Analysis, and Power Analysis from a Bayesian Perspective.” Psychon Bull Rev 25 (1): 178–206. https://doi.org/10.3758/s13423-016-1221-4.

Williams, Christopher KI, and Carl Edward Rasmussen. 2006. Gaussian Processes for Machine Learning. Vol. 2. 3. MIT press Cambridge, MA.