Gaussian and elliptical processes, and uncertainty-guided experimental design.
When an experiment or a simulation is expensive, a model has to do more than predict. It has to say how sure it is, and where to look next. That puts uncertainty at the center of the work, because a plan for the next experiment is only as good as the uncertainty it is built on. This theme develops probabilistic models whose uncertainty is trustworthy enough to steer real decisions.
Gaussian processes are the standard tool here, and they have a surprising blind spot: their posterior variance depends on the measurements only through the hyperparameters, so exploration barely reacts to what was actually observed. Warping the input space with a learned, monotone transformation fixes this, letting the acquisition policy stretch or compress regions in response to observed variability (Jarl et al., 2026). A second, subtler problem is geometric: stationary kernels inflate the variance near the boundary of the domain, which quietly pushes sampling toward corners and edges regardless of the objective. We characterize this effect and give a diagnostic for it (Bånkestad et al., 2026).
On the modeling side, elliptical processes generalize Gaussian and Student’s t processes into one family with flexible heavy tails, trained by variational inference (Bånkestad et al., 2023). They are useful whenever the likelihood is non-Gaussian or the tails genuinely matter.
These ideas are meant to be used. An ongoing project applies active learning to molecular solubility, choosing which measurements to run so that a useful model emerges from as few experiments as possible.
Bayesian active learning relies on the precise quantification of predictive uncertainty to explore unknown function landscapes. While Gaussian process surrogates are the standard for such tasks, an underappreciated fact is that their posterior variance depends on the observed outputs only through the hyperparameters, rendering exploration largely insensitive to the actual measurements. We propose to inject observation-dependent feedback by warping the input space with a learned, monotone reparameterization. This mechanism allows the design policy to expand or compress regions of the input space in response to observed variability, thereby shaping the behavior of variance-based acquisition functions. We demonstrate that while such warps can be trained via marginal likelihood, a novel self-supervised objective yields substantially better performance. Our approach improves sample efficiency across a range of active learning benchmarks, particularly in regimes where non-stationarity challenges traditional methods.
@article{jarl2026observation,title={Observation-dependent Bayesian active learning via input-warped Gaussian processes},author={Jarl, Sanna and B{\aa}nkestad, Maria and Scragg, Jonathan J. S. and Sj{\"o}lund, Jens},journal={arXiv preprint arXiv:2602.01898},year={2026},}
Boundary Variance Inflation Causes Acquisition Bias in Gaussian Processes
Gaussian processes with stationary kernels on bounded domains exhibit inflated posterior variance near the boundary. Despite being a long-recognized artifact in geostatistics and a source of over-exploration in Bayesian optimization, the causes and effects of boundary-induced acquisition bias are underexplored. We trace the root cause to a simple geometric mechanism: the truncation of the kernel correlation neighborhood at the domain boundary creates an observation-independent distortion that worsens with dimensionality. We show how this distortion manifests across three acquisition classes: variance maximization concentrates selections at the corners, whereas negative integrated posterior variance and expected predictive information gain move selections inward to axis-aligned interior shells. These patterns arise without reference to any objective function, meaning that acquisition behavior can be dominated by kernel geometry rather than the desired task-specific uncertainty. To quantify this, we introduce a function-free selection-profile diagnostic for arbitrary acquisitions, kernels, and bounded-domain geometries.
@article{bankestad2026boundary,title={Boundary Variance Inflation Causes Acquisition Bias in Gaussian Processes},author={B{\aa}nkestad, Maria and Jarl, Sanna and Sj{\"o}lund, Jens},journal={arXiv preprint arXiv:2606.07561},year={2026},}
2023
TMLR
Variational Elliptical Processes
Maria Bånkestad, Jens Sjölund, Jalil Taghia, and 1 more author
We present elliptical processes—a family of non-parametric probabilistic models that subsumes Gaussian processes and Student’s t processes. This generalization includes a range of new heavy-tailed behaviors while retaining computational tractability. Elliptical processes are based on a representation of elliptical distributions as a continuous mixture of Gaussian distributions. We parameterize this mixture distribution as a spline normalizing flow, which we train using variational inference. The proposed form of the variational posterior enables a sparse variational elliptical process applicable to large-scale problems. We highlight advantages compared to Gaussian processes through regression and classification experiments. Elliptical processes can supersede Gaussian processes in several settings, including cases where the likelihood is non-Gaussian or when accurate tail modeling is essential.
@article{bankestad2023variational,title={Variational Elliptical Processes},author={B{\aa}nkestad, Maria and Sj{\"o}lund, Jens and Taghia, Jalil and Sch{\"o}n, Thomas B.},journal={Transactions on Machine Learning Research},issn={2835-8856},year={2023},url={https://openreview.net/forum?id=djN3TaqbdA},}
