When an experiment or a simulation is expensive, the model’s job is not just to predict. It also has to say how certain it is, and where to look next. That puts uncertainty at the center: a design policy is only as good as the uncertainty estimates it relies on. This project develops probabilistic models whose uncertainty is trustworthy enough to steer real decisions.
Gaussian processes are the default tool for this, and they have a surprising blind spot. Their posterior variance depends on the observed outputs only through the hyperparameters, so exploration is largely insensitive to what the measurements actually were. We address this by warping the input space with a learned, monotone transformation, which lets the acquisition policy stretch or compress regions in response to observed variability (Jarl et al., 2026).
A learned, monotone warp reshapes the input geometry so that the same design choice, seen in the warped space, responds to where the function actually varies.
In related work, we traced a second issue: stationary kernels inflate the posterior variance near the boundary of the domain, which quietly biases acquisition toward corners and edges regardless of the objective. We characterize this effect and give a diagnostic for measuring it (Bånkestad et al., 2026).
Acquisition surfaces for a single design (left) and the density of selected points over 10,000 designs (right), for three acquisition functions. The selections concentrate at the corners and edges of the domain, driven by kernel geometry rather than the objective.
On the modeling side, elliptical processes generalize Gaussian and Student’s t processes into a single family with flexible heavy-tailed behavior, trained with variational inference (Bånkestad et al., 2023). They are useful when the likelihood is non-Gaussian, or when getting the tails right actually matters.
Bayesian active learning relies on the precise quantification of predictive uncertainty to explore unknown function landscapes. While Gaussian process surrogates are the standard for such tasks, an underappreciated fact is that their posterior variance depends on the observed outputs only through the hyperparameters, rendering exploration largely insensitive to the actual measurements. We propose to inject observation-dependent feedback by warping the input space with a learned, monotone reparameterization. This mechanism allows the design policy to expand or compress regions of the input space in response to observed variability, thereby shaping the behavior of variance-based acquisition functions. We demonstrate that while such warps can be trained via marginal likelihood, a novel self-supervised objective yields substantially better performance. Our approach improves sample efficiency across a range of active learning benchmarks, particularly in regimes where non-stationarity challenges traditional methods.
@article{jarl2026observation,title={Observation-dependent Bayesian active learning via input-warped Gaussian processes},author={Jarl, Sanna and B{\aa}nkestad, Maria and Scragg, Jonathan J. S. and Sj{\"o}lund, Jens},journal={arXiv preprint arXiv:2602.01898},year={2026},}
Boundary Variance Inflation Causes Acquisition Bias in Gaussian Processes
Gaussian processes with stationary kernels on bounded domains exhibit inflated posterior variance near the boundary. Despite being a long-recognized artifact in geostatistics and a source of over-exploration in Bayesian optimization, the causes and effects of boundary-induced acquisition bias are underexplored. We trace the root cause to a simple geometric mechanism: the truncation of the kernel correlation neighborhood at the domain boundary creates an observation-independent distortion that worsens with dimensionality. We show how this distortion manifests across three acquisition classes: variance maximization concentrates selections at the corners, whereas negative integrated posterior variance and expected predictive information gain move selections inward to axis-aligned interior shells. These patterns arise without reference to any objective function, meaning that acquisition behavior can be dominated by kernel geometry rather than the desired task-specific uncertainty. To quantify this, we introduce a function-free selection-profile diagnostic for arbitrary acquisitions, kernels, and bounded-domain geometries.
@article{bankestad2026boundary,title={Boundary Variance Inflation Causes Acquisition Bias in Gaussian Processes},author={B{\aa}nkestad, Maria and Jarl, Sanna and Sj{\"o}lund, Jens},journal={arXiv preprint arXiv:2606.07561},year={2026},}
2023
TMLR
Variational Elliptical Processes
Maria Bånkestad, Jens Sjölund, Jalil Taghia, and 1 more author
We present elliptical processes—a family of non-parametric probabilistic models that subsumes Gaussian processes and Student’s t processes. This generalization includes a range of new heavy-tailed behaviors while retaining computational tractability. Elliptical processes are based on a representation of elliptical distributions as a continuous mixture of Gaussian distributions. We parameterize this mixture distribution as a spline normalizing flow, which we train using variational inference. The proposed form of the variational posterior enables a sparse variational elliptical process applicable to large-scale problems. We highlight advantages compared to Gaussian processes through regression and classification experiments. Elliptical processes can supersede Gaussian processes in several settings, including cases where the likelihood is non-Gaussian or when accurate tail modeling is essential.
@article{bankestad2023variational,title={Variational Elliptical Processes},author={B{\aa}nkestad, Maria and Sj{\"o}lund, Jens and Taghia, Jalil and Sch{\"o}n, Thomas B.},journal={Transactions on Machine Learning Research},issn={2835-8856},year={2023},url={https://openreview.net/forum?id=djN3TaqbdA},}
