Surrogate optimisation minimises an expensive black-box objective by fitting a cheap probabilistic surrogate — typically a Gaussian process — and using an acquisition function to choose where to sample next. Each evaluation refines the surrogate, balancing exploitation of promising regions against exploration of uncertain ones.
$$ x^* = \arg\min_x f(x) $$
1. Fit Surrogate
Fit a Gaussian process to the sampled points, giving:
$$ \hat{\mu}(x), \qquad \hat{\sigma}(x) $$
2. Evaluate Acquisition Function
Score every candidate in the design space $\mathcal{X}$:
$$ \alpha(x), \qquad x \in \mathcal{X} $$
3. Select Next Sample
The point that optimises the acquisition (max or min, per the table below):
$$ x^{(k+1)} = \arg\max_x \alpha(x) $$
4. Sample Objective
$$ y^{(k+1)} = f\left(x^{(k+1)}\right) $$
5. Augment and Repeat
Add the new pair $\left(x^{(k+1)}, y^{(k+1)}\right)$ to the dataset and return to step 1.