Bayesian Optimization

Bayesian Optimization

class Bayesian_optimization(search_space, f_calls, verbose=True, surrogate=<class 'botorch.models.gp_regression.SingleTaskGP'>, likelihood=<class 'gpytorch.mlls.exact_marginal_log_likelihood.ExactMarginalLogLikelihood'>, acquisition=<class 'botorch.acquisition.analytic.ExpectedImprovement'>, initial_size=10, gpu=False, **kwargs)[source]

Bases: zellij.core.metaheuristic.Metaheuristic

Bayesian optimization (BO) is a surrogate based optimization method which interpolates the actual loss function with a surrogate model, here it is a gaussian process. By sampling into this surrogate, BO determines promising points, which are worth to evaluate with the actual loss function. Once done, the gaussian process is updated using results obtained by evaluating these promising solutions with the loss function.

It is based on BoTorch and GPyTorch.

search_space

Search space object containing bounds of the search space

Type

Searchspace

f_calls

Maximum number of Loss Function calls

Type

int

verbose

If False, there will be no print and no progress bar.

Type

bool

surrogate

Gaussian Process Regressor object from ‘botorch’. Determines the surrogate model that Bayesian optimization will use to interpolate the loss function

Type

botorch.models.model.Model, default=SingleTaskGP

likelihood

gpytorch.mlls object it determines which MarginalLogLikelihood to use when optimizing kernel’s hyperparameters

Type

gpytorch.mlls, default=ExactMarginalLogLikelihood

acquisition

An acquisition function or infill criteria, determines how ‘promising’ a point sampled from the surrogate is.

Type

botorch.acquisition.acquisition.AcquisitionFunction, default = ExpectedImprovement

initial_size

Size of the initial set of solution to draw randomly.

Type

int, default=10

gpu

Use GPU if available

Type

bool, default=True

\*\*kwargs

Key word arguments linked to the surrogate and the acquisition function.

See also

Metaheuristic

Parent class defining what a Metaheuristic is

Loss Function

Describes what a loss function is in Zellij

Search space

Describes what a loss function is in Zellij

Examples

>>> from zellij.core import Loss
>>> from zellij.core import ContinuousSearchspace
>>> from zellij.core import FloatVar, ArrayVar
>>> from zellij.utils.benchmark import himmelblau
>>> from zellij.strategies.bayesian_optimization import Bayesian_optimization
>>> import botorch
>>> import gpytorch
...
>>> lf = Loss()(himmelblau)
>>> sp = ContinuousSearchspace(ArrayVar(FloatVar("a",-5,5), FloatVar("b",-5,5)),lf)
>>> bo = Bayesian_optimization(sp, 500,
...       acquisition=botorch.acquisition.monte_carlo.qExpectedImprovement,
...       q=5)
>>> bo.run()
run(n_process=1)[source]

Runs BO.

Parameters

  • H (Fractal, optional) – When used by Decomposition Based Algorithm, a fractal corresponding to the current subspace is given

  • n_process (int, default=1) – Determine the number of best solution found to return.

Returns

  • best_sol (list[float]) – Returns a list of the n_process best found points to the continuous format.

  • best_scores (list[float]) – Returns a list of the n_process best found scores associated to best_sol.