Search space

The Search space will merge the Loss Function and defined Variables. Search space will alows to draw random points and attributes. Search space features can be extended by grafting Addons to it.

Abstract Class

class Searchspace(values, loss, **kwargs)[source]

Bases: abc.ABC

Searchspace is an essential class for Zellij. Define your search space with this object.

values

Determines the decision space. See MixedSearchspace, ContinuousSearchspace, DiscreteSearchspace for more info.

Type: Variable

loss

Callable of type LossFunc. See Loss Function for more information. loss will be used by the Search space object and by optimization

Type: LossFunc

See also

LossFunc: Parent class for a loss function.

random_attribute(size=1, replace=True, exclude=None)[source]

Draw random features from the search space.

Parameters

size (int, default=1) – Select randomly <size> features.
replace (boolean, default=True) – Select randomly <size> features with replacement if True. See numpy.random.choice
exclude (Variable or list[Variable] or type or list[type] or int or list[int], default=None) – Exclude one or several Variable to be drawn. Can also exclude types.For example one can exclude all Constant type. Can also exclude variables according to their index or value.

Returns

out – Array of index, corresponding to the selected Variable in values.

Return type

numpy.array(dtype=int)

Examples

>>> rand_att = sp.random_attribute(3)
>>> print(rand_att)
array([FloatVar(float_1, [2;12]), CatVar(cat_1, ['Hello', 87, 2.56]),
 FloatVar(float_1, [2;12])], dtype=object)

random_point(size=1)[source]

Return a random point from the search space

Parameters: size (int, default=1) – Draw <size> points.
Returns: points – List of <point>.
Return type: list[list[{int, float, str}]]

Examples

>>> rand_pts = sp.random_point(3)
>>> print(f"Random points: {rand_pts}")
Random points: [[-3.830114043118622, 9, 'sigmoid'],
...             [3.065902630698311, 3, 'sigmoid'],
...             [-0.6839762230289024, 10, 'relu']]

subspace(self, lo_bounds, up_bounds)[source]

Build a sub space according to the actual Searchspace using two vectors containing lower and upper bounds of the subspace. Can change type to Constant if necessary

Parameters

lo_bounds (list) – Lower bounds of the subspace. See Variable for more info.
up_bounds (boolean, default=False) – Upper bounds of the subspace. See Variable for more info.

Returns

out – Return a subspace of the actual Searchspace.

Return type

Searchspace

Examples

Concrete class

class Searchspace(values, loss, **kwargs)[source]

Bases: abc.ABC

Searchspace is an essential class for Zellij. Define your search space with this object.

values

Determines the decision space. See MixedSearchspace, ContinuousSearchspace, DiscreteSearchspace for more info.

Type: Variable

loss

Callable of type LossFunc. See Loss Function for more information. loss will be used by the Search space object and by optimization

Type: LossFunc

See also

LossFunc: Parent class for a loss function.

random_attribute(size=1, replace=True, exclude=None)[source]

Draw random features from the search space.

Parameters

size (int, default=1) – Select randomly <size> features.
replace (boolean, default=True) – Select randomly <size> features with replacement if True. See numpy.random.choice
exclude (Variable or list[Variable] or type or list[type] or int or list[int], default=None) – Exclude one or several Variable to be drawn. Can also exclude types.For example one can exclude all Constant type. Can also exclude variables according to their index or value.

Returns

out – Array of index, corresponding to the selected Variable in values.

Return type

numpy.array(dtype=int)

Examples

>>> rand_att = sp.random_attribute(3)
>>> print(rand_att)
array([FloatVar(float_1, [2;12]), CatVar(cat_1, ['Hello', 87, 2.56]),
 FloatVar(float_1, [2;12])], dtype=object)

random_point(size=1)[source]

Return a random point from the search space

Parameters: size (int, default=1) – Draw <size> points.
Returns: points – List of <point>.
Return type: list[list[{int, float, str}]]

Examples

>>> rand_pts = sp.random_point(3)
>>> print(f"Random points: {rand_pts}")
Random points: [[-3.830114043118622, 9, 'sigmoid'],
...             [3.065902630698311, 3, 'sigmoid'],
...             [-0.6839762230289024, 10, 'relu']]

subspace(self, lo_bounds, up_bounds)[source]

Build a sub space according to the actual Searchspace using two vectors containing lower and upper bounds of the subspace. Can change type to Constant if necessary

Parameters

lo_bounds (list) – Lower bounds of the subspace. See Variable for more info.
up_bounds (boolean, default=False) – Upper bounds of the subspace. See Variable for more info.

Returns

out – Return a subspace of the actual Searchspace.

Return type

Searchspace

Examples

class MixedSearchspace(values, loss, **kwargs)[source]

Bases: zellij.core.search_space.Searchspace

MixedSearchspace is a search space made for HyperParameter Optimization (HPO). The decision space can be made of various Variable types.

values

Determines the bounds of the search space. For ContinuousSearchspace the values must be an ArrayVar of FloatVar, IntVar, CatVar. The Search space will then manipulate this array.

Type: ArrayVar

loss

Callable of type LossFunc. See Loss Function for more information. loss will be used by the Search space object and by optimization

Type: LossFunc

random_attribute(self, size=1, replace=True, exclude=None): Draw random features from the search space. Return the selected Variable

random_point(self, size=1): Return random points from the search space

subspace(self, lo_bounds, up_bounds): Build a sub space according to the actual Searchspace using two vectors containing lower and upper bounds of the subspace.

See also

LossFunc: Parent class for a loss function.

Examples

>>> from zellij.core.variables import ArrayVar, IntVar, FloatVar, CatVar
>>> from zellij.utils.distances import Mixed
>>> from zellij.core.loss_func import Loss
>>> from zellij.utils.benchmark import himmelblau
>>> from zellij.core.search_space import MixedSearchspace
>>> a = ArrayVar(IntVar("int_1", 0,8),
>>>              IntVar("int_2", 4,45),
>>>              FloatVar("float_1", 2,12),
>>>              CatVar("cat_1", ["Hello", 87, 2.56]))
>>> lf = Loss()(himmelblau)
>>> sp = MixedSearchspace(a,lf, distance=Mixed())
>>> p1,p2 = sp.random_point(), sp.random_point()
>>> print(p1)
[5, 34, 4.8808143412719485, 87]
>>> print(p2)
[3, 42, 2.8196595134477738, 'Hello']

class ContinuousSearchspace(values, loss, **kwargs)[source]

Bases: zellij.core.search_space.Searchspace

ContinuousSearchspace is a search space made for continuous optimization. The decision space is made of FloatVar.

values

Determines the bounds of the search space. For ContinuousSearchspace the values must be an ArrayVar of FloatVar. The Search space will then manipulate this array.

Type: ArrayVar

loss

Callable of type LossFunc. See Loss Function for more information. loss will be used by the Search space object and by optimization

Type: LossFunc

random_attribute(self, size=1, replace=True, exclude=None): Draw random features from the search space. Return the selected Variable

random_point(self, size=1): Return random points from the search space

subspace(self, lo_bounds, up_bounds): Build a sub space according to the actual Searchspace using two vectors containing lower and upper bounds of the subspace.

See also

LossFunc: Parent class for a loss function.

Examples

>>> from zellij.core.variables import ArrayVar, FloatVar
>>> from zellij.core.loss_func import Loss
>>> from zellij.utils.benchmark import himmelblau
>>> from zellij.core.search_space import ContinuousSearchspace
>>> lf = Loss()(himmelblau)
>>> a = ArrayVar(FloatVar("float_1", 0,1),
...              FloatVar("float_2", 0,1))
>>> sp = ContinuousSearchspace(a,lf)
>>> p1,p2 = sp.random_point(), sp.random_point()
>>> print(p1)
[0.8922761649920034, 0.12709277668616326]
>>> print(p2)
[0.7730279148456985, 0.14715728189857524]

class DiscreteSearchspace(values, loss, **kwargs)[source]

Bases: zellij.core.search_space.Searchspace

DiscreteSearchspace is a search space made for continuous optimization. The decision space is made of IntVar.

values

Determines the bounds of the search space. For DiscreteSearchspace the values must be an ArrayVar of IntVar. The Search space will then manipulate this array.

Type: ArrayVar

loss

Callable of type LossFunc. See Loss Function for more information. loss will be used by the Search space object and by optimization

Type: LossFunc

random_attribute(self, size=1, replace=True, exclude=None): Draw random features from the search space. Return the selected Variable

random_point(self, size=1): Return random points from the search space

subspace(self, lo_bounds, up_bounds): Build a sub space according to the actual Searchspace using two vectors containing lower and upper bounds of the subspace.

See also

LossFunc: Parent class for a loss function.

Examples

>>> from zellij.core.variables import ArrayVar, IntVar
>>> from zellij.core.loss_func import Loss
>>> from zellij.utils.benchmark import himmelblau
>>> from zellij.core.search_space import DiscreteSearchspace
>>> a = ArrayVar(IntVar("int_1", 0,8),
>>>              IntVar("int_2", 4,45))
>>> lf = Loss()(himmelblau)
>>> sp = DiscreteSearchspace(a,lf)
>>> p1,p2 = sp.random_point(), sp.random_point()
>>> print(p1)
[5, 34]
>>> print(p2)
[3, 42]

Search space

Contents

Search space

Abstract Class

Concrete class