Loss Function

Abstract Class

Loss(model=None, save=False, verbose=True, MPI=False, only_score=False, kwargs_mode=False)[source]

Wrap a function of type $f(x)=y$ . See LossFunc for more info.

Parameters

model (function, default=None) – Function of type f(x)=y. x must be a solution. A solution can be a list of float, int… It can also be of mixed types, containing, strings, float, int…
objective (Objective, default=Minimizer) – Objectve object determines what and and how to optimize. (minimization, maximization, ratio…)
save (string, optional) – Filename where to save the best found model. Only one model is saved for memory issues.
MPI (boolean, optional) – Wrap the function with MPILoss if True, with SerialLoss else.
only_score (boolean, optional) – If a save is not False, then if True, only the objective values will be saved.
kwargs_mode (boolean, optional) – If True, then points will be passed as kwargs to the model. Keys will be the labels, if they are of the same size as the point.

Returns

wrapper – Wrapped original function

Return type

LossFunc

Examples

>>> import numpy as np
>>> from zellij.core.loss_func import Loss
>>> @Loss(save=False, verbose=True)
... def himmelblau(x):
...   x_ar = np.array(x)
...   return np.sum(x_ar**4 -16*x_ar**2 + 5*x_ar) * (1/len(x_ar))
>>> print(f"Best solution found: f({himmelblau.best_point}) = {himmelblau.best_score}")
Best solution found: f(None) = inf
>>> print(f"Number of evaluations:{himmelblau.calls}")
Number of evaluations:0
>>> print(f"All evaluated solutions:{himmelblau.all_solutions}")
All evaluated solutions:[]
>>> print(f"All loss values:{himmelblau.all_scores}")
All loss values:[]

Main wrapper

class LossFunc(model, objective=<class 'zellij.core.objective.Minimizer'>, historic=True, save=False, verbose=True, only_score=False, kwargs_mode=False)[source]

Bases: abc.ABC

LossFunc allows to wrap function of type $f(x)=y$ . With $x$ a set of hyperparameters. However, Zellij supports alternative pattern: $f(x)=results,model$ for example. Where:

$results$ can be a list or a dictionary. Be default the first element of the list or the dictionary is considered as the loss vale.

$model$ is optionnal, it is an object with a save() method. (e.g. a neural network from Tensorflow)

You must wrap your function so it can be used in Zellij by adding several features, such as calls count, saves, parallelization, historic…

model

Function of type $f(x)=y$ or $f(x)=results,model. :math:$ must be a solution. A solution can be a list of float, int… It can also be of mixed types…

Type: function

objective

Objectve object determines what and and how to optimize. (minimization, maximization, ratio…)

Type: Objective, default=Minimizer

best_score

Best score found so far.

Type: float

best_point

Best solution found so far.

Type: list

best_argmin

Index of the best solution found so far.

Type: int

all_scores

Historic of all evaluated scores.

Type: float

all_solutions

Historic of all evaluated solutions.

Type: float

calls

Number of loss function calls

Type: int

See also

Loss: Wrapper function
MPILoss: Distributed version of LossFunc
SerialLoss: Basic version of LossFunc

build_bar(total)[source]

build_bar is a method to build a progress bar. It is a purely aesthetic feature to get info on the execution. You can deactivate it, with verbose=False.

Parameters: total (int) – Length of the progress bar.

close_bar()[source]: Delete the progress bar.

get_best(n_process=1, idx=None)[source]

reset()[source]: Reset all attributes of LossFunc at their initial values.

Concrete loss

class SerialLoss(model, objective=<class 'zellij.core.objective.Minimizer'>, historic=True, save=False, verbose=True, only_score=False, kwargs_mode=False)[source]

Bases: zellij.core.loss_func.LossFunc

SerialLoss adds methods to save and evaluate the original loss function.

__call__(X, filename='', \*\*kwargs)[source]: Evaluate a list X of solutions with the original loss function.

_save_model(score, source)[source]: See LossFunc, save a model according to its score and the worker rank.

See also

Loss: Wrapper function
LossFunc: Inherited class
MPILoss: Distributed version of LossFunc

class MPILoss(model, objective=<class 'zellij.core.objective.Minimizer'>, historic=True, save=False, verbose=True, only_score=False, kwargs_mode=False)[source]

Bases: zellij.core.loss_func.LossFunc

MPILoss adds method to dynamically distribute the evaluation of multiple solutions within a distributed environment, where a version of MPI is available.

comm

All created processes and their communication context are grouped in comm.

Type: MPI_COMM_WORLD

status

Data structure containing information about a received message.

Type: MPI_Status

rank

Process rank

Type: int

p

comm size

Type: int

master

If True the process is the master, else it is the worker.

Type: boolean

__call__(X, filename='', \*\*kwargs)[source]: Evaluate a list X of solutions with the original loss function.

worker()[source]: Initialize a worker.

stop()[source]: Stops all the workers and master.

_save_model(score, source)[source]: See LossFunc, save a model according to its score and the worker rank.

See also

Loss: Wrapper function
LossFunc: Inherited class
SerialLoss: Basic version of LossFunc

worker()[source]: Initialize worker. While it does not receive a stop message, a worker will wait for a solution to evaluate.

stop()[source]: Send a stop message to all workers.

Mock Model

class MockModel(outputs={'o1': <function MockModel.<lambda>>}, return_format='dict', return_model=True, verbose=True)[source]

Bases: object

This object allows to replace your real model with a costless object, by mimicking different available configurations in Zellij. ** Be carefull: This object does not replace any Loss wrapper**

Parameters

outputs (dict, default={“o1”,lambda *args, **kwargs: np.random.random()}) – Dictionnary containing outputs name (keys) and functions to execute to obtain outputs. Pass *args and **kwargs to these functions when calling this MockModel
verbose (bool) – If True print information when saving and __call___.
return_format (string) –
Output format. It can be:
- ”dict” -> {“o1”:value1,”o2”:value2,…}
- ”list” -> [value1,value2,…]
return_model (boolean) –
Return MockModel (self) or not. Return if:
- True -> (outputs, MockModel)
- False -> outputs

See also

Loss: Wrapper function
MPILoss: Distributed version of LossFunc
SerialLoss: Basic version of LossFunc

Examples

>>> from zellij.core.loss_func import MockModel, Loss
>>> mock = MockModel()
>>> print(mock("test", 1, 2.0, param1="Mock", param2=True))
I am Mock !
    ->*args: ('test', 1, 2.0)
    ->**kwargs: {'param1': 'Mock', 'param2': True}
({'o1': 0.3440051802032301},
<zellij.core.loss_func.MockModel at 0x7f5c8027a100>)
>>> loss = Loss(save=True, verbose=False)(mock)
>>> print(loss([["test", 1, 2.0, "Mock", True]], other_info="Hi !"))
I am Mock !
    ->*args: (['test', 1, 2.0, 'Mock', True],)
    ->**kwargs: {}
I am Mock !
    ->saving in MockModel_zlj_save/model/MockModel_best/i_am_mock.txt
[0.7762604280531996]

save(filepath)[source]

Objective

Abstract Class

class Objective(target=0)[source]

Bases: abc.ABC

This absract object allows to define what is the objective of the optimization process.

Parameters: target (int or str, default=0) – Which outputs of the loss function should it target. Default is 0, it will consider the value at index 0 in the outputs of the loss function. If output is a dict, it can target one of its key.

target

reset()[source]

class Minimizer(target=0)[source]

Bases: zellij.core.objective.Objective

Minimizer allows to minimize the given target. Do, $f(y)=y$ . With $y$ a given scores. /!By default Zellij metaheuristics minimize the loss value. So this object will just return the given scores.

Parameters: target (int or str, default=0) – Which outputs of the loss function should it target. Default is 0, it will consider the value at index 0 in the outputs of the loss function. If output is a dict, it can target one of its key.

target

class Maximizer(target=0)[source]

Bases: zellij.core.objective.Objective

Maximizer allows to maximize the given target. Do, $f(y)=-y$ . With $y$ a given scores. /!By default Zellij metaheuristics minimize the loss value. So this object will compute the negative of the given scores.

Parameters: target (int or str, default=0) – Which outputs of the loss function should it target. Default is 0, it will consider the value at index 0 in the outputs of the loss function. If output is a dict, it can target one of its key.

target

class Lambda(function, selector='min', target=0)[source]

Bases: zellij.core.objective.Objective

Lambda allows to transform the given target. Do, $f(y)=function(y)$ . With $y$ a given scores. /!By default Zellij metaheuristics minimize the loss value.

Parameters

function (Callable) – Function with len(target) parameters which return an objective value.
selector ({"min","max"}) – Minimize or maximize the results from function
target ({int,str,list[{int, str}]} default=0) – Which outputs of the loss function should it target. Default is 0, it will consider the value at index 0 in the outputs of the loss function. If output is a dict, it can target one of its key.

target

Concrete objective

class Minimizer(target=0)[source]

Bases: zellij.core.objective.Objective

Minimizer allows to minimize the given target. Do, $f(y)=y$ . With $y$ a given scores. /!By default Zellij metaheuristics minimize the loss value. So this object will just return the given scores.

Parameters: target (int or str, default=0) – Which outputs of the loss function should it target. Default is 0, it will consider the value at index 0 in the outputs of the loss function. If output is a dict, it can target one of its key.

target

class Maximizer(target=0)[source]

Bases: zellij.core.objective.Objective

Parameters: target (int or str, default=0) – Which outputs of the loss function should it target. Default is 0, it will consider the value at index 0 in the outputs of the loss function. If output is a dict, it can target one of its key.

target

class Lambda(function, selector='min', target=0)[source]

Bases: zellij.core.objective.Objective

Lambda allows to transform the given target. Do, $f(y)=function(y)$ . With $y$ a given scores. /!By default Zellij metaheuristics minimize the loss value.

Parameters

function (Callable) – Function with len(target) parameters which return an objective value.
selector ({"min","max"}) – Minimize or maximize the results from function
target ({int,str,list[{int, str}]} default=0) – Which outputs of the loss function should it target. Default is 0, it will consider the value at index 0 in the outputs of the loss function. If output is a dict, it can target one of its key.

target

Loss Function

Contents

Loss Function

Abstract Class

Main wrapper

Concrete loss

Mock Model

Objective

Abstract Class

Concrete objective