KullbackLeibler¶

class odl.solvers.functional.default_functionals.KullbackLeibler(*args, **kwargs)[source]¶

Bases: Functional

The Kullback-Leibler divergence functional.

See also

KullbackLeiblerConvexConj: the convex conjugate functional
KullbackLeiblerCrossEntropy: related functional

Notes

The functional $F$ with prior $g>=0$ is given by:

$F(x) = \begin{cases} \int \left( x(t) - g(t) + g(t) \log \left( \frac{g(t)}{x(t)} \right) \right) dt & \text{if } x(t) > 0 \; \forall t \\ +\infty & \text{else.} \end{cases}$

Note that we use the common definition 0 log(0) := 0. KL based objectives are common in MLEM optimization problems and are often used as data-matching term when data noise governed by a multivariate Poisson probability distribution is significant.

The functional is related to the Kullback-Leibler cross entropy functional KullbackLeiblerCrossEntropy. The KL cross entropy is the one diescribed in this Wikipedia article, and the functional $F$ is obtained by switching place of the prior and the varialbe in the KL cross entropy functional.

For a theoretical exposition, see Csiszar1991.

References

Attributes:

adjoint: Adjoint of this operator (abstract).
convex_conj: The convex conjugate functional of the KL-functional.
domain: Set of objects on which this operator can be evaluated.
grad_lipschitz: Lipschitz constant for the gradient of the functional.
gradient: Gradient of the KL functional.
inverse: Return the operator inverse.
is_functional: True if this operator's range is a Field.
is_linear: True if this operator is linear.
prior: The prior in the Kullback-Leibler functional.
proximal: Return the proximal factory of the functional.
range: Set in which the result of an evaluation of this operator lies.

Methods

`__call__`(x[, out])	Return `self(x[, out, **kwargs])`.
`bregman`(point, subgrad)	Return the Bregman distance functional.
`derivative`(point)	Return the derivative operator in the given point.
`norm`([estimate])	Return the operator norm of this operator.
`translated`(shift)	Return a translation of the functional.

__init__(space, prior=None)[source]¶

Initialize a new instance.

Parameters:

spaceDiscretizedSpace or TensorSpace: Domain of the functional.
priorspace element-like, optional: Depending on the context, the prior, target or data distribution. It is assumed to be nonnegative. Default: if None it is take as the one-element.

Examples

Test that KullbackLeibler(x,x) = 0

>>> space = odl.rn(3)
>>> prior = 3 * space.one()
>>> func = odl.solvers.KullbackLeibler(space, prior=prior)
>>> func(prior)
0.0

Test that zeros in the prior are handled correctly

>>> prior = space.zero()
>>> func = odl.solvers.KullbackLeibler(space, prior=prior)
>>> x = space.one()
>>> func(x)
3.0