pyxu.operator.func#

Norms & Loss Functions#

class L1Norm(dim_shape)[source]#

Bases: ProxFunc

\(\ell_{1}\)-norm, \(\Vert\mathbf{x}\Vert_{1} := \sum_{i} |x_{i}|\).

Parameters:

dim_shape (Integral | tuple[Integral, ...])

class L2Norm(dim_shape)[source]#

Bases: ProxFunc

\(\ell_{2}\)-norm, \(\Vert\mathbf{x}\Vert_{2} := \sqrt{\sum_{i} |x_{i}|^{2}}\).

Parameters:

dim_shape (Integral | tuple[Integral, ...])

class SquaredL2Norm(dim_shape)[source]#

Bases: QuadraticFunc

\(\ell^{2}_{2}\)-norm, \(\Vert\mathbf{x}\Vert^{2}_{2} := \sum_{i} |x_{i}|^{2}\).

Parameters:

dim_shape (Integral | tuple[Integral, ...])

class SquaredL1Norm(dim_shape)[source]#

Bases: ProxFunc

\(\ell^{2}_{1}\)-norm, \(\Vert\mathbf{x}\Vert^{2}_{1} := (\sum_{i} |x_{i}|)^{2}\).

Note

  • Computing prox() is unavailable with DASK inputs. (Inefficient exact solution at scale.)

Parameters:

dim_shape (Integral | tuple[Integral, ...])

__init__(dim_shape)[source]#
Parameters:

  • dim_shape (NDArrayShape) – (M1,…,MD) operator input shape.

  • codim_shape (NDArrayShape) – (N1,…,NK) operator output shape.

prox(arr, tau)[source]#

Evaluate proximity operator of \(\tau f\) at specified point(s).

Parameters:

  • arr (NDArray) – (…, M1,…,MD) input points.

  • tau (Real) – Positive scale factor.

Returns:

out – (…, M1,…,MD) proximal evaluations.

Return type:

NDArray

Notes

For \(\tau >0\), the proximity operator of a scaled functional \(f: \mathbb{R}^{M_{1} \times\cdots\times M_{D}} \to \mathbb{R}\) is defined as:

\[\mathbf{\text{prox}}_{\tau f}(\mathbf{z}) := \arg\min_{\mathbf{x}\in\mathbb{R}^{M_{1} \times\cdots\times M_{D}}} f(x)+\frac{1}{2\tau} \|\mathbf{x}-\mathbf{z}\|_{2}^{2}, \quad \forall \mathbf{z} \in \mathbb{R}^{M_{1} \times\cdots\times M_{D}}.\]
class KLDivergence(data)[source]#

Bases: ProxFunc

Generalised Kullback-Leibler divergence \(D_{KL}(\mathbf{y}||\mathbf{x}) := \sum_{i} y_{i} \log(y_{i} / x_{i}) - y_{i} + x_{i}\).

Parameters:

data (NDArray)

__init__(data)[source]#
Parameters:

data (NDArray) – (M1,…,MD) non-negative input data.

Examples

import numpy as np
from pyxu.operator import KLDivergence

y = np.arange(5)
loss = KLDivergence(y)

loss(2 * y)  # [3.06852819]
np.round(loss.prox(2 * y, tau=1))  # [0. 2. 4. 6. 8.]

Notes

  • When \(\mathbf{y}\) and \(\mathbf{x}\) sum to one, and hence can be interpreted as discrete probability distributions, the KL-divergence corresponds to the relative entropy of \(\mathbf{y}\) w.r.t. \(\mathbf{x}\), i.e. the amount of information lost when using \(\mathbf{x}\) to approximate \(\mathbf{y}\). It is particularly useful in the context of count data with Poisson distribution; the KL-divergence then corresponds (up to an additive constant) to the likelihood of \(\mathbf{y}\) where each component is independent with Poisson distribution and respective intensities given by \(\mathbf{x}\). See [FuncSphere] Chapter 7, Section 5 for the computation of its proximal operator.

  • KLDivergence is not backend-agnostic: inputs to arithmetic methods must have the same backend as data.

  • If data is a DASK array, it’s entries are assumed non-negative de-facto. Reason: the operator should be quick to build under all circumstances, and this is not guaranteed if we have to check that all entries are positive for out-of-core arrays.

  • If data is a DASK array, the core-dimensions of arrays supplied to arithmetic methods must have the same chunk-size as data.

class LInfinityNorm(dim_shape)[source]#

Bases: ProxFunc

\(\ell_{\infty}\)-norm, \(\Vert\mathbf{x}\Vert_{\infty} := \max_{i} |x_{i}|\).

Note

  • Computing prox() is unavailable with DASK inputs. (Inefficient exact solution at scale.)

Parameters:

dim_shape (Integral | tuple[Integral, ...])

class L21Norm(dim_shape, l2_axis=(0,))[source]#

Bases: ProxFunc

Mixed \(\ell_{2}-\ell_{1}\) norm, \(\Vert\mathbf{x}\Vert_{2, 1} := \sum_{i} \sqrt{\sum_{j} x_{i, j}^{2}}\).

Parameters:
__init__(dim_shape, l2_axis=(0,))[source]#
Parameters:
class PositiveL1Norm(dim_shape)[source]#

Bases: ProxFunc

\(\ell_{1}\)-norm, with a positivity constraint.

\[f(\mathbf{x}) := \lVert\mathbf{x}\rVert_{1} + \iota_{+}(\mathbf{x}),\]
\[\textbf{prox}_{\tau f}(\mathbf{z}) := \max(\mathrm{soft}_\tau(\mathbf{z}), \mathbf{0})\]

See also

PositiveOrthant

Parameters:

dim_shape (Integral | tuple[Integral, ...])

Indicator Functions#

L1Ball(dim_shape, radius=1)[source]#

Indicator function of the \(\ell_{1}\)-ball.

\[\begin{split}\iota_{1}^{r}(\mathbf{x}) := \begin{cases} 0 & \|\mathbf{x}\|_{1} \le r \\ \infty & \text{otherwise}. \end{cases}\end{split}\]
\[\text{prox}_{\tau\, \iota_{1}^{r}}(\mathbf{x}) := \mathbf{x} - \text{prox}_{r\, \ell_{\infty}}(\mathbf{x})\]
Parameters:
Returns:

op

Return type:

OpT

Note

  • Computing prox() is unavailable with DASK inputs. (Inefficient exact solution at scale.)

L2Ball(dim_shape, radius=1)[source]#

Indicator function of the \(\ell_{2}\)-ball.

\[\begin{split}\iota_{2}^{r}(\mathbf{x}) := \begin{cases} 0 & \|\mathbf{x}\|_{2} \le r \\ \infty & \text{otherwise}. \end{cases}\end{split}\]
\[\text{prox}_{\tau\, \iota_{2}^{r}}(\mathbf{x}) := \mathbf{x} - \text{prox}_{r\, \ell_{2}}(\mathbf{x})\]
Parameters:
Returns:

op

Return type:

OpT

LInfinityBall(dim_shape, radius=1)[source]#

Indicator function of the \(\ell_{\infty}\)-ball.

\[\begin{split}\iota_{\infty}^{r}(\mathbf{x}) := \begin{cases} 0 & \|\mathbf{x}\|_{\infty} \le r \\ \infty & \text{otherwise}. \end{cases}\end{split}\]
\[\text{prox}_{\tau\, \iota_{\infty}^{r}}(\mathbf{x}) := \mathbf{x} - \text{prox}_{r\, \ell_{1}}(\mathbf{x})\]
Parameters:
Returns:

op

Return type:

OpT

class PositiveOrthant(dim_shape)[source]#

Bases: _IndicatorFunction

Indicator function of the positive orthant.

\[\begin{split}\iota_{+}(\mathbf{x}) := \begin{cases} 0 & \min{\mathbf{x}} \ge 0,\\ \infty & \text{otherwise}. \end{cases}\end{split}\]
\[\text{prox}_{\tau\, \iota_{+}}(\mathbf{x}) := \max(\mathbf{x}, \mathbf{0})\]
Parameters:

dim_shape (Integral | tuple[Integral, ...])

class HyperSlab(a, lb, ub)[source]#

Bases: _IndicatorFunction

Indicator function of a hyperslab.

\[\begin{split}\iota_{\mathbf{a}}^{l,u}(\mathbf{x}) := \begin{cases} 0 & l \le \langle \mathbf{a}, \mathbf{x} \rangle \le u \\ \infty & \text{otherwise}. \end{cases}\end{split}\]
\[\begin{split}\text{prox}_{\tau\, \iota_{\mathbf{a}}^{l,u}}(\mathbf{x}) := \begin{cases} \mathbf{x} + \frac{l - \langle \mathbf{a}, \mathbf{x} \rangle}{\|\mathbf{a}\|^{2}} \mathbf{a} & \langle \mathbf{a}, \mathbf{x} \rangle < l, \\ \mathbf{x} + \frac{u - \langle \mathbf{a}, \mathbf{x} \rangle}{\|\mathbf{a}\|^{2}} \mathbf{a} & \langle \mathbf{a}, \mathbf{x} \rangle > u, \\ \mathbf{x} & \text{otherwise}. \end{cases}\end{split}\]
Parameters:
__init__(a, lb, ub)[source]#
Parameters:

  • A (LinFunc) – Linear functional with domain (M1,…,MD).

  • lb (Real) – Lower bound.

  • ub (Real) – Upper bound.

  • a (LinFunc)

class RangeSet(A)[source]#

Bases: _IndicatorFunction

Indicator function of a range set.

\[\begin{split}\iota_{\mathbf{A}}^{R}(\mathbf{x}) := \begin{cases} 0 & \mathbf{x} \in \text{span}(\mathbf{A}) \\ \infty & \text{otherwise}. \end{cases}\end{split}\]
\[\text{prox}_{\tau\, \iota_{\mathbf{A}}^{R}}(\mathbf{x}) := \mathbf{A} (\mathbf{A}^{T} \mathbf{A})^{-1} \mathbf{A}^{T} \mathbf{x}.\]
Parameters:

A (LinOp)

__init__(A)[source]#
Parameters: