inox.nn.dropout#

Dropout layers

Classes#

`Dropout`	Creates a dropout layer.
`TrainingDropout`	Creates a training-bound dropout layer.

Descriptions#

class inox.nn.dropout.Dropout(p=0.5)#

Creates a dropout layer.

\[y = \frac{m \odot x}{1 - p}\]

where the binary mask \(m\) is drawn from a Bernoulli distribution such that \(P(m_i = 0) = p\). This has proven to be an effective technique for regularization and preventing overfitting.

References

A Simple Way to Prevent Neural Networks from Overfitting (Srivastava et al., 2014)

https://jmlr.org/papers/v15/srivastava14a.html

Parameters:: p (float | Array) – The dropout rate \(p \in [0, 1]\).

__call__(x, key)#

Parameters:

x (Array) – The input tensor \(x\), with shape \((*)\).
key (Array) – A PRNG key.

Returns:

The output tensor \(y\), with shape \((*)\).

Return type:

Array

class inox.nn.dropout.TrainingDropout(p=0.5)#

Creates a training-bound dropout layer.

When self.training = False,

\[y = x\]