VariationalDropoutCell

class VariationalDropoutCell(base_cell, drop_inputs=0.0, drop_states=0.0, drop_outputs=0.0)

Applies Variational Dropout on base cell. https://arxiv.org/pdf/1512.05287.pdf

Variational dropout uses the same dropout mask across time-steps. It can be applied to RNN inputs, outputs, and states. The masks for them are not shared.

The dropout mask is initialized when stepping forward for the first time and will remain the same until .reset() is called. Thus, if using the cell and stepping manually without calling .unroll(), the .reset() should be called after each sequence.

Parameters:

base_cell : RecurrentCell

The cell on which to perform variational dropout.

drop_inputs : float, default 0.

The dropout rate for inputs. Won’t apply dropout if it equals 0.

drop_states : float, default 0.

The dropout rate for state inputs on the first state channel. Won’t apply dropout if it equals 0.

drop_outputs : float, default 0.

The dropout rate for outputs. Won’t apply dropout if it equals 0.