GRUCell¶
-
class
GRUCell(hidden_size, i2h_weight_initializer=None, h2h_weight_initializer=None, i2h_bias_initializer='zeros', h2h_bias_initializer='zeros', input_size=0, prefix=None, params=None)¶ Gated Rectified Unit (GRU) network cell. Note: this is an implementation of the cuDNN version of GRUs (slight modification compared to Cho et al. 2014; the reset gate \(r_t\) is applied after matrix multiplication).
Each call computes the following function:
(21)¶\[\begin{split}\begin{array}{ll} r_t = sigmoid(W_{ir} x_t + b_{ir} + W_{hr} h_{(t-1)} + b_{hr}) \\ i_t = sigmoid(W_{ii} x_t + b_{ii} + W_{hi} h_{(t-1)} + b_{hi}) \\ n_t = \tanh(W_{in} x_t + b_{in} + r_t * (W_{hn} h_{(t-1)} + b_{hn})) \\ h_t = (1 - i_t) * n_t + i_t * h_{(t-1)} \\ \end{array}\end{split}\]where \(h_t\) is the hidden state at time t, \(x_t\) is the hidden state of the previous layer at time t or \(input_t\) for the first layer, and \(r_t\), \(i_t\), \(n_t\) are the reset, input, and new gates, respectively.
Parameters: - hidden_size : int
Number of units in output symbol.
- i2h_weight_initializer : str or Initializer
Initializer for the input weights matrix, used for the linear transformation of the inputs.
- h2h_weight_initializer : str or Initializer
Initializer for the recurrent weights matrix, used for the linear transformation of the recurrent state.
- i2h_bias_initializer : str or Initializer, default ‘zeros’
Initializer for the bias vector.
- h2h_bias_initializer : str or Initializer, default ‘zeros’
Initializer for the bias vector.
- prefix : str, default
'gru_' prefix for name of Block`s (and name of weight if params is `None).
- params : Parameter or None, default None
Container for weight sharing between cells. Created if None.
- Inputs:
- data: input tensor with shape (batch_size, input_size).
- states: a list of one initial recurrent state tensor with shape (batch_size, num_hidden).
- Outputs:
- out: output tensor with shape (batch_size, num_hidden).
- next_states: a list of one output recurrent state tensor with the same shape as states.