LSTMPCell

class LSTMPCell(hidden_size, projection_size, i2h_weight_initializer=None, h2h_weight_initializer=None, h2r_weight_initializer=None, i2h_bias_initializer='zeros', h2h_bias_initializer='zeros', input_size=0, prefix=None, params=None)

Long-Short Term Memory Projected (LSTMP) network cell. (https://arxiv.org/abs/1402.1128)

Each call computes the following function:

(31)\[\begin{split}\begin{array}{ll} i_t = sigmoid(W_{ii} x_t + b_{ii} + W_{ri} r_{(t-1)} + b_{ri}) \\ f_t = sigmoid(W_{if} x_t + b_{if} + W_{rf} r_{(t-1)} + b_{rf}) \\ g_t = \tanh(W_{ig} x_t + b_{ig} + W_{rc} r_{(t-1)} + b_{rg}) \\ o_t = sigmoid(W_{io} x_t + b_{io} + W_{ro} r_{(t-1)} + b_{ro}) \\ c_t = f_t * c_{(t-1)} + i_t * g_t \\ h_t = o_t * \tanh(c_t) \\ r_t = W_{hr} h_t \end{array}\end{split}\]

where \(r_t\) is the projected recurrent activation at time t, \(h_t\) is the hidden state at time t, \(c_t\) is the cell state at time t, \(x_t\) is the input at time t, and \(i_t\), \(f_t\), \(g_t\), \(o_t\) are the input, forget, cell, and out gates, respectively.

Parameters:

hidden_size : int

Number of units in cell state symbol.

projection_size : int

Number of units in output symbol.

i2h_weight_initializer : str or Initializer

Initializer for the input weights matrix, used for the linear transformation of the inputs.

h2h_weight_initializer : str or Initializer

Initializer for the recurrent weights matrix, used for the linear transformation of the hidden state.

h2r_weight_initializer : str or Initializer

Initializer for the projection weights matrix, used for the linear transformation of the recurrent state.

i2h_bias_initializer : str or Initializer, default ‘lstmbias’

Initializer for the bias vector. By default, bias for the forget gate is initialized to 1 while all other biases are initialized to zero.

h2h_bias_initializer : str or Initializer

Initializer for the bias vector.

prefix : str, default 'lstmp_’

Prefix for name of Block`s (and name of weight if params is `None).

params : Parameter or None

Container for weight sharing between cells. Created if None.

Inputs:

• data: input tensor with shape (batch_size, input_size).
• states: a list of two initial recurrent state tensors, with shape (batch_size, projection_size) and (batch_size, hidden_size) respectively.

Outputs:

• out: output tensor with shape (batch_size, num_hidden).
• next_states: a list of two output recurrent state tensors. Each has the same shape as states.