RMSProp

class RMSProp(learning_rate=0.001, gamma1=0.9, gamma2=0.9, epsilon=1e-08, centered=False, clip_weights=None, **kwargs)

The RMSProp optimizer.

Two versions of RMSProp are implemented:

If centered=False, we follow http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf by Tieleman & Hinton, 2012. For details of the update algorithm see rmsprop_update.

If centered=True, we follow http://arxiv.org/pdf/1308.0850v5.pdf (38)-(45) by Alex Graves, 2013. For details of the update algorithm see rmspropalex_update.

This optimizer accepts the following parameters in addition to those accepted by Optimizer.

Parameters:

gamma1: float, optional

A decay factor of moving average over past squared gradient.

gamma2: float, optional

A “momentum” factor. Only used if centered`=``True`.

epsilon : float, optional

Small value to avoid division by 0.

centered : bool, optional

Flag to control which version of RMSProp to use.:

True: will use Graves's version of `RMSProp`,
False: will use Tieleman & Hinton's version of `RMSProp`.

clip_weights : float, optional

Clips weights into range [-clip_weights, clip_weights].