Nadam

class Nadam(learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08, schedule_decay=0.004, **kwargs)

The Nesterov Adam optimizer.

Much like Adam is essentially RMSprop with momentum, Nadam is Adam RMSprop with Nesterov momentum available at http://cs229.stanford.edu/proj2015/054_report.pdf.

This optimizer accepts the following parameters in addition to those accepted by Optimizer.

Parameters:

beta1 : float, optional

Exponential decay rate for the first moment estimates.

beta2 : float, optional

Exponential decay rate for the second moment estimates.

epsilon : float, optional

Small value to avoid division by 0.

schedule_decay : float, optional

Exponential decay rate for the momentum schedule