Adamax

class Adamax(learning_rate=0.002, beta1=0.9, beta2=0.999, **kwargs)

The AdaMax optimizer.

It is a variant of Adam based on the infinity norm available at http://arxiv.org/abs/1412.6980 Section 7.

The optimizer updates the weight by:

grad = clip(grad * rescale_grad + wd * weight, clip_gradient)
m = beta1 * m_t + (1 - beta1) * grad
u = maximum(beta2 * u, abs(grad))
weight -= lr / (1 - beta1**t) * m / u

This optimizer accepts the following parameters in addition to those accepted by Optimizer.

Parameters:

beta1 : float, optional

Exponential decay rate for the first moment estimates.

beta2 : float, optional

Exponential decay rate for the second moment estimates.