LRScheduler

class LRScheduler(base_lr=0.01, warmup_steps=0, warmup_begin_lr=0, warmup_mode='linear')

Base class of a learning rate scheduler.

A scheduler returns a new learning rate based on the number of updates that have been performed.

Parameters:

base_lr : float, optional

The initial learning rate.

warmup_steps: int

number of warmup steps used before this scheduler starts decay

warmup_begin_lr: float

if using warmup, the learning rate from which it starts warming up

warmup_mode: string

warmup can be done in two modes. ‘linear’ mode gradually increases lr with each step in equal increments ‘constant’ mode keeps lr at warmup_begin_lr for warmup_steps