RsqrtDecay

class opennmt.schedules.RsqrtDecay(scale, warmup_steps)

Decay based on the reciprocal of the step square root. This corresponds to rsqrt_decay in Tensor2Tensor.

\[\text{schedule}(\text{step}) = \frac{\text{scale}} {\sqrt{\max(\text{step},\text{warmup_steps})}}\]

See also

• opennmt.schedules.InvSqrtDecay

Inherits from: keras.src.optimizers.schedules.learning_rate_schedule.LearningRateSchedule

__init__(scale, warmup_steps)

Initializes the decay function.

Parameters

• scale – The scale constant.

• warmup_steps – The number of warmup steps.