NoamDecay

class opennmt.schedules.NoamDecay(scale, model_dim, warmup_steps)[source]

Defines the decay function described in https://arxiv.org/abs/1706.03762.

Inherits from: keras.src.optimizers.schedules.learning_rate_schedule.LearningRateSchedule

__init__(scale, model_dim, warmup_steps)[source]

Initializes the decay function.

Parameters

  • scale – The scale constant.

  • model_dim – The model dimension.

  • warmup_steps – The number of warmup steps.