RNMTPlusDecay

class opennmt.schedules.RNMTPlusDecay(scale, num_replicas, warmup_steps=500, start_step=600000, end_step=1200000)

Defines the decay function described in https://arxiv.org/abs/1804.09849.

Inherits from: keras.src.optimizers.schedules.learning_rate_schedule.LearningRateSchedule

__init__(scale, num_replicas, warmup_steps=500, start_step=600000, end_step=1200000)

Initializes the decay function.

Parameters

• scale – The scale constant.

• num_replicas – The number of concurrent model replicas.

• warmup_steps – The number of warmup steps.

• start_step – The start step of the exponential decay.

• end_step – The end step of the exponential decay.