TransformerLayerWrapper

class opennmt.layers.TransformerLayerWrapper(*args, **kwargs)[source]

Layer wrapper that applies a standard Transformer preprocessing and postprocessing.

With pre_norm=True:

y = layer_norm(x)
y = dropout(layer(y)) + x

With pre_norm=False:

y = dropout(layer(x)) + x
y = layer_norm(y)

Inherits from: opennmt.layers.LayerWrapper

__init__(layer, output_dropout, pre_norm=True, **kwargs)[source]

Initializes the wrapper.

Parameters

  • layer – The Transformer layer to wrap.

  • output_dropout – The dropout to apply on the layer output.

  • pre_norm – If True, layer normalization is applied before calling the layer. Otherwise it is applied after.

  • **kwargs – Additional layer arguments.

map_v1_weights(weights)[source]