FeedForwardNetwork

class opennmt.layers.FeedForwardNetwork(*args, **kwargs)[source]

Implements the Transformer’s “Feed Forward” layer.

\[ffn(x) = max(0, x*W_1 + b_1)*W_2 + b_2\]

Inherits from: keras.src.engine.base_layer.Layer

__init__(inner_dim, output_dim, dropout=0.1, activation=<function relu>, **kwargs)[source]

Initializes this layer.

Parameters

  • inner_dim – The number of units of the inner linear transformation.

  • output_dim – The number of units of the ouput linear transformation.

  • dropout – The probability to drop units from the activation output.

  • activation – The activation function to apply between the two linear transformations.

  • kwargs – Additional layer arguments.

call(inputs, training=None)[source]

Runs the layer.

map_v1_weights(weights)[source]