LanguageModel

class opennmt.models.LanguageModel(*args, **kwargs)[source]

A language model.

Inherits from: opennmt.models.SequenceGenerator

Extended by:

__init__(decoder, embedding_size=None, reuse_embedding=True)[source]

Initializes the language model.

Parameters
  • decoder – A opennmt.decoders.Decoder instance.

  • embedding_size – The size of the word embedding. If not set, pretrained embeddings should be defined in the configuration.

  • reuse_embedding – If True, reuse the embedding weights in the output layer.

Raises

ValueError – if the decoder type is invalid.

property features_inputter

The inputter producing features.

auto_config(num_replicas=1)[source]

Returns automatic configuration values specific to this model.

Parameters

num_replicas – The number of synchronous model replicas used for the training.

Returns

A partial training configuration.

initialize(data_config, params=None)[source]

Initializes the model from the data configuration.

Parameters
  • data_config – A dictionary containing the data configuration set by the user (e.g. vocabularies, tokenization, pretrained embeddings, etc.).

  • params – A dictionary of hyperparameters.

build(input_shape)[source]

Creates the variables of the layer (for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call. It is invoked automatically before the first execution of call().

This is typically used to create the weights of Layer subclasses (at the discretion of the subclass implementer).

Parameters

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call(features, labels=None, training=None, step=None)[source]

Runs the model.

Parameters
  • features – A nested structure of features tf.Tensor.

  • labels – A nested structure of labels tf.Tensor.

  • training – If True, run in training mode.

  • step – The current training step.

Returns

A tuple containing,

  • The model outputs (usually unscaled probabilities).

  • The model predictions.

compute_loss(outputs, labels, training=True)[source]

Computes the loss.

Parameters
  • outputs – The model outputs (usually unscaled probabilities).

  • labels – The dict of labels tf.Tensor.

  • training – If True, compute the loss for training.

Returns

The loss or a tuple (numerator, train_denominator, stats_denominator) to use a different normalization for training compared to reporting (e.g. batch-normalized for training vs. token-normalized for reporting).

format_prediction(prediction, params=None)[source]

Formats the model prediction for file saving.

Parameters
  • prediction – The model prediction (same structure as the second output of opennmt.models.Model.call()).

  • params – (optional) Dictionary of formatting parameters.

Returns

A string or list of strings.