LanguageModel
- class opennmt.models.LanguageModel(*args, **kwargs)[source]
A language model.
Inherits from:
opennmt.models.SequenceGeneratorExtended by:
- __init__(decoder, embedding_size=None, reuse_embedding=True)[source]
Initializes the language model.
- Parameters
decoder – A
opennmt.decoders.Decoderinstance.embedding_size – The size of the word embedding. If not set, pretrained embeddings should be defined in the configuration.
reuse_embedding – If
True, reuse the embedding weights in the output layer.
- Raises
ValueError – if the decoder type is invalid.
- property features_inputter
The inputter producing features.
- auto_config(num_replicas=1)[source]
Returns automatic configuration values specific to this model.
- Parameters
num_replicas – The number of synchronous model replicas used for the training.
- Returns
A partial training configuration.
- initialize(data_config, params=None)[source]
Initializes the model from the data configuration.
- Parameters
data_config – A dictionary containing the data configuration set by the user (e.g. vocabularies, tokenization, pretrained embeddings, etc.).
params – A dictionary of hyperparameters.
- build(input_shape)[source]
Creates the variables of the layer (for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call. It is invoked automatically before the first execution of call().
This is typically used to create the weights of Layer subclasses (at the discretion of the subclass implementer).
- Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).
- call(features, labels=None, training=None, step=None)[source]
Runs the model.
- Parameters
features – A nested structure of features
tf.Tensor.labels – A nested structure of labels
tf.Tensor.training – If
True, run in training mode.step – The current training step.
- Returns
A tuple containing,
The model outputs (usually unscaled probabilities).
The model predictions.
- compute_loss(outputs, labels, training=True)[source]
Computes the loss.
- Parameters
outputs – The model outputs (usually unscaled probabilities).
labels – The dict of labels
tf.Tensor.training – If
True, compute the loss for training.
- Returns
The loss or a tuple
(numerator, train_denominator, stats_denominator)to use a different normalization for training compared to reporting (e.g. batch-normalized for training vs. token-normalized for reporting).
- format_prediction(prediction, params=None)[source]
Formats the model prediction for file saving.
- Parameters
prediction – The model prediction (same structure as the second output of
opennmt.models.Model.call()).params – (optional) Dictionary of formatting parameters.
- Returns
A string or list of strings.