filter_examples_by_length

opennmt.data.filter_examples_by_length(maximum_features_length=None, maximum_labels_length=None, features_length_fn=None, labels_length_fn=None)[source]

Transformation that filters out examples with zero length or length that is greater than the configured maximum.

Example

>>> dataset = dataset.apply(opennmt.data.filter_examples_by_length(...))
Parameters

  • maximum_features_length – The maximum length or list of maximum lengths of the features sequence(s). None to not constrain the length.

  • maximum_labels_length – The maximum length or list of maximum lengths of the labels sequence(s). None to not constrain the length.

  • features_length_fn – A function mapping features to a sequence length.

  • labels_length_fn – A function mapping labels to a sequence length.

Returns

A tf.data.Dataset transformation.