make_cardinality_multiple_of

opennmt.data.make_cardinality_multiple_of(divisor)[source]

Transformation that ensures that the dataset cardinality is a multiple of divisor.

Example

>>> dataset = tf.data.Dataset.range(7)
>>> dataset = dataset.apply(opennmt.data.make_cardinality_multiple_of(10))
>>> len(list(iter(dataset)))
10

Parameters: divisor – The value that should divide the dataset size.
Returns: A tf.data.Dataset transformation.

Tip

This transformation is useful when training multiple replicas on a finite dataset. It ensures that each replica receives a non empty batch in the last training iteration.