opennmt.utils.misc module

Various utility functions to use throughout the project.

opennmt.utils.misc.print_bytes(str_as_bytes, stream=None)[source]

Prints a string viewed as bytes.

Parameters:
  • str_as_bytes – The bytes to print.
  • stream – The stream to print to (sys.stdout if not set).
opennmt.utils.misc.format_translation_output(sentence, score=None, token_level_scores=None, attention=None, alignment_type=None)[source]

Formats a translation output with possibly scores, alignments, etc., e.g:

1.123214 ||| Hello world ||| 0.30907777 0.030488174 ||| 0-0 1-1

Parameters:
  • sentence – The translation to output.
  • score – If set, attach the score.
  • token_level_scores – If set, attach the token level scores.
  • attention – The attention vector.
  • alignment_type – The type of alignments to format (can be: “hard”).
opennmt.utils.misc.item_or_tuple(x)[source]

Returns x as a tuple or its single element.

opennmt.utils.misc.classes_in_module(module, public_only=False)[source]

Returns a generator over the classes defined in module.

opennmt.utils.misc.function_args(fun)[source]

Returns the name of fun arguments.

opennmt.utils.misc.get_third_party_dir()[source]

Returns a path to the third_party directory.

opennmt.utils.misc.count_lines(filename)[source]

Returns the number of lines of the file filename.

opennmt.utils.misc.count_parameters()[source]

Returns the total number of trainable parameters.

opennmt.utils.misc.shape_list(x)[source]

Return list of dims, statically where possible.

opennmt.utils.misc.extract_prefixed_keys(dictionary, prefix)[source]

Returns a dictionary with all keys from dictionary that are prefixed with prefix.

opennmt.utils.misc.extract_suffixed_keys(dictionary, suffix)[source]

Returns a dictionary with all keys from dictionary that are suffixed with suffix.

opennmt.utils.misc.extract_batches(tensors)[source]

Returns a generator to iterate on each batch of a Numpy array or dict of Numpy arrays.

opennmt.utils.misc.merge_dict(dict1, dict2)[source]

Merges dict2 into dict1.

Parameters:
  • dict1 – The base dictionary.
  • dict2 – The dictionary to merge.
Returns:

The merged dictionary dict1.

class opennmt.utils.misc.OrderRestorer(index_fn, callback_fn)[source]

Bases: object

Helper class to restore out-of-order elements in order.

__init__(index_fn, callback_fn)[source]

Initializes this object.

Parameters:
  • index_fn – A callable mapping an element to a unique index.
  • callback_fn – A callable taking an element that will be called in order.
push(x)[source]

Push event x.

opennmt.utils.misc.add_dict_to_collection(collection_name, dict_)[source]

Adds a dictionary to a graph collection.

Parameters:
  • collection_name – The name of the collection to add the dictionary to
  • dict – A dictionary of string keys to tensor values
opennmt.utils.misc.get_dict_from_collection(collection_name)[source]

Gets a dictionary from a graph collection.

Parameters:collection_name – A collection name to read a dictionary from
Returns:A dictionary with string keys and tensor values