make_tokenizer
- opennmt.tokenizers.make_tokenizer(config=None)[source]
Creates a tokenizer instance from the configuration.
- Parameters
config – Tokenization configuration as a Python dictionary, or a path to a YAML configuration file, or a JSON string.
- Returns
A
opennmt.tokenizers.Tokenizerinstance.- Raises
ValueError – if
configis invalid.