CTranslate2 supports Transformer models trained with OpenNMT-py. The conversion simply requires the PyTorch model path, e.g.:

pip install OpenNMT-py
ct2-opennmt-py-converter --model_path --output_dir ct2_model

Alternatively, you can also convert the model directly from OpenNMT-py with the model release script:

onmt_release_model --model --format ctranslate2 --output ct2_model


See the quickstart for a complete example using an OpenNMT-py model.

Text generation with transformer_lm

Decoder-only models using the transformer_lm decoder type are supported and can be converted with the same command line.

During generation, make sure to always include <s> in the start tokens, e.g.:

generator = ctranslate2.Generator(model_path)
generator.generate_batch([["<s>", "▁Hello"]])