Text generation

CTranslate2 exposes high-level classes to run generative language models such as GPT-2. The main entrypoint is the Generator class which can run conditional or unconditional generation.

Tip

See the GPT-2 example in the Transformers guide.

Perplexity

The Generator class exposes the method score_batch which can be used to calculate the perplexity for full sequences. It returns the log-likelihood of each token.

Tip

See the WMT19 language model example in the Fairseq guide.

Special tokens

Special tokens such as the decoder start token <s> should be explicitly included in the input if required by the model. No special tokens are added by the generator methods.

Note

This is different from the translator methods which usually include these special tokens implicitly.