Text generation
CTranslate2 exposes high-level classes to run generative language models such as GPT-2. The main entrypoint is the Generator
class which can run conditional or unconditional generation.
Tip
See the GPT-2 example in the Transformers guide.
Perplexity
The Generator
class exposes the method score_batch
which can be used to calculate the perplexity for full sequences. It returns the log-likelihood of each token.
Tip
See the WMT19 language model example in the Fairseq guide.
Special tokens
Special tokens such as the decoder start token <s>
should be explicitly included in the input if required by the model. No special tokens are added by the generator methods.
Note
This is different from the translator methods which usually include these special tokens implicitly.