TransformerEncoderModelSpec

class ctranslate2.specs.TransformerEncoderModelSpec

Describes a Transformer encoder model (e.g. BERT).

Inherits from: ctranslate2.specs.LanguageModelSpec

Attributes:

• config

• name

• revision

Methods:

• get_default_config

• get_vocabulary_size

• optimize

• register_file

• register_vocabulary

• save

• validate

• variables

__init__(encoder: TransformerEncoderSpec, pooling_layer: bool = False, pooling_activation: Activation = Activation.Tanh)

Initializes a Transformer encoder model specification.

Parameters

• encoder – The encoder specification.

• pooling_layer – Add the pooling layer.

• pooling_activation – The activation to apply after the pooling layer.

get_default_config()

Returns the default configuration used by this model.

get_vocabulary_size()

Returns the vocabulary size expected by the model.

optimize(quantization: Optional[str] = None) → None

Recursively applies some optimizations to this layer:

• Alias variables with the same shape and value.

• Quantize weights.

Parameters

quantization – Weight quantization scheme (possible values are: int8, int8_float32, int8_float16, int8_bfloat16, int16, float16, bfloat16, float32).

register_file(path: str, filename: Optional[str] = None) → None

Registers a file to be saved in the model directory.

register_vocabulary(tokens: List[str]) → None

Registers the vocabulary of tokens.

Parameters

tokens – List of tokens.

save(output_dir: str) → None

Saves this model on disk.

Parameters

output_dir – Output directory where the model is saved.

validate() → None

Verify that the required weights are set.

Raises

ValueError – If a required weight is not set in the specification.

variables(prefix: str = '', ordered: bool = False) → Dict[str, ndarray]

Recursively returns the weights from this layer and its children.

Parameters

• prefix – Prefix to prepend to all variable names.

• ordered – If set, an ordered list is returned instead.

Returns

Dictionary mapping variables name to value.

property config

The model configuration.

property name

The name of the model specification.

property revision

The model specification revision.

This value is incremented each time the weights layout of the model is changed (e.g. a weight is renamed).