Target to source alignments are returned as part of the translation API. They are also used by some decoding features such as
To improve their quality, OpenNMT-tf supports guided alignment training. It is a supervised approach to constrain one or more attention heads by alignments produced by external tools such as fast_align.
Guided alignment should be configured in the YAML configuration file:
data: train_alignments: train-alignment.txt params: guided_alignment_type: ce guided_alignment_weight: 1
train-alignment.txt is a text file that uses the “Pharaoh” alignment format, e.g.:
0-0 1-1 2-4 3-2 4-3 5-5 6-6 0-0 1-1 2-2 2-3 3-4 4-5 0-0 1-2 2-1 3-3 4-4 5-5
where a pair
i-j indicates that the
ith word of the source sentence is aligned with the
jth word of the target sentence (zero-indexed).
Alignments and Transformer models¶
Since Transfomer models have multiple attention heads, training with guided alignment (see above) is required to get usable alignments. The current implementation will constrain and return the first attention head of the last decoder layer.