Incorporating sentence embeddings into Transformer architecture for Neural Machine Translation built on the idea of this paper.
The illustration of proposed embed-fusion module. Sentence embedding represents the universal sentence embedding from SimCSE model. e1 ,e2 ,…, eL represent word feature vectors of source input from Transformer encoder. Concatmeans concatenation operation, and Dense means fully-connected network.
