About the 'Word Embed', how did you get it?

[cassiaaaaaa]

Thanks for sharing your great work!
I have a question about the paper. As said in the paper, "'Word Embed' represents the word embedding dictionary of MLM". It seems to be a fixed module. Where did you get the module weights of Word Embed? From other MLM model? How to find it?

[ByungKwanLee]

All Language models have word embedding. We used it in backbone mlm model.