deliciouscat / LLM_pretrain Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

integrate SotA pretraining methods in Korean LLM, and trying out my ideas.

0 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
README.md		README.md
data_loader.py		data_loader.py
goorm.manifest		goorm.manifest
model.py		model.py
trainer.py		trainer.py

Repository files navigation

LLM_pretrain

##DeBERTa

##Masked Language Model

##Multi-Task Self-Training

https://arxiv.org/abs/2108.11353

About

integrate SotA pretraining methods in Korean LLM, and trying out my ideas.

Report repository

Releases

No releases published

Packages

No packages published

Languages

Python 100.0%