Skip to content
/ numeracy-literacy Public

Code for paper: "Numeracy Enhances the Literacy of Language Models"

4 stars 1 fork Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

avi-jit/numeracy-literacy

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
valids
valids
 
 
.gitignore
.gitignore
 
 
LICENSE
LICENSE
 
 
dataset.py
dataset.py
 
 
eval.py
eval.py
 
 
readme.md
readme.md
 
 
requirements.txt
requirements.txt
 
 
train.py
train.py
 
 

Repository files navigation

Numeracy Enhances the Literacy of Language Models

Anonymous ACL-IJCNLP 2021 Submission

train.py: model description and training

nice python train.py --batch-size 256 --gpus 0, --tsamples 100_000 --dsamples 10_000 --max_epochs 10 --enc exp --hidden 200 --accumulate_grad_batches 4 --seed 0 --dataset WC

eval.py: reports perplexity and hit@k scores

nice python eval.py --limit 10_000 --ckpt checkpoints/read-WC-def-adj-noun/epoch=9.ckpt --maxtoks 150 --batch-size 128 --device 0

dataset.py: tokenized dataset description

common...txt: list of sentence indices to evaluate

About

Code for paper: "Numeracy Enhances the Literacy of Language Models"

Topics

nlp machine-learning encoder numbers representation-learning language-model bert language-modelling perplexity pytorch-lightning huggingface-transformers

Resources

Readme
Activity

Stars

4 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

No packages published

Languages