Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: mmlu translated professionally by OpenAI #2312

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

giuliolovisotto
Copy link
Contributor

@giuliolovisotto giuliolovisotto commented Sep 17, 2024

Following up from #2305

@baberabb

  • I grouped by language in subfolders
  • I created a group per language
  • I mostly copied the existing mmlu/default so I dropped some options to do cot/generative/fewshots.

I'll try to run something like llama 70b in the next days to make sure numbers make sense.

@giuliolovisotto giuliolovisotto changed the title Draft: mmlu but it is translated professionally by OpenAI Draft: mmlu translated professionally by OpenAI Sep 24, 2024
@giuliolovisotto
Copy link
Contributor Author

Openai uploaded the dataset on hf, I'll use that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant