New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request] Gemma2 support & models #111

Open

kristaller486 opened this issue Jul 4, 2024 · 3 comments

kristaller486 commented Jul 4, 2024

Gemma2 is incredible model. With AQLM, it will fit into 12GB GPU.

Owner

Vahe1994 commented Jul 12, 2024

Hello!
Sorry for the late answer. It should fit .I'm planning to get to this soon. I'll keep you updated in this thread, but it may take weeks.

Author

kristaller486 commented Jul 24, 2024

Thank you for answer. I'll be waiting!

igvasilev commented Aug 23, 2024 •

edited

Loading

Will you add Gemma2 27B as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment