Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[New Model]: bump a new version of vllm to support Qwen2 series #5773

Closed
AlphaINF opened this issue Jun 23, 2024 · 4 comments
Closed

[New Model]: bump a new version of vllm to support Qwen2 series #5773

AlphaINF opened this issue Jun 23, 2024 · 4 comments
Labels
new model Requests to new models

Comments

@AlphaINF
Copy link

The model to consider.

Qwen2 series, in https://huggingface.co/Qwen/Qwen2-7B

The closest model vllm already supports.

Qwen1.5 series

What's your difficulty of supporting the model you want?

Current version （v0.5.0-post1）didn't have the kernel to run Qwen2 series.
The intermediate size of all qwen2 models is not supported by punica yet.
And now the problem have been solved by #5441
I'm glad to see that the code is bumped and a new version will be introduced in the next days!
@AlphaINF AlphaINF added the new model Requests to new models label Jun 23, 2024
@marcelbra
Copy link

Would love an update on that, too, and am pretty sure many ppl would appreciate it. Thanks!!

@DarkLight1337
Copy link
Member

Please check out #5806 regarding the next release.

@DarkLight1337 DarkLight1337 mentioned this issue Jun 28, 2024
Closed
@DarkLight1337
Copy link
Member

Closing as v0.5.1 has been released.

@AlphaINF
Copy link
Author

Thank you ！！！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new model Requests to new models
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@marcelbra @DarkLight1337 @AlphaINF