Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: lora微调模型无法在vllm加载 #241

Closed
1 task done
Caismis opened this issue Sep 18, 2024 · 3 comments
Closed
1 task done

[Bug]: lora微调模型无法在vllm加载 #241

Caismis opened this issue Sep 18, 2024 · 3 comments
Labels
bug Something isn't working triage

Comments

@Caismis
Copy link

Caismis commented Sep 18, 2024

Is there an existing issue ? / 是否已有相关的 issue ?

  • I have searched, and there is no existing issue. / 我已经搜索过了,没有相关的 issue。

Describe the bug / 描述这个 bug

按照官方教程进行lora微调和模型合并保存后,调用vllm serve无法加载模型,报

ValueError: Model architectures ['MiniCPM3Model'] are not supported for now. Supported architectures: ['MiniCPM3ForCausalLM', 'MiniCPM3Model', 'AquilaModel', 'AquilaForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'Blip2ForConditionalGeneration', 'ChameleonForConditionalGeneration', 'ChatGLMModel', 'ChatGLMForConditionalGeneration', 'CohereForCausalLM', 'DbrxForCausalLM', 'DeciLMForCausalLM', 'DeepseekForCausalLM', 'DeepseekV2ForCausalLM', 'FalconForCausalLM', 'FuyuForCausalLM', 'GemmaForCausalLM', 'Gemma2ForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'InternLM2ForCausalLM', 'InternVLChatModel', 'JAISLMHeadModel', 'LlamaForCausalLM', 'LlavaForConditionalGeneration', 'LlavaNextForConditionalGeneration', 'LLaMAForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'QuantMixtralForCausalLM', 'MptForCausalLM', 'MPTForCausalLM', 'MiniCPMForCausalLM', 'MiniCPMV', 'NemotronForCausalLM', 'OlmoForCausalLM', 'OPTForCausalLM', 'OrionForCausalLM', 'PersimmonForCausalLM', 'PaliGemmaForConditionalGeneration', 'PhiForCausalLM', 'Phi3ForCausalLM', 'Phi3VForCausalLM', 'QWenLMHeadModel', 'Qwen2ForCausalLM', 'Qwen2MoeForCausalLM', 'RWForCausalLM', 'StableLMEpochForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM', 'ArcticForCausalLM', 'XverseForCausalLM', 'Phi3SmallForCausalLM', 'MedusaModel', 'MLPSpeculatorPreTrainedModel', 'JambaForCausalLM', 'MistralModel']
查看模型的config.json显示模型architecturesMiniCPM3Model而非base模型中的MiniCPM3ForCausalLM

To Reproduce / 如何复现

按照官方llama factory教程,对示例dpo数据采取lora微调,并按官方教程对模型进行合并保存,通过vllm serve调用后报错

Expected behavior / 期望的结果

正常运行微调模型

Screenshots / 截图

No response

Environment / 环境

- OS: wsl2-Ubuntu 
- Pytorch: torch 2.4.0
- Device: RTX 4090
python环境和[官方教程](https://modelbest.feishu.cn/docx/ATfTdGsSXotRDex0OzycvRjsn0e)中一致

Additional context / 其他信息

No response

@Caismis Caismis added bug Something isn't working triage labels Sep 18, 2024
@LDLINGLINGLING
Copy link
Collaborator

你好,手动修改config文件试一试吧

@Caismis
Copy link
Author

Caismis commented Sep 19, 2024

尝试了修改为MiniCPM3ForCausalLM和MiniCPMModel,均报错,前者在加载模型bin文件时出错

@Caismis
Copy link
Author

Caismis commented Sep 19, 2024

当把config文件中改成MiniCPM3ForCausalLM时,报
File "/home/.local/lib/python3.10/site-packages/sglang/srt/models/minicpm3.py", line 650, in load_weights
param = params_dict[name]
KeyError: 'embed_tokens.weight'
, detoken_init_state: init ok
改成MiniCPMForCausalLM时,报
RuntimeError: The size of tensor a (16) must match the size of tensor b (32) at non-singleton dimension 0
, detoken_init_state: init ok

@Caismis Caismis closed this as completed Sep 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

2 participants