Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update: 使用通义千问大模型 #8

Merged
merged 2 commits into from
Jan 27, 2024
Merged

update: 使用通义千问大模型 #8

merged 2 commits into from
Jan 27, 2024

Conversation

leyiang
Copy link
Contributor

@leyiang leyiang commented Jan 27, 2024

国内的几家都试了下,只有通义千问返回的是正确的。
看了下llama_index支持的大模型,国内大模型都不在上面。
按它的Interface实现了一个QwenLLM, 可以直接用通义千问的API

用法:

  1. 根据教程安装sdk, 设置key
  2. config.yaml 中 llm.name 设置成 "qwen" (用的是qwen-max模型,目前免费)
  3. 按原教程用就可以了

问题:

  1. 代码好像会对同一个问题多轮提问,以优化回答,API大部分都会回复:"新的上下文没什么帮助..."
  2. Qwen 对于部分问题会报错 "DataInspectionFailed"。这个没有什么办法,好像只有API会这样回复,我在代码里把prompt打印出来,直接到网页里问就没问题,问了阿里的工程师,还没回复。

样例:
2024-01-26-232909_2401x231_scrot
2024-01-26-232953_2418x210_scrot

@leyiang
Copy link
Contributor Author

leyiang commented Jan 27, 2024

2024-01-27-001028_1579x285_scrot

QwenLLM.py Outdated
LLMMetadata,
)

DEFAULT_MODEL = "qwen-turobo"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

qwen-turobo 这是一个typo吗

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

是的,但是这个没用到,可以删掉

@wxywb
Copy link
Owner

wxywb commented Jan 27, 2024

为行动力点赞👍,可以把QwenLLM.py 放入custom/llms/ 文件夹下,然后建一个新的config_qwen.yml, 我在Update里加上对于Qwen的支持。

@leyiang
Copy link
Contributor Author

leyiang commented Jan 27, 2024

为行动力点赞👍,可以把QwenLLM.py 放入custom/llms/ 文件夹下,然后建一个新的config_qwen.yml, 我在Update里加上对于Qwen的支持。

放到 custom/llms这部OK了,但是创建新的config_qwen.yml没理解,里边放什么呢?

@wxywb
Copy link
Owner

wxywb commented Jan 27, 2024

现在config.yaml里面是openai的模型,config_qwen.yml使用qwen作为llm。

@wxywb
Copy link
Owner

wxywb commented Jan 27, 2024

我先合了吧,我来处理好了。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants