add debug dir for prompts #205
Conversation
opendevin/llm/llm.py
Outdated
| message_back = resp['choices'][0]['message']['content'] | ||
| self.write_debug(messages, message_back) | ||
| return resp | ||
| self._completion = wrapper |
There was a problem hiding this comment.
I guess by wrapping the ._completion function, we will break the documentation and argument hints
There was a problem hiding this comment.
hmm, that's a bummer. What happens if we only wrap it when debug mode is on?
There was a problem hiding this comment.
I think that that should make it work? since the wrapping is only happening at the runtime, and the argument hints only depends on static type?
|
|
||
| DEFAULT_MODEL = os.getenv("LLM_MODEL", "gpt-4-0125-preview") | ||
| DEFAULT_API_KEY = os.getenv("LLM_API_KEY") | ||
| PROMPT_DEBUG_DIR = os.getenv("PROMPT_DEBUG_DIR", "") |
There was a problem hiding this comment.
We probably need a centralized documents to keep track of all the Env Vars at some point :)
|
@rbren opps looks like we need to fix 
|
|
👍 found a way to make an exception |
* add debug dir for prompts * add indent to dumps * only wrap completion in debug mode * fix mypy
This PR allows you to set
PROMPT_DEBUG_DIR=./debugto get a record of every prompt and response. This helps agent creators debug their agents.