[Feature]: supports_prompt_caching property for LMs

[neubig]

The Feature

Currently, claude and gemini support prompt caching, but other LMs don't. It'd be great if there could be a supports_prompt_caching property (like supports_vision etc.) that tells us which models support prompt caching.

Motivation, pitch

In OpenHands we would like to turn prompt caching on by default, but we can't easily do so without this feature, because if we attempt to use prompt caching with an LM that doesn't support it then it throws an error.

We tried to fix this by checking for claude in the model name, but that doesn't work because claude on GCP apparently doesn't support prompt caching yet.

Twitter / LinkedIn details

No response