Prompt cache

Pay full price the first time you send a system block. Half after that.

Provider prompt caching discounts repeated prefixes. Promptimizer models that on every session.

What is hashed

Every system message, plus the first 800 characters of a long user context. SHA-256 of the canonical JSON. Repeats set prefix_cache_hit and bill those tokens at 50% input.

Exact cache

The full messages + model tuple is stored. An identical retry returns the stored completion.

Backends