A write-once cache for provider answers, keyed on what would be sent.
Every PhoenixKitAI verb accepts cache: true (default TTL),
cache: [ttl: seconds | :infinity, key: term, refresh: boolean] or
cache: :refresh (bypass a stored answer and overwrite it). Without
key: the key is a SHA-256 over the verb, the endpoint (uuid and
updated_at, so editing the endpoint invalidates its entries), the model
and the request-shaping inputs — messages or prompt, image bytes, request
options, the JSON shape asked for — never the caller's source,
attribution, idempotency key or user_uuid. With key: ("product:123",
say) the caller's term replaces the request material, so a re-rendered
prompt still hits; the verb, endpoint, model and JSON shape stay in the
key.
Entries are shared across users: two callers asking the same thing get
the same answer, and a caller key is global to the endpoint. Scope it
yourself (cache: [key: {user_uuid, "profile"}]) when the answer is
personal.
A hit returns the stored result and makes no provider call; the verbs
pass an :on_hit callback through meta that writes a zero-cost usage
row marked cached: true (attribution and the prompt link intact), so
reports stay truthful — a hit does not count against a spend cap because
it costs nothing. [:phoenix_kit_ai, :cache, :hit | :miss | :refresh]
fires per lookup. Only {:ok, _} results are stored; concurrent misses
on the same key each call the provider (no single-flight).
Entries live in a public ETS table owned by this process (reads and
writes never queue behind it); the process only sweeps expired entries.
config :phoenix_kit_ai, request_cache: [ttl: 86_400, sweep: 300, max_entries: 10_000, max_value_bytes: 8_000_000, default: false] —
read per call; default: true caches every cacheable verb unless a call
says cache: false. A full table or an oversized value (an image edit
result, typically) is simply not stored. Restarting the node empties the
cache, and without the module's supervisor tree (PhoenixKitAI.children/0)
every lookup is a miss — this is a cost saver, not a store of record.
Summary
Functions
The caller's own key term from cache: [key: …], or nil.
Returns a specification to start this module under a supervisor.
Drops every entry.
Default TTL in seconds.
Runs fun unless a fresh answer for key is stored. key may be a
zero-arity function, called only when caching is on (hashing image bytes
is not free). fun returns {:ok, result} | {:error, _}; only {:ok, _}
is stored. meta is the telemetry metadata; its :on_hit entry, a
one-arity function, runs on a hit with the stored value and is not
emitted.
A stored, unexpired value.
The cache key for a verb on an endpoint over material (any term). The
endpoint's updated_at is part of it, so an admin edit starts fresh.
How the caller asked for caching: false, {:use, ttl} or {:refresh, ttl}.
Stores value for ttl seconds (or forever with :infinity). Returns
:ok, or {:error, :full | :too_large} when the table is at
max_entries or the value is over max_value_bytes (nothing stored).
Number of stored entries (expired ones included until the next sweep).
Types
@type key() :: binary()
@type ttl() :: pos_integer() | :infinity
Functions
The caller's own key term from cache: [key: …], or nil.
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec clear() :: :ok
Drops every entry.
@spec default_ttl() :: pos_integer()
Default TTL in seconds.
@spec fetch( key() | (-> key()), keyword(), (-> {:ok, term()} | {:error, term()}), map() ) :: {:ok, term()} | {:error, term()}
Runs fun unless a fresh answer for key is stored. key may be a
zero-arity function, called only when caching is on (hashing image bytes
is not free). fun returns {:ok, result} | {:error, _}; only {:ok, _}
is stored. meta is the telemetry metadata; its :on_hit entry, a
one-arity function, runs on a hit with the stored value and is not
emitted.
A stored, unexpired value.
@spec key(atom(), PhoenixKitAI.Endpoint.t(), String.t() | nil, term()) :: key()
The cache key for a verb on an endpoint over material (any term). The
endpoint's updated_at is part of it, so an admin edit starts fresh.
How the caller asked for caching: false, {:use, ttl} or {:refresh, ttl}.
Stores value for ttl seconds (or forever with :infinity). Returns
:ok, or {:error, :full | :too_large} when the table is at
max_entries or the value is over max_value_bytes (nothing stored).
@spec size() :: non_neg_integer()
Number of stored entries (expired ones included until the next sweep).