What a model can do — images in, documents in, images out, tools, a strict JSON schema, any JSON — as the provider's catalogue says, or as an admin declared it on the endpoint where the catalogue says nothing.
Where the answers come from
- OpenRouter (
GET /models):imageamongarchitecture.input_modalities(or on the left side of the oldermodalitystring) →vision;fileamong them →file_input;imageamongarchitecture.output_modalities(or on the right side ofmodality) →image_output;toolsinsupported_parameters→tools;structured_outputs→json_schema;response_formatwithoutstructured_outputs→jsononly.supported_parametersis the union over OpenRouter's upstream providers; the list itself is kept assupported_parameters(the call-time guard leaves out what it lacks when it asks OpenRouter to require every parameter). Neither it nor the model names, prices, output kind orthinking_*flags are in the snapshot: after a restart they are unknown until the first refresh. - Anthropic (native
GET /v1/models, read byOpenRouterClient.fetch_models/2):capabilities.image_input→visionandcapabilities.pdf_input→file_input(capabilities: nullleaves them:unknown); no Claude generates images, soimage_outputis:no;max_input_tokens→context_length,lifecycle. Chat goes through Anthropic's OpenAI-compatible layer, which ignoresresponse_format, sojson_schemais:noandjsonis:yes(the schema is written into the prompt instead);toolsis:yes;capabilities.thinking.types.{enabled,adaptive,disabled,between_tools}→thinking_enabled/thinking_adaptive/thinking_disabled/thinking_between_tools(whichthinking: {type: …}forms the model accepts). - DeepSeek (
GET /models):imageamonginput_modalities→vision,fileamong them →file_input(absent says nothing),imageamongoutput_modalities→image_output;context_window→context_length. - Mistral (
GET /v1/models):capabilities.vision→vision,capabilities.function_calling→tools,max_context_length→context_length;completion_chat: falsemarks a model that does not answer in text. Documents and images out are not listed::unknown. - OpenAI, xAI, Z.ai (
GET /models, a plain list of ids): the models are kept —source_kind/2says:plain— with every capability:unknownand no price, so the recommendations can name them; declarations fill them as for any catalogue. - Every other provider —
:unknown, unless declared.
Declared capabilities
An endpoint may carry provider_settings["capabilities"] —
%{"model" => id, "vision" => "yes", ...} — for a model its catalogue says
nothing about. The declaration counts only for the model it names
(PhoenixKitAI.Endpoint drops it when the model changes) and only where
the catalogue is :unknown: a catalogue :no against a declared :yes is
a conflict (conflicts/2) and the catalogue wins.
The cache
This module is also the process PhoenixKitAI.children/0 starts. It owns
an ETS table keyed by {resolved base URL, model} holding the whole
catalogue of every provider an endpoint points at, and a snapshot of the
endpoints' own models in phoenix_kit_ai_model_capabilities (this
module's migration chain), loaded into the table at start so a restart
without network keeps the answers. Reads (lookup/2, for_endpoint/1) are
ETS only, never network; a miss is :unknown and asks for a refresh in the
background.
Refreshes run every 6 hours, on refresh/0, after an endpoint's model or
provider changes, and on a miss for a base URL whose catalogue has not been
read this cycle (at most once a minute; a miss in a catalogue that was read
is a model it does not have, and asks for nothing). They are single flight
— callers arriving while one runs share it. A catalogue that cannot be read
keeps what the cache had and is tried again on its own after 1, 5 and 30
minutes, then at the next 6-hourly refresh, which starts the 1, 5, 30
minute retries over if it fails too; an empty list counts as a failure,
and a base URL whose endpoints have no API key is not retried.
When a refresh changes what the cache holds,
{:capabilities_refreshed, base_urls} is broadcast on topic/0 with the
base URLs that changed.
Summary
Functions
Every model the cache holds for base_url (the whole catalogue, not just
the endpoints' models), as {model_id, capabilities}. [] when there is
none or the cache is not running. Reads ETS only.
Which catalogue a provider has: :openrouter, :anthropic, :deepseek,
:mistral, or nil for none. Decided by the provider key or by the host
of the base URL — not by the Integrations registry, which may not know the
provider.
Returns a specification to start this module under a supervisor.
The flags declared :yes that the catalogue says :no to.
The capabilities an admin can declare on an endpoint.
The capabilities declared on the endpoint for its current model, as a map
of the flags that say :yes or :no. A declaration naming another model
counts for nothing.
What the endpoint's model can do: the catalogue, with the endpoint's
declaration filling what the catalogue does not know. :unknown when
neither says anything.
Reads an Anthropic /v1/models row.
Reads a DeepSeek /models row.
Reads a Mistral /v1/models row.
Reads an OpenRouter /models row (or an AIModel built from one).
The catalogue's answer for model at base_url, from the cache only.
:unknown when the cache does not have it (or is not running); a miss on
a provider with a catalogue asks for a refresh in the background.
Fills the flags the catalogue leaves :unknown from a declaration. The
result's source is :declared when a declaration filled anything.
Refreshes the cache from the catalogues now and returns when done — or
joins a refresh already running. {:error, reason} when a catalogue could
not be read; what the cache had for it stays.
Asks for a refresh after an endpoint's model or provider changed. Unlike a miss, it is not satisfied by a refresh that started before the change: one running now is followed by another. A no-op when the process is not running.
The base URL the endpoint's requests go to: its own, or the provider's
default when it has none. Trimmed, without a trailing slash — the key the
cache and the snapshot use, so an empty and an explicit default URL are
one entry. nil when there is neither.
What the cache reads at a provider: its catalogue kind (catalogue_kind/2),
else :plain for a direct operator whose /models lists ids only
(OpenAI, xAI, Z.ai — by provider key or API host), else nil.
Subscribes the caller to {:capabilities_refreshed, base_urls}, sent after
a refresh that changed what the cache holds, with the base URLs that
changed.
The PubSub topic refreshes are broadcast on.
Types
@type flag() :: :yes | :no | :unknown
@type t() :: %PhoenixKitAI.Capabilities{ completion_price: float() | nil, context_length: pos_integer() | nil, fetched_at: DateTime.t() | nil, file_input: flag(), image_output: flag(), json: flag(), json_schema: flag(), lifecycle: :active | :deprecated | :retired | nil, name: String.t() | nil, prompt_price: float() | nil, source: :catalogue | :declared, supported_parameters: [String.t()] | nil, text_output: boolean() | nil, thinking_adaptive: flag(), thinking_between_tools: flag(), thinking_disabled: flag(), thinking_enabled: flag(), tools: flag(), vision: flag() }
Functions
Every model the cache holds for base_url (the whole catalogue, not just
the endpoints' models), as {model_id, capabilities}. [] when there is
none or the cache is not running. Reads ETS only.
@spec catalogue_kind(String.t() | nil, String.t() | nil) :: :openrouter | :anthropic | :deepseek | :mistral | nil
Which catalogue a provider has: :openrouter, :anthropic, :deepseek,
:mistral, or nil for none. Decided by the provider key or by the host
of the base URL — not by the Integrations registry, which may not know the
provider.
Returns a specification to start this module under a supervisor.
See Supervisor.
The flags declared :yes that the catalogue says :no to.
@spec declarable() :: [:vision | :file_input | :image_output | :tools | :json_schema]
The capabilities an admin can declare on an endpoint.
@spec declared(PhoenixKitAI.Endpoint.t() | map()) :: %{optional(atom()) => :yes | :no}
The capabilities declared on the endpoint for its current model, as a map
of the flags that say :yes or :no. A declaration naming another model
counts for nothing.
@spec for_endpoint(PhoenixKitAI.Endpoint.t()) :: t() | :unknown
What the endpoint's model can do: the catalogue, with the endpoint's
declaration filling what the catalogue does not know. :unknown when
neither says anything.
Reads an Anthropic /v1/models row.
Reads a DeepSeek /models row.
Reads a Mistral /v1/models row.
@spec from_openrouter(PhoenixKitAI.AIModel.t() | map()) :: t()
Reads an OpenRouter /models row (or an AIModel built from one).
The catalogue's answer for model at base_url, from the cache only.
:unknown when the cache does not have it (or is not running); a miss on
a provider with a catalogue asks for a refresh in the background.
Fills the flags the catalogue leaves :unknown from a declaration. The
result's source is :declared when a declaration filled anything.
Refreshes the cache from the catalogues now and returns when done — or
joins a refresh already running. {:error, reason} when a catalogue could
not be read; what the cache had for it stays.
@spec refresh_after_change() :: :ok
Asks for a refresh after an endpoint's model or provider changed. Unlike a miss, it is not satisfied by a refresh that started before the change: one running now is followed by another. A no-op when the process is not running.
@spec resolved_base_url(PhoenixKitAI.Endpoint.t() | map()) :: String.t() | nil
The base URL the endpoint's requests go to: its own, or the provider's
default when it has none. Trimmed, without a trailing slash — the key the
cache and the snapshot use, so an empty and an explicit default URL are
one entry. nil when there is neither.
@spec source_kind(String.t() | nil, String.t() | nil) :: :openrouter | :anthropic | :deepseek | :mistral | :plain | nil
What the cache reads at a provider: its catalogue kind (catalogue_kind/2),
else :plain for a direct operator whose /models lists ids only
(OpenAI, xAI, Z.ai — by provider key or API host), else nil.
@spec subscribe() :: :ok | {:error, term()}
Subscribes the caller to {:capabilities_refreshed, base_urls}, sent after
a refresh that changed what the cache holds, with the base URLs that
changed.
@spec topic() :: String.t()
The PubSub topic refreshes are broadcast on.