Caps how many provider calls run at once per key, and lines up the rest in arrival order.

A provider rate-limits a burst (429) long before a spend cap matters, and a batch job that fires 50 calls at once mostly buys retries. The limiter gives a key (by default the endpoint's provider, "provider:openrouter") a number of slots; a call that finds them taken waits its turn, and gives up with {:error, :concurrency_limit_timeout} after wait milliseconds or with {:error, :concurrency_queue_full} when max_waiting calls are already in line.

Nothing is limited unless configured — globally per key:

config :phoenix_kit_ai, concurrency: %{
  "provider:openrouter" => 8,            # all OpenRouter endpoints together
  "endpoint:Label reader" => 2           # one endpoint, by name
}

or per call, which wins over the configuration:

PhoenixKitAI.complete(ep, messages, concurrency: [limit: 4])            # the default key
PhoenixKitAI.complete(ep, messages, concurrency: [key: "batch", limit: 4, wait: 60_000])
PhoenixKitAI.complete(ep, messages, concurrency: false)                 # bypass

A waiting call can be told its place in line: concurrency: [notify: pid] sends {:phoenix_kit_ai_queue, key, position} (1 = next) to pid whenever it changes.

The slot covers the whole provider call, retries included (see PhoenixKitAI.Completion), so backing off after a 429 also keeps the pressure off. A caller that dies — crashed, killed, a closed LiveView — gives its slot or its place back (monitors). Limits are per node; in a cluster each node keeps its own count. Without the module's supervisor tree (PhoenixKitAI.children/0) every call runs straight through.

Options (concurrency: keyword or the app env :concurrency_defaults): wait (default 30_000 ms), max_waiting (default 1_000).