# `PhoenixKitAI.Limiter`
[🔗](https://github.com/BeamLabEU/phoenix_kit_ai/blob/v0.26.1/lib/phoenix_kit_ai/limiter.ex#L1)

Caps how many provider calls run at once per key, and lines up the rest in
arrival order.

A provider rate-limits a burst (429) long before a spend cap matters, and a
batch job that fires 50 calls at once mostly buys retries. The limiter gives a
key (by default the endpoint's provider, `"provider:openrouter"`) a number of
slots; a call that finds them taken waits its turn, and gives up with
`{:error, :concurrency_limit_timeout}` after `wait` milliseconds or with
`{:error, :concurrency_queue_full}` when `max_waiting` calls are already in line.

Nothing is limited unless configured — globally per key:

    config :phoenix_kit_ai, concurrency: %{
      "provider:openrouter" => 8,            # all OpenRouter endpoints together
      "endpoint:Label reader" => 2           # one endpoint, by name
    }

or per call, which wins over the configuration:

    PhoenixKitAI.complete(ep, messages, concurrency: [limit: 4])            # the default key
    PhoenixKitAI.complete(ep, messages, concurrency: [key: "batch", limit: 4, wait: 60_000])
    PhoenixKitAI.complete(ep, messages, concurrency: false)                 # bypass

A waiting call can be told its place in line: `concurrency: [notify: pid]` sends
`{:phoenix_kit_ai_queue, key, position}` (1 = next) to `pid` whenever it changes.

The slot covers the whole provider call, retries included (see
`PhoenixKitAI.Completion`), so backing off after a 429 also keeps the pressure
off. A caller that dies — crashed, killed, a closed LiveView — gives its slot or
its place back (monitors). Limits are per node; in a cluster each node keeps its
own count. Without the module's supervisor tree (`PhoenixKitAI.children/0`) every
call runs straight through.

Options (`concurrency:` keyword or the app env `:concurrency_defaults`):
`wait` (default 30_000 ms), `max_waiting` (default 1_000).

# `child_spec`

Returns a specification to start this module under a supervisor.

See `Supervisor`.

# `limit_for`

```elixir
@spec limit_for(map(), keyword()) ::
  {String.t() | term(), pos_integer(), keyword()} | nil
```

The `{key, limit, settings}` that applies, or nil when the call is not limited.

# `run`

```elixir
@spec run(map(), keyword(), (-> term())) :: term()
```

Runs `fun` under the limit that applies to `endpoint` and `opts`; returns what
`fun` returns, or `{:error, :concurrency_limit_timeout | :concurrency_queue_full}`.

# `start_link`

# `stats`

```elixir
@spec stats(term()) :: %{
  optional(:limit) => pos_integer(),
  running: non_neg_integer(),
  waiting: non_neg_integer()
}
```

Current state of one key: slots, running, waiting (zeros for an unknown key).

---

*Consult [api-reference.md](api-reference.md) for complete listing*