# `PhoenixKitAI.Images`
[🔗](https://github.com/BeamLabEU/phoenix_kit_ai/blob/0.23.1/lib/phoenix_kit_ai/images.ex#L1)

Generic image processing over any provider: named operations composed
into one edit, vision (describe an image, optionally as JSON), and a
before/after check.

Everything takes an endpoint and returns bytes. The module stores
nothing — the caller keeps the result wherever it keeps files (core's
Storage, usually). The logged request row is the only trace.

## Editing by operation

    PhoenixKitAI.Images.process(endpoint, [photo_jpeg], [
      :remove_reflections,
      {:clean_background, color: "light grey"},
      {:upscale, resolution: "2K"}
    ])

Operations (`PhoenixKitAI.Images.Operations`) become one numbered
instruction list, followed by a preservation clause
(`preserve: :subject` by default — the subject, its text and logos stay
as they are; `:scene` for room photos; `false` for none; or your own
sentence). Options the operations imply (`:remove_background` wants
`background: "transparent"` + PNG) merge under the caller's options and
are then fitted to the model: an option the model does not list is
dropped with a warning, or refused with `strict: true`. When an
operation loses its implied options it falls back to its plain-prompt
wording (a white background instead of a transparent one).

## Options

Canonical request options, mapped per provider by the adapter:
`:aspect_ratio`, `:resolution`, `:size`, `:quality`, `:background`,
`:output_format`, `:output_compression`, `:n`, `:seed`,
`:response_format`, `:style`, `:mask`. Control options: `:model` (override the endpoint's
model — e.g. to compare two OpenRouter models on one endpoint),
`:transport`, `:provider_options` (passed through untouched),
`:provider_routing` (OpenRouter), `:strict`, `:preserve`, `:finish`,
`:prompt_overrides`.

Endpoint defaults apply underneath: `provider_settings["aspect_ratio"]`
/ `["resolution"]`, and the `image_size` / `image_quality` columns.

## Vision

    {:ok, %{json: %{"brand" => _}}} =
      PhoenixKitAI.Images.describe(endpoint, [label_jpeg],
        prompt: "Read the label.",
        schema: %{"type" => "object", "properties" => %{"brand" => %{"type" => "string"}}})

`describe/3` runs a chat completion with the images attached; with
`schema:` (a JSON Schema map) or `json: true` the answer is parsed and
returned under `:json`. `compare/4` is a fixed-schema `describe/3` over
an original and its edit, answering whether the subject survived.

# `error`

```elixir
@type error() ::
  :empty_input
  | :invalid_image_input
  | :reference_image_required
  | :not_supported
  | {:image_too_large, pos_integer(), pos_integer()}
  | {:unsafe_url, String.t()}
  | {:fetch_failed, String.t(), term()}
  | {:unknown_operation, term()}
  | {:missing_parameter, atom(), atom()}
  | {:conflicting_operations, atom(), atom()}
  | {:unsupported_option, atom(), term()}
  | {:too_many_images, pos_integer(), pos_integer()}
  | {:model_not_listed, String.t()}
  | {:capabilities_unavailable, term()}
  | {:no_image_in_response, String.t() | nil}
  | {:no_json_in_response, String.t() | nil}
  | {:content_policy, String.t()}
  | atom()
  | {atom(), term()}
```

Every error the image verbs return, so a caller can match exhaustively.
Transport and provider errors come from `PhoenixKitAI.Completion`
(`:rate_limited`, `:request_timeout`, `{:api_error, status}`,
`{:connection_error, reason}`, …).

# `input`

```elixir
@type input() ::
  binary()
  | String.t()
  | %{data: binary(), content_type: String.t()}
  | %{url: String.t()}
```

# `plan`

```elixir
@type plan() :: %{
  prompt: String.t(),
  operations: [atom()],
  warnings: [term()],
  model: String.t() | nil,
  options: map(),
  images: [],
  text: nil,
  usage: map(),
  latency_ms: 0,
  dry_run: true
}
```

What `process/4` returns for `dry_run: true`: the plan, no images.

# `result`

```elixir
@type result() :: %{
  images: [PhoenixKitAI.Provider.image()],
  text: String.t() | nil,
  usage: map(),
  latency_ms: non_neg_integer(),
  model: String.t() | nil,
  prompt: String.t(),
  operations: [atom()],
  warnings: [term()]
}
```

# `build_prompt`

```elixir
@spec build_prompt([term()], keyword()) :: {:ok, String.t()} | {:error, term()}
```

The prompt `process/4` sends for `operations` (normalised or not):
numbered instructions, the preservation clause (`preserve:`), the
closing line (`finish: false` drops it). Public so admin pages can
preview it and tests can pin it.

# `compare`

```elixir
@spec compare(PhoenixKitAI.Endpoint.t(), input(), input(), keyword()) ::
  {:ok, map()} | {:error, error()}
```

Checks an edit against its original with a vision model: did the
subject, its text and logos survive, and what changed that was not
asked for. `intent:` describes the edit that was requested.

Returns `{:ok, %{passed: boolean, same_subject, text_and_logos_preserved,
unwanted_changes, summary, usage, latency_ms, model}}`.

# `default_question`

```elixir
@spec default_question() :: String.t()
```

The question `describe/3` asks when the caller gives none.

# `describe`

```elixir
@spec describe(PhoenixKitAI.Endpoint.t(), [input()], keyword()) ::
  {:ok, map()} | {:error, error()}
```

Asks a vision-capable chat endpoint about `images`.

Options: `:prompt` (default "Describe this image in detail."),
`:system`, `:schema` (JSON Schema map → the answer is requested and
parsed as that object), `:json` (true → any JSON object), `:model`,
and the chat sampling options (`:temperature`, `:max_tokens`, `:top_p`,
`:seed`).

Returns `{:ok, %{text, json, usage, latency_ms, model}}`; `json` is nil
unless requested. A JSON answer that does not parse is
`{:error, {:no_json_in_response, text}}`; `:on_no_json`, a one-arity
function, first receives the unparsed result (usage and `:prompt`
included) so the paid call can still be logged.

# `dimensions`

```elixir
@spec dimensions(binary()) :: {:ok, {pos_integer(), pos_integer()}} | :error
```

Pixel dimensions from a PNG, JPEG, WebP or GIF header, without decoding.

# `extract_text`

```elixir
@spec extract_text(PhoenixKitAI.Endpoint.t(), [input()] | input(), keyword()) ::
  {:ok, map()} | {:error, error()}
```

Reads the text in `images` with a vision model (OCR without an OCR
engine): a product label, a receipt, a sign, a page of a document.

Returns `{:ok, %{text, blocks, language, confidence, has_illegible_text,
fields, json, prompt, usage, latency_ms, model}}` — `text` is everything
transcribed in reading order, `blocks` splits it into typed pieces
(`%{text, kind, language, page}`, kind one of heading / paragraph /
label / list / table / caption / code / handwriting / other), `fields`
holds every requested name (`nil` when the model found nothing, string
keys as given), `has_illegible_text` says some print was visible but
unreadable (retake the photo rather than trust the gaps), `json` is the
raw object. Several images are read as pages of one document, in order;
`page` says which image a block came from.

The prompt forbids guessing: vision models will otherwise invent
plausible label text (an ingredients line, a net weight) for print
they cannot resolve, at full confidence.

Options: `fields:` — a map of name → description of a value to pull
out (`%{"ean" => "the barcode digits", "best_before" => "expiry date as
YYYY-MM-DD"}`, or a list of names), `language:` — a hint when the
script is ambiguous, `layout: :markdown` — keep tables and lists as
Markdown instead of plain lines, `instructions:` — extra wording for the
prompt, `schema:` — replace the base schema (`fields:` is still added to
it), plus `describe/3`'s `:model`, `:system` and sampling options.

# `fetch_outputs`

```elixir
@spec fetch_outputs(map(), keyword()) :: {:ok, map()}
```

Downloads output images a provider returned as URLs (xAI, OpenAI's
`response_format: "url"`) so callers always get bytes, and adds
`width` / `height` to every image whose header can be read. A URL that
cannot be fetched stays a URL, with an `{:output_not_fetched, url,
reason}` warning. `fetch_outputs: false` skips the download.

# `fit_options`

```elixir
@spec fit_options(
  map(),
  PhoenixKitAI.Images.ImageModel.t() | nil,
  [atom()],
  boolean()
) ::
  {:ok, map(), [term()]} | {:error, {:unsupported_option, atom(), term()}}
```

Keeps `options` inside what the adapter can send (`adapter_options`)
and, when there is a model listing, what the model accepts. Returns the
fitted options and a warning per change, `{:dropped_option, key,
value}`, in the canonical option order; with `strict: true` the first
offender is an error instead.

# `normalize_inputs`

```elixir
@spec normalize_inputs([input()], keyword()) ::
  {:ok, [String.t()]} | {:error, error()}
```

Turns caller inputs into what adapters take: data URLs (bytes inlined)
or http(s) URLs. Accepts raw bytes (type sniffed), `%{data, content_type}`,
`%{url}`, and bare `data:` / `http(s):` strings. Never hand a provider a
permanent URL it could cache — pass bytes.

# `normalize_mask`

```elixir
@spec normalize_mask(map(), input() | nil, keyword()) ::
  {:ok, map()} | {:error, error()}
```

Normalises a `mask:` input into the request option adapters receive (a
data URL) — the OpenAI adapter sends it as the `mask` file, the others
drop it with a `{:dropped_option, :mask, _}` warning. Used by
`process/4` and by the thin `PhoenixKitAI.Completion.edit_image/4`.

# `options`

```elixir
@spec options(PhoenixKitAI.Endpoint.t(), keyword() | map()) :: map()
```

The option map an adapter receives: the endpoint's stored defaults
underneath the caller's keyword options, canonical keys only.

# `process`

```elixir
@spec process(PhoenixKitAI.Endpoint.t(), [input()], [term()], keyword()) ::
  {:ok, result() | plan()} | {:error, error()}
```

Edits `images` (the first is the subject, the rest are references)
with the given `operations` on `endpoint`. See the moduledoc.

# `redact_warning`

```elixir
@spec redact_warning(term()) :: term()
```

A warning with any long binary payload replaced by its size — a dropped
`:mask` carries a whole data URL; logs and screens get
`{:dropped_option, :mask, {:bytes, 12345}}` instead.

# `request_options`

```elixir
@spec request_options() :: [atom()]
```

The canonical request option names.

# `text_schema`

```elixir
@spec text_schema() :: map()
```

The JSON Schema `extract_text/3` asks the model to fill (before `fields:` are added).

---

*Consult [api-reference.md](api-reference.md) for complete listing*