Skip to content

Add a codex-cli image provider: no-API-key generation via the signed-in Codex CLI#4129

Open
fancy-agent wants to merge 5 commits into
nexu-io:mainfrom
fancy-agent:feat/codex-cli-image-provider
Open

Add a codex-cli image provider: no-API-key generation via the signed-in Codex CLI#4129
fancy-agent wants to merge 5 commits into
nexu-io:mainfrom
fancy-agent:feat/codex-cli-image-provider

Conversation

@fancy-agent

Copy link
Copy Markdown

Fixes #4128

Why

I'm scratching my own itch: I run Open Design with Claude Code as the chat
agent, and I have a ChatGPT subscription with a signed-in Codex CLI — but no
OPENAI_API_KEY. Today that combination can't generate an image at all.

The only no-key image path that exists is PR #622's
renderCodexImagegenOverride, and it only fires when the chat agent is
Codex
. Drive OD with Claude Code / Gemini / Cursor and you're back to needing
a provider API key, even though the machine already has a perfectly good,
already-authenticated image generator sitting in the Codex CLI.

This adds a local codex-cli provider so any agent can generate images through
the operator's own Codex login — no OPENAI_API_KEY, billed to their ChatGPT
subscription.

What users will see

  • A new image model codex-image-gen (provider "Codex CLI") in the model
    picker, alongside gpt-image-2, dall-e-3, etc.
  • Selecting it generates images with no API key — it drives the local,
    signed-in Codex CLI's built-in image_gen tool.
  • It does not add a Settings → Media card and needs no stored credentials
    (same treatment as the hyperframes local renderer).
  • The default image model is unchanged (gpt-image-2); codex-image-gen is
    strictly opt-in via the picker.
  • If the Codex CLI isn't signed in to ChatGPT (or is pointed at a third-party
    relay where image_gen isn't mounted), generation fails with a clear,
    actionable error instead of a blank/placeholder image.

Surface area

  • UI — a new model appears in the New Project / model picker (no new
    page or dialog; it flows from the apps/web/src/media/models.ts registry)
  • Keyboard shortcut
  • CLI / env var
  • API / contract
  • Extension point
  • i18n keys
  • New top-level dependency
  • Default behavior change
  • None

Screenshots

Entry point — New project → Media → Image → "Model". The new
codex-image-gen model appears under a Codex CLI group marked
Configured (no key needed) and is selectable as the image model:

codex-image-gen selected in the New Project image model picker

Filtering the picker confirms it resolves with no credentials required:

codex-image-gen in the picker, marked Configured

Bug fix verification

  • Not a bug fix — this is a feature. (New failure-path tests are included
    anyway: see apps/daemon/tests/media-codex-cli.test.ts.)

Validation

Run under Node 24 / pnpm 10.33.2:

  • pnpm guard → pass (54/54)
  • pnpm typecheck → pass (all workspaces, including @open-design/web)
  • node scripts/verify-media-models.mjs → OK (TS + JS registries match)
  • pnpm --filter @open-design/daemon test → the new
    media-codex-cli.test.ts (5 cases) passes: success returns the real image
    bytes; the four failure branches (image_gen unavailable, clean-exit-but-no-
    file, non-PNG output, non-zero exit) all throw instead of falling back to a
    stub. Tests use a fake Codex CLI injected via CODEX_BIN — zero tokens.
  • Real generation (manual, live daemon, no OPENAI_API_KEY set):
    od media generate --project … --surface image --model codex-image-gen --prompt "a cozy reading nook with a cat, warm flat illustration" produced a
    real 2.58 MB PNG in the project. The artifact came back as
    providerId: "codex-cli", usedStubFallback: false, providerError: null,
    providerNote: "codex-cli/image_gen · 1:1 · 2579406 bytes · ChatGPT subscription (no API key)". Same path exercised through generateMedia
    directly with the real Codex CLI (independent file/sips confirmed a
    1254×1254 PNG). The image_gen smoke (orange-cat) under both workspace-write
    and the default config showed the built-in tool path with no
    scripts/image_gen.py / API-key fallback.

Notes / scope

  • The model picker has a per-surface provider allowlist in
    NewProjectPanel.tsx (supportedModels); codex-cli had to be added to the
    image set or the model stays hidden in the UI even though the daemon
    dispatches it — exactly how hyperframes is allowlisted for video. There's
    a regression test for this in apps/web/tests/components/NewProjectPanel.test.ts.
  • What I did not drive: a fully autonomous chat run where Claude Code (as
    the chat agent) decides to generate an image. The provider spawns its own
    Codex process and is decoupled from the chat agent, and od media generate --model codex-image-gen (the exact call any agent runtime makes) is verified
    end-to-end above — so agent-independence holds by construction — but I didn't
    script a full Claude-Code-in-the-loop session.
  • First version is t2i only — no i2i / inpaint / reference image yet.
  • Sandbox: uses the daemon's existing codex policy (workspace-write + network
    on macOS/Linux; danger-full-access only on Windows/WSL via the existing
    codexNeedsDangerFullAccessSandbox() helper). Verified image_gen works
    under workspace-write.
  • Not verified on Windows.
  • image_gen leaves no distinct event in codex exec --json, so success is
    confirmed from ground truth (exit 0 + the file exists at the path we handed
    codex + a PNG signature), and failures throw — no silent stub fallback (same
    policy as hyperframes).

Drives the operator's signed-in Codex CLI (its built-in image_gen tool) so any coding agent can generate images through `od media generate` with no OPENAI_API_KEY — the bytes bill against the user's ChatGPT subscription.

Today the only no-key image path is PR nexu-io#622's prompt override, which only fires when the chat agent is Codex itself. This adds a real media provider (credentialsRequired/settingsVisible false, like hyperframes) so the no-key path works for Claude Code, Gemini, and every other agent.

The daemon spawns a headless `codex exec --json` turn, has it copy the generated PNG to a temp workspace, and confirms success from ground truth (exit 0 + the file exists + a PNG signature); on any failure it throws rather than falling back to a stub. Registered in both media-model registries and added to the New Project image picker's supported-provider allowlist.
@lefarcen lefarcen requested review from Siri-Ray and elihahah666 June 11, 2026 04:50
@lefarcen lefarcen added size/L PR changes 300-700 lines risk/high High risk: apps/desktop, daemon, auth, migration, workflows, package deps type/feature New feature labels Jun 11, 2026

@Siri-Ray Siri-Ray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fancy-agent thanks for putting this together. I reviewed the daemon provider wiring, the model registries, and the picker visibility test path. I found one launch-environment issue worth tightening before this ships broadly, but I’m leaving it as non-blocking because it depends on how the daemon was launched and how Codex is installed.

🔁 Powered by Looper · runner=reviewer · agent=codex · An autonomous AI dev team for your GitHub repos.

Comment thread apps/daemon/src/media.ts Outdated
…efault

The codex-cli provider is credentialsRequired:false, so the model picker
treated it as always-ready and auto-selected codex-image-gen whenever no
API-key image provider was configured. That broke two existing contracts:
the picker must show "Pick a model" (submit no imageModel) when nothing is
eligible, and must switch to a configured provider's model — not codex-cli.

It also violated the feature's own rule: codex-cli spends the operator's
ChatGPT subscription, so it must be chosen on purpose, never a silent
default. Mark the provider optInOnly and skip such providers when picking
the auto-default; they stay fully pickable in the list. Adds a regression
test that codex-image-gen is offered but never auto-selected.

@Siri-Ray Siri-Ray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fancy-agent thanks for the follow-up here. I reviewed the current daemon provider wiring, the model registries, and the New Project picker change that keeps codex-image-gen explicit opt-in. The picker/default behavior looks covered; I still see one launch-environment issue in the daemon path that is worth tightening, but I’m keeping it non-blocking because it depends on desktop/packaged launch conditions and Codex install shape.

🔁 Powered by Looper · runner=reviewer · agent=codex · An autonomous AI dev team for your GitHub repos.

Comment thread apps/daemon/src/media.ts Outdated
…-launch PATH)

The provider resolved Codex with resolveAgentExecutable and spawned it with
the daemon's raw process.env. The repo's normal Codex launch path instead
uses resolveAgentLaunch + applyAgentLaunchEnv: the former swaps an npm
wrapper for the native binary, the latter prepends the running Node dir and
Codex toolchain dirs to the child PATH.

Without that symmetry a GUI-launched daemon (the desktop/packaged app, where
PATH is minimal) could resolve a shell-installed Codex but still fail at
spawn — its '#!/usr/bin/env node' shim can't find an interpreter. Mirror the
launcher: resolve via resolveAgentLaunch, spawn launch.launchPath with
env: applyAgentLaunchEnv(process.env, launch).

Adds a regression test that strips PATH of any node dir and points CODEX_BIN
at the node-shebang fake Codex, so the spawn only succeeds via the augmented
PATH the launcher injects. Addresses Looper review on media.ts.

@Siri-Ray Siri-Ray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fancy-agent thanks for the quick follow-up here. I verified the launcher-path fix on the current head, reviewed the daemon provider and picker changes again, and ran the focused daemon/web suites plus the media registry verifier. I found one remaining config-parity issue worth tightening, but I’m keeping it non-blocking because it only affects deployments that explicitly disable Codex plugins.

🔁 Powered by Looper · runner=reviewer · agent=codex · An autonomous AI dev team for your GitHub repos.

Comment thread apps/daemon/src/media.ts
…s for codex-image-gen

The codex-cli image provider only emitted passive errors AFTER running a
full (multi-minute) codex turn, so an API-key / programmatic / third-party
login wasted a turn before failing with a vague message.

- Add codex-image-auth.ts: read $CODEX_HOME/auth.json + config.toml and
  classify the login BEFORE spawning. Mirror codex's resolved_mode (OAuth
  tokens win, then OPENAI_API_KEY); fail fast on api-key / programmatic /
  third-party model_provider / not-signed-in with precise switch-to-
  ChatGPT-plan guidance. Schema matches openai/codex codex-rs/login
  (AuthDotJson + AuthMode lowercase serde).
- Split a non-zero codex exit into auth / quota / transient, each with an
  actionable message (quota = ChatGPT usage limit, ~5h reset). Transient
  keeps the original 'codex image_gen exited' wording.
- Tests: pure classifier + two auth.json fixtures (subscription vs api-key)
  + integration (fail-fast under api-key/not-signed-in, quota classified).

Verified: daemon tsc clean, vitest 29/29, pnpm guard 54/54, and the live
~/.codex (real ChatGPT subscription) classifies ok (no false-block).
@fancy-agent

Copy link
Copy Markdown
Author

Follow-up in f5422f3: the codex-cli image provider now fails fast on a non-subscription login before spawning a turn, and splits runtime failures into actionable classes.

Previously auth problems only surfaced after a full (multi-minute) codex exec turn — an API-key or third-party-gateway login burned the whole turn just to land on a vague "no image" error. The new codex-image-auth.ts reads $CODEX_HOME/auth.json + config.toml up front and classifies the login the same way Codex's own resolved_mode does (OAuth tokens first, then OPENAI_API_KEY; schema taken from codex-rs/login AuthDotJson + the lowercase AuthMode serde). API-key / programmatic / third-party model_provider / not-signed-in each fail fast with specific "switch to a ChatGPT-plan login" guidance.

A non-zero codex exit is also split into auth / quota / transient — a ChatGPT usage-limit (the image_gen tool shares your plan quota, ~5h reset) now reads differently from a network blip; transient keeps the original wording.

Verified: daemon tsc clean, vitest 29/29 (two auth.json fixtures + fail-fast/quota integration tests, RED-checked against a reverted wiring), pnpm guard 54/54, and my real ~/.codex ChatGPT subscription classifies ok (no false-block of a legitimate user).

@Siri-Ray Siri-Ray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fancy-agent thanks for the quick follow-up on the auth classification path. I reviewed the current daemon provider/auth gate, model registry sync, and New Project picker behavior. The new fail-fast auth handling and opt-in picker behavior look on track; I’m leaving one non-blocking config-parity issue because this media-specific Codex spawn still bypasses the existing global plugin-disable knob.

🔁 Powered by Looper · runner=reviewer · agent=codex · An autonomous AI dev team for your GitHub repos.

Comment thread apps/daemon/src/media.ts
runCodexImageGen mirrored codexAgentDef.buildArgs' sandbox and
default-permissions args but skipped the OD_CODEX_DISABLE_PLUGINS=1 branch, so
an operator who globally disabled Codex plugins would still get a plugin-enabled
Codex turn on this image-gen path while it handles user prompt input.

Append the same '--disable plugins' pair when the env var is set, in the same
position as runtimes/defs/codex.ts. Extend media-codex-cli.test.ts: the fake
Codex now records its argv, and two tests lock both directions (forwarded when
the var is '1', absent when unset).
@lefarcen lefarcen added the needs-validation Runtime change detected; needs human or /explore agent validation. label Jun 11, 2026

@Siri-Ray Siri-Ray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fancy-agent thanks for the thoughtful follow-through on this provider. I reviewed the changed daemon image-provider path, the Codex auth/failure classification, launcher argument parity, the media model registries, and the New Project picker behavior. The current head addresses the earlier launcher and plugin-disable parity concerns, keeps codex-image-gen opt-in rather than a silent default, and has focused coverage for the main success/failure paths. I also ran the media registry verifier successfully; the focused Vitest commands could not run in this reviewer worktree because node_modules/vitest is not installed here. Nice work getting the no-API-key path wired through cleanly.

🔁 Powered by Looper · runner=reviewer · agent=codex · An autonomous AI dev team for your GitHub repos.

@lefarcen lefarcen requested a review from chaoxiaoche June 11, 2026 11:27
@lefarcen lefarcen removed the needs-validation Runtime change detected; needs human or /explore agent validation. label Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk/high High risk: apps/desktop, daemon, auth, migration, workflows, package deps size/L PR changes 300-700 lines type/feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a codex-cli image provider: no-API-key generation via the signed-in Codex CLI

3 participants