Skip to content

pg push service#547

Open
mxfeinberg wants to merge 29 commits into
kenn-io:mainfrom
mxfeinberg:pg-push-service
Open

pg push service#547
mxfeinberg wants to merge 29 commits into
kenn-io:mainfrom
mxfeinberg:pg-push-service

Conversation

@mxfeinberg
Copy link
Copy Markdown

@mxfeinberg mxfeinberg commented May 26, 2026

Problem

I track token usage across AI tools with agentsview, and lately I've been
running more remote agent tools on machines other than my primary dev box.
Each of those machines records its own sessions locally. The Postgres support
(pg push / pg serve) already lets me review everything together, but
keeping the shared database current meant either remembering to run pg push
by hand on each machine or wiring up a bespoke cron job per host. I wanted a
touchless way to sync data from multiple machines into one place so remote
agent token usage shows up in my primary dashboard automatically, without
maintaining custom cron entries everywhere.

I also made some updates to hermes specific code because although tokens
were being passed through, the associated costs were not being aggregated
properly.

pg push service

A long-running auto-push daemon and a first-class way to install it as an OS
service, so a recorder machine keeps the shared Postgres current on its own.

agentsview pg push --watch start a foreground daemon that pushes to Postgres
shortly after sessions change, with a periodic floor as a safety net:

  • A file watcher (reusing the same machinery as serve) coalesces change
    events over a debounce window (--debounce, default 30s) and triggers a
    local sync + incremental push.
  • A periodic floor (--interval, default 15m) pushes regardless of file
    events and covers any directories the watch budget couldn't cover.
  • Pushes are serialized by a single goroutine and connect to Postgres lazily,
    reconnecting on error. A transiently unreachable database is logged and
    retried on the next trigger rather than crashing the daemon.
  • A single-instance lock prevents two watchers from racing on the push
    watermarks. On shutdown (SIGINT/SIGTERM) it performs one bounded final flush.
  • It reads the same [pg] config and project filters as pg push, and
    honors result_content_blocked_categories so the push path no longer
    diverges from serve.

agentsview pg service install|uninstall|status|start|stop|logs
installs and manages the daemon as a per-user OS service:

  • launchd LaunchAgent on macOS, systemd --user unit on Linux, behind a small
    platform abstraction with pure (golden-tested) unit-file rendering.
  • install validates that the Postgres DSN is resolvable before creating the
    service, and surfaces the systemd linger requirement so the service keeps
    running on headless boxes after logout.
  • status reports the manager state plus the last successful push time;
    logs -f tails the daemon log and survives log rotation.

The daemon reads the DSN from the existing config file (no credentials are
copied into unit files), so protecting config.toml is sufficient.

Notes

  • Platform support targets macOS (launchd) and Linux (systemd --user); the
    daemon command itself is cross-platform Go.
  • The generated unit always pins AGENTSVIEW_DATA_DIR to the install-time data
    dir, so the service resolves the same config regardless of the environment it
    starts in.
  • The watch daemon's initial resync is push-oriented and skips the serve-side
    Vacuum/BackfillSignals steps, since signal recomputation happens on the
    serve side.
  • A windows service was not added because I don't currently have access to a
    windows machine that I could use for testing.

mxfeinberg and others added 23 commits May 25, 2026 16:10
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sts)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a defaultFlushTimeout (30 s) and a flushTimeout field on pushLoop so
a SIGTERM'd daemon cannot hang indefinitely on a stalled PostgreSQL
connection during the final shutdown-flush push. Zero means unbounded,
preserving the existing test behaviour where newTestLoop leaves the field
unset.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…op/logs)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… and promptYesNo

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… --watch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…note KeepAlive tradeoff

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@roborev-ci
Copy link
Copy Markdown

roborev-ci Bot commented May 26, 2026

roborev: Combined Review (6a35195)

No Medium, High, or Critical issues found.

All review outputs are clean; no actionable findings to include.


Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

@roborev-ci
Copy link
Copy Markdown

roborev-ci Bot commented May 26, 2026

roborev: Combined Review (5101e30)

High confidence issues remain; PR should not merge until the import break is fixed.

High

  • cmd/agentsview/pg_service.go:14
    New files import go.kenn.io/agentsview/internal/..., but the module is github.com/wesm/agentsview, with no module-path change in the diff. These packages will not resolve.
    Fix: Use the repository’s existing import path, e.g. github.com/wesm/agentsview/internal/config, in all new files.

Medium

  • cmd/agentsview/pg_service_manager.go:26
    buildServiceSpec validates pg.url using the installer’s current environment, but the rendered launchd/systemd units only preserve AGENTSVIEW_DATA_DIR. A config like pg.url = "${PG_URL}" installs successfully, then the service fails at runtime because PG_URL is absent.
    Fix: Either reject env-dependent PG URLs during service install or render the required PG environment variables into the service unit.

  • internal/pidlock/pidlock.go:26
    Acquire is not atomic: two concurrent processes can both see no live lock, write the shared .tmp file, rename over the lock path, and both return success.
    Fix: Acquire with an atomic primitive such as os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, ...), removing stale locks and retrying as needed.


Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

@roborev-ci
Copy link
Copy Markdown

roborev-ci Bot commented May 26, 2026

roborev: Combined Review (93270cc)

Medium-risk issues remain around single-instance locking and service environment propagation.

Medium

  • internal/pidlock/pidlock.go:41
    Acquire is not atomic: after checking/removing a stale lock, it writes a temp file and renames it over the target. Two near-simultaneous starts can both acquire the “single-instance” lock, allowing multiple pg push --watch daemons and potentially concurrent resyncs against the same temp DB path.
    Fix: Create the lock with an atomic primitive such as os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, ...) or an OS file lock, and avoid releasing a lock file that no longer belongs to the current process.

  • cmd/agentsview/pg_service_manager.go:27
    Service install validates the resolved PG URL using the current environment, but the generated launchd/systemd units only preserve AGENTSVIEW_DATA_DIR. Configurations that use AGENTSVIEW_PG_URL or ${VAR} expansion can pass install and then fail when the background service starts without those variables.
    Fix: Persist the required resolved PG settings into the service environment/unit, support an environment file, or reject env-dependent PG URLs during service install with a clear error.


Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

@roborev-ci
Copy link
Copy Markdown

roborev-ci Bot commented May 26, 2026

roborev: Combined Review (9b511a2)

High severity issue found.

High

  • cmd/agentsview/pg_service_systemd.go:40

    systemdManager.render interpolates spec.BinPath and spec.DataDir directly into a systemd unit using quoted strings. systemd unit files are not shell scripts, but newlines and quote characters inside these values can break out of the intended directive and inject additional unit directives.

    Since spec.DataDir comes from configuration, a crafted data dir containing quote/newline-injected directives could cause agentsview pg service install to write, enable, and start a user service that runs attacker-chosen commands as the current user.

    Suggested remediation: escape or quote values using systemd-safe syntax, or reject unsafe path characters such as \n, \r, ", and control bytes before writing the unit. Cover both ExecStart and Environment with tests.


Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

Reject control characters and double quotes in BinPath/DataDir/LogPath at
the buildServiceSpec chokepoint so a crafted data dir cannot break out of a
quoted value and inject directives into the systemd unit (ExecStart,
Environment) or launchd plist.

Export config.IsEnvDependentURL and reuse it in the service manager so the
env-dependent pg.url rejection shares expandBracedEnv's exact patterns and
cannot silently desync.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@roborev-ci
Copy link
Copy Markdown

roborev-ci Bot commented May 26, 2026

roborev: Combined Review (1929c73)

High: PR has compile-breaking import path issues and a service environment persistence bug that can make installed services run with different config than validated.

High

  • cmd/agentsview/pg_watch.go:13
    The new command files import internal packages via go.kenn.io/agentsview/..., but this repository’s module path is github.com/wesm/agentsview. Because these are internal packages, the mismatched import path will break compilation for the added watch/service code.
    Fix: Change the new imports in pg_watch.go, pg_service.go, pg_service_manager.go, and their tests to github.com/wesm/agentsview/internal/....

Medium

  • cmd/agentsview/pg_service_manager.go:20
    Service installation only rejects env-dependent pg.url, but the installed service environment only preserves AGENTSVIEW_DATA_DIR. If the install-time config depends on other env vars such as AGENTSVIEW_PG_SCHEMA, AGENTSVIEW_PG_MACHINE, or agent session directory env vars, foreground install validation can succeed while the background service later runs with different schema, machine, or source directories.
    Fix: Either reject all env-derived config that affects service runtime, or render the required resolved environment values into both launchd and systemd units.

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

The background service unit only carries AGENTSVIEW_DATA_DIR, so env vars
that affect runtime behavior (AGENTSVIEW_PG_SCHEMA, AGENTSVIEW_PG_MACHINE,
agent dir overrides) set in the installing shell are not seen by the
service, which instead uses config.toml or defaults. Install now warns
about any such vars that are set so the divergence is not silent. This is
advisory only and does not change the intended environment-overrides-config
semantics; AGENTSVIEW_PG_URL remains a hard rejection since a missing URL
breaks the service outright.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@roborev-ci
Copy link
Copy Markdown

roborev-ci Bot commented May 26, 2026

roborev: Combined Review (b559140)

No Medium, High, or Critical findings were reported.

All completed reviews found no actionable issues; one review was skipped due to quota exhaustion.


Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

Note: gemini review skipped (agent quota exhausted)

@mxfeinberg mxfeinberg marked this pull request as ready for review May 26, 2026 21:57
@roborev-ci
Copy link
Copy Markdown

roborev-ci Bot commented May 26, 2026

roborev: Combined Review (bfd3d7d)

No Medium, High, or Critical findings were reported.

All completed reviews found no issues; one review was skipped due to quota exhaustion.


Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

Note: gemini review skipped (agent quota exhausted)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant