Skip to content

fix: configure filtered Debezium publications#4008

Closed
vpol wants to merge 1 commit into
mainfrom
codex/debezium-filtered-heartbeat
Closed

fix: configure filtered Debezium publications#4008
vpol wants to merge 1 commit into
mainfrom
codex/debezium-filtered-heartbeat

Conversation

@vpol

@vpol vpol commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What changed

  • adds a database-backed Debezium heartbeat model and migration
  • creates explicit filtered API and ClickHouse publications with publish_generated_columns = stored
  • disables connector-side publication autocreation and points both connectors at the new publications
  • excludes post.views so view-only updates can be skipped, and excludes generated post.slug
  • executes heartbeat upserts and filters heartbeat row events from the application SMT pipeline

Why

After the separate publication rollout, PostgreSQL 18 rejected updates to post with Replica identity must not contain unpublished generated columns. The table uses REPLICA IDENTITY FULL and slug is a stored generated column, while connector-created publications do not enable generated-column publishing.

The publications are now migration-owned so they can explicitly publish stored generated columns. Publication setup includes only configured tables that exist, which handles removed legacy tables such as user_state.

Validation

  • complete 538-migration chain against PostgreSQL 18
  • publication metadata verified with pubgencols = stored
  • UPDATE post SET views = views WHERE false succeeds without SQLSTATE 42P10
  • heartbeat upsert succeeds
  • filtered-publication migration revert and reapply
  • pnpm run build
  • pnpm run lint
  • git diff --check

@vpol vpol closed this Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant