Skip to content

Commit 46be541

Browse files
authored
Merge pull request #6534 from ClickHouse/postgres-cdc-json
Update FAQ around native JSON support in Postgres CDC ClickPipes
2 parents db4d7c2 + aca8f4c commit 46be541

1 file changed

Lines changed: 7 additions & 2 deletions

File tree

  • docs/integrations/data-ingestion/clickpipes/postgres

docs/integrations/data-ingestion/clickpipes/postgres/faq.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -164,9 +164,14 @@ ClickPipes for Postgres aims to map Postgres data types as natively as possible
164164

165165
Currently, we don't support defining custom data type mappings as part of the pipe. However, note that the default data type mapping used by ClickPipes is highly native. Most column types in Postgres are replicated as closely as possible to their native equivalents on ClickHouse. Integer array types in Postgres, for instance, are replicated as integer array types on ClickHouse.
166166
167-
### How are JSON and JSONB columns replicated from Postgres? {#how-are-json-and-jsonb-columns-replicated-from-postgres}
167+
### How are `json` and `jsonb` columns replicated from Postgres? {#how-are-json-and-jsonb-columns-replicated-from-postgres}
168168
169-
JSON and JSONB columns are replicated as String type in ClickHouse. Since ClickHouse supports a native [JSON type](/sql-reference/data-types/newjson), you can create a materialized view over the ClickPipes tables to perform the translation if needed. Alternatively, you can use [JSON functions](/sql-reference/functions/json-functions) directly on the String columns. We're actively working on a feature that replicates JSON and JSONB columns directly to the JSON type in ClickHouse. This feature is expected to be available in a few months.
169+
`json` and `jsonb` columns are replicated as String type in ClickHouse due to incompatibilities with the [native JSON type](https://clickhouse.com/docs/sql-reference/data-types/newjson). For example:
170+
171+
* PostgreSQL allows any valid JSON value at the top level (strings, numbers, arrays), while ClickHouse's JSON type only supports objects.
172+
* Keys containing dots (e.g., "app.kubernetes.io/name") are also interpreted as nested paths by ClickHouse's JSON type, which could alter the data structure.
173+
174+
To handle JSON strings downstream of a Postgres CDC ClickPipe, you can use [JSON functions](https://clickhouse.com/docs/sql-reference/functions/json-functions) directly on the columns to extract and work with the data. For better query performance, you can also use a materialized view to extract specific fields into typed columns, or to cast the column to the native JSON type (keeping in mind that the cast might fail if it hits a cross-platform type incompatibility).
170175
171176
### What happens to inserts when a mirror is paused? {#what-happens-to-inserts-when-a-mirror-is-paused}
172177

0 commit comments

Comments
 (0)