Connector Name
destination-snowflake
Connector Version
4.0.31
What step the error happened?
During the sync
Relevant information
During a routine hourly CDC sync, a transient Snowflake API error during the destination's pre-sync reconciliation check caused the connector to rebuild one stream's table from scratch with CREATE OR REPLACE TABLE, instead of appending to it as normal. Because the MySQL CDC source's saved binlog cursor was untouched by this, the very next read only pulled the small number of rows changed since the last checkpoint — not a full resync — so the destination table was left permanently short of its prior history, with no error, warning, or failed sync status to flag it.
Sequence of events (job 44345, stream byod-orders.fieldservicecases → RAW.SP2_CDC_RAW.FIELDSERVICECASES):
11:01:36 — Destination issues its normal pre-sync reconciliation query: SELECT COUNT(*) AS TOTAL FROM "RAW"."SP2_CDC_RAW"."FIELDSERVICECASES"
11:01:52 — That request fails after exhausting all retries: Stop retrying as max retries have been reached ... Max retry count: 7, with the underlying error being HTTP Response code: 503 from Snowflake.
11:02:14 — The stream loader for fieldservicecases starts, but instead of the normal DESCRIBE TABLE → append path used for every other stream in the same job (~269 tables), it goes straight to: CREATE OR REPLACE TABLE "RAW"."SP2_CDC_RAW"."FIELDSERVICECASES" (...) — no DESCRIBE TABLE call is logged for this table anywhere in the job.
On the read side, this stream also never gets the per-stream Attempting bootstrap using class io.airbyte.cdk.read.cdc.CdcPartitionsCreatorFactory line that every other stream gets in a normal run — meaning no full resnapshot was considered or triggered for it.
The sync summary shows only recordsEmitted: 116 / recordsCommitted: 116 for fieldservicecases — consistent with a normal incremental delta from the shared Debezium binlog reader (records-emitted=1014 total across the whole sync), not a fresh backfill.
The job completed with a SUCCEEDED status. There was no indication anywhere in the job history or UI that the destination table had just been wiped and was now missing its historical rows.
Why this looks like a bug rather than intended behavior:
The destination's response to a failed reconciliation check (an Snowflake-side transient 503) was to silently rebuild the table via CREATE OR REPLACE, discarding all existing rows, rather than failing the sync/stream and retrying, or leaving the existing table untouched.
The MySQL CDC source has no visibility into destination-side table state, so it had no way to know a resync was needed — it just kept incrementing from its last saved binlog position. The result is a silent, partial data loss with a "succeeded" sync status.
Impact: RAW.SP2_CDC_RAW.FIELDSERVICECASES currently contains only the rows synced since 2026-08-06 11:02, having previously held its full history. This was only caught because we happened to notice unexpected CREATE OR REPLACE queries in Snowflake's query history and traced them back through the Airbyte job logs.
Expected behavior
One of the following, rather than a silent partial rebuild:
If the reconciliation check fails, the sync/stream should fail loudly (or retry the check) rather than falling back to CREATE OR REPLACE.
If a CREATE OR REPLACE is genuinely necessary (e.g. schema drift the connector can't reconcile another way), it should trigger a full resnapshot of that stream from the source so the destination table ends up complete — not just resume from the existing incremental cursor.
Relevant log output
2026-08-06 11:01:36 info INFO main i.a.i.d.s.s.SnowflakeDirectLoadSqlGeneratorKt(andLog):33 SELECT COUNT(*) AS TOTAL FROM "RAW"."SP2_CDC_RAW"."FIELDSERVICECASES"
2026-08-06 11:01:52 error ERROR main n.s.c.j.RestRequest(handleMaxRetriesExceeded):1535 [requestId=056b3732-fa61-4923-88e6-666cbf6801da] Stop retrying as max retries have been reached for request: POST https://<account>.snowflakecomputing.com:443/queries/v1/query-request?requestId=056b3732-fa61-4923-88e6-666cbf6801da HTTP/1.1! Max retry count: 7
2026-08-06 11:01:52 error ERROR main n.s.c.j.RestRequest(executeWithRetries):1011 [requestId=056b3732-fa61-4923-88e6-666cbf6801da] Error response: HTTP Response code: 503, request: POST https://<account>.snowflakecomputing.com:443/queries/v1/query-request?requestId=056b3732-fa61-4923-88e6-666cbf6801da HTTP/1.1
2026-08-06 11:01:52 info INFO main i.a.i.d.s.s.SnowflakeDirectLoadSqlGeneratorKt(andLog):33 SELECT COUNT(*) AS TOTAL FROM "RAW"."airbyte_internal"."SP2_CDC_RAWFIELDSERICECASES1ac49549844176b53823b480c4db53b0"
... (~40s later, ~269 other streams meanwhile go through the normal DESCRIBE TABLE -> append path) ...
2026-08-06 11:02:14 info INFO DefaultDispatcher-worker-2 i.a.c.l.d.DestinationLifecycle$initializeIndividualStreams$1$result$1$1(invokeSuspend):64 Starting stream loader for stream SP2_CDC_RAW:fieldservicecases
2026-08-06 11:02:14 info INFO DefaultDispatcher-worker-2 i.a.c.l.o.d.d.DirectLoadTableAppendStreamLoader(start):38 AppendStreamLoader starting for stream SP2_CDC_RAW.fieldservicecases
2026-08-06 11:02:14 info INFO DefaultDispatcher-worker-2 i.a.i.d.s.s.SnowflakeDirectLoadSqlGeneratorKt(andLog):33 CREATE OR REPLACE TABLE "RAW"."SP2_CDC_RAW"."FIELDSERVICECASES" (
"_AIRBYTE_RAW_ID" VARCHAR NOT NULL,
"_AIRBYTE_EXTRACTED_AT" TIMESTAMP_TZ NOT NULL,
"_AIRBYTE_META" VARIANT NOT NULL,
"_AIRBYTE_GENERATION_ID" NUMBER(38,0),
"ID" NUMBER(38,0),
... (full column list omitted for brevity) ...
)
2026-08-06 11:02:15 info INFO DefaultDispatcher-worker-2 i.a.i.d.s.s.SnowflakeDirectLoadSqlGeneratorKt(andLog):33 CREATE STAGE IF NOT EXISTS "RAW"."SP2_CDC_RAW"."airbyte_stage_FIELDSERVICECASES"
... (note: no DESCRIBE TABLE call for FIELDSERVICECASES anywhere in this job) ...
2026-08-06 11:02:22 info INFO DefaultDispatcher-worker-13#byod-orders.fieldservicecases i.a.c.r.FeedReader(createPartitions):107 Attempting bootstrap using class io.airbyte.integrations.source.mysql.MySqlJdbcConcurrentPartitionsCreatorFactory.
2026-08-06 11:02:22 info INFO DefaultDispatcher-worker-13#byod-orders.fieldservicecases i.a.c.r.FeedReader(read):72 no more partitions to read for 'byod-orders.fieldservicecases' in round 1
... (note: unlike every other stream in a normal run, no per-stream
"Attempting bootstrap using class io.airbyte.cdk.read.cdc.CdcPartitionsCreatorFactory"
line appears for fieldservicecases here) ...
Sync summary excerpt:
{
"streamName" : "fieldservicecases",
"streamNamespace" : "byod-orders",
"stats" : {
"bytesCommitted" : 459112,
"bytesEmitted" : 459112,
"estimatedBytes" : 0,
"estimatedRecords" : 0,
"recordsEmitted" : 116,
"recordsCommitted" : 116,
"recordsFilteredOut" : 0,
"bytesFilteredOut" : 0
}
}Contribute
Internal Tracking: https://github.com/airbytehq/oncall/issues/13251
Connector Name
destination-snowflake
Connector Version
4.0.31
What step the error happened?
During the sync
Relevant information
During a routine hourly CDC sync, a transient Snowflake API error during the destination's pre-sync reconciliation check caused the connector to rebuild one stream's table from scratch with CREATE OR REPLACE TABLE, instead of appending to it as normal. Because the MySQL CDC source's saved binlog cursor was untouched by this, the very next read only pulled the small number of rows changed since the last checkpoint — not a full resync — so the destination table was left permanently short of its prior history, with no error, warning, or failed sync status to flag it.
Sequence of events (job 44345, stream byod-orders.fieldservicecases → RAW.SP2_CDC_RAW.FIELDSERVICECASES):
11:01:36 — Destination issues its normal pre-sync reconciliation query: SELECT COUNT(*) AS TOTAL FROM "RAW"."SP2_CDC_RAW"."FIELDSERVICECASES"
11:01:52 — That request fails after exhausting all retries: Stop retrying as max retries have been reached ... Max retry count: 7, with the underlying error being HTTP Response code: 503 from Snowflake.
11:02:14 — The stream loader for fieldservicecases starts, but instead of the normal DESCRIBE TABLE → append path used for every other stream in the same job (~269 tables), it goes straight to: CREATE OR REPLACE TABLE "RAW"."SP2_CDC_RAW"."FIELDSERVICECASES" (...) — no DESCRIBE TABLE call is logged for this table anywhere in the job.
On the read side, this stream also never gets the per-stream Attempting bootstrap using class io.airbyte.cdk.read.cdc.CdcPartitionsCreatorFactory line that every other stream gets in a normal run — meaning no full resnapshot was considered or triggered for it.
The sync summary shows only recordsEmitted: 116 / recordsCommitted: 116 for fieldservicecases — consistent with a normal incremental delta from the shared Debezium binlog reader (records-emitted=1014 total across the whole sync), not a fresh backfill.
The job completed with a SUCCEEDED status. There was no indication anywhere in the job history or UI that the destination table had just been wiped and was now missing its historical rows.
Why this looks like a bug rather than intended behavior:
The destination's response to a failed reconciliation check (an Snowflake-side transient 503) was to silently rebuild the table via CREATE OR REPLACE, discarding all existing rows, rather than failing the sync/stream and retrying, or leaving the existing table untouched.
The MySQL CDC source has no visibility into destination-side table state, so it had no way to know a resync was needed — it just kept incrementing from its last saved binlog position. The result is a silent, partial data loss with a "succeeded" sync status.
Impact: RAW.SP2_CDC_RAW.FIELDSERVICECASES currently contains only the rows synced since 2026-08-06 11:02, having previously held its full history. This was only caught because we happened to notice unexpected CREATE OR REPLACE queries in Snowflake's query history and traced them back through the Airbyte job logs.
Expected behavior
One of the following, rather than a silent partial rebuild:
If the reconciliation check fails, the sync/stream should fail loudly (or retry the check) rather than falling back to CREATE OR REPLACE.
If a CREATE OR REPLACE is genuinely necessary (e.g. schema drift the connector can't reconcile another way), it should trigger a full resnapshot of that stream from the source so the destination table ends up complete — not just resume from the existing incremental cursor.
Relevant log output
Contribute
Internal Tracking: https://github.com/airbytehq/oncall/issues/13251