Skip to content

GCP: data disk snapshot is never refreshed after the first stop cycle, causing data loss on restart #388

Description

@arnaud-robin

Describe the bug

On GCP, the data disk snapshot is only created on the first stop of an instance. On every subsequent stop, the snapshot is silently not updated: Pulumi sees no change and keeps the stale snapshot. On the next start, the data disk is restored from that stale snapshot, so all data written after the first stop cycle is lost.

Other providers do not seem affected, but I didn't try to reproduce there.

Steps to reproduce

  1. Create a GCP instance with data disk snapshot enabled.
  2. Write some data to /cloudy/data (e.g. touch /cloudy/data/file-1).
  3. Stop the instance (cloudypad stop <instance>). A snapshot <instance>-data-volume-snapshot is created.
  4. Start the instance (cloudypad start <instance>). Data disk is restored from snapshot, file-1 is present.
  5. Write more data (e.g. touch /cloudy/data/file-2).
  6. Stop the instance again. The snapshot is not updated (same creation timestamp as step 3).
  7. Start the instance. file-2 is gone — the disk was restored from the step-3 snapshot.

Root cause

The GCP data disk snapshot stack (src/providers/gcp/pulumi/data-volume-snapshot.ts) follows the same pattern as AWS: a snapshot resource with replaceOnChanges: ["sourceDisk"], expecting the disk ID to change on every stop/start cycle and force a snapshot replacement.

  • On AWS, this works: a restored EBS volume gets a brand new random ID (vol-xxx), so replaceOnChanges: ["volumeId"] fires on every stop.
  • On GCP, the value used as disk ID is the disk name (<instance>-data, see dataDiskId = dataDisk.name in src/providers/gcp/pulumi/main.ts), which is deterministic and identical across every delete/restore cycle. sourceDisk therefore never changes, Pulumi sees zero diff on subsequent stops, and the snapshot stays at its first version forever.

Proposed fix

Since GCP provides no naturally-changing input, introduce a synthetic one: pass a unique snapshotVersion (e.g. timestamp) from the provisioner to the snapshot stack on every snapshot provision, and include it in the snapshot name. GCP snapshot names are immutable, so the name change forces Pulumi to replace the snapshot on every stop cycle.

I think I can provide a working patch for this and can open a PR if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions