In testing with the large Colombia datastack, the job ran out of disk space. We need to make more disk space available to jobs to use for the duration of the job until everything gets uploaded to the GCS bucket. It seems that a common pattern is to mount a large scratch volume that is shared between all nodes.
In testing with the large Colombia datastack, the job ran out of disk space. We need to make more disk space available to jobs to use for the duration of the job until everything gets uploaded to the GCS bucket. It seems that a common pattern is to mount a large scratch volume that is shared between all nodes.