Backup Errors: Container logs inundated with errors about archiver failing.

[zgjimgjonbalaj]

Describe the bug
In the docker container logs for the rudder-server I am getting a lot of the following errors:

2026-01-25T01:44:30.031Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 225, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

2026-01-25T01:44:35.085Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

2026-01-25T01:44:35.085Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 232, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

2026-01-25T01:44:40.143Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

2026-01-25T01:44:40.143Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 238, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

2026-01-25T01:44:45.202Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

2026-01-25T01:44:45.202Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 233, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

Steps to reproduce the bug
Start the server without configuring any backups/archiving or providing the necessary variables as given via the sample docker-compose.yml. I do see some env vars in the sample config for archival settings for MinIO, S3, GCS and others but no environment vars for disabling. Additionally, online documents also have no mentions on how to disable.

Expected behavior
The default behavior I would think should be not to backup/archive automatically unless there are configuration settings for a bucket. In the web docs it offers information on how to configure locations via the "Data Management" tab however this tab does not exist for self-hosted instances. We are currently just testing the RudderStack and wanted to check logs to ensure no other issues present but these make it difficult to filter. I am doing so through grep but can't help but wonder if there is something I am missing here.

Thanks for all your work on this platform!

[contributor-support]

Thanks for opening this issue! We'll get back to you shortly. If it is a bug, please make sure to add steps to reproduce the issue.

[gitcommitshow]

@zgjimgjonbalaj thanks for reporting the issue.

Can you please confirm following

1. Your rudder-server version
2. Do you have any of the env set up for the bucket configurations? I agree with what you said that they should be disabled by default so those configurations should be commented out.

[zgjimgjonbalaj]

Hey @gitcommitshow,

Thanks for taking a look at this, I haven't configuted any bucket settings to my knowledge. Below are the hash of the container images being used and my complete docker-compose.yml. Let me know if theres anything else I can provide.

rudderlabs/rudder-server:latest                           1fb42d686628        293MB             0B    U
rudderstack/rudder-transformer:latest                     5c8f56c4aa7e        665MB             0B    U

services:
  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: <hidden>
      POSTGRES_PASSWORD: <hidden>
      POSTGRES_DB: <hidden>
    expose:
      - 5432
    shm_size: 128mb
    networks:
      - net

  server:
    image: rudderlabs/rudder-server:latest
    depends_on:
      - db
      - metrics-exporter
      - transformer
    entrypoint: sh -c '/wait-for db:5432 -- /rudder-server'
    expose:
      - 8080
    environment:
      JOBS_DB_HOST: <hidden>
      JOBS_DB_USER: <hidden>
      JOBS_DB_PORT: <hidden>
      JOBS_DB_DB_NAME: <hidden>
      JOBS_DB_PASSWORD: <hidden>
      DEST_TRANSFORM_URL: <hidden>
      CONFIG_BACKEND_URL: <hidden>
      WORKSPACE_TOKEN: <hidden>
      STATSD_SERVER_URL: <hidden>
    networks:
      - net
      - proxyable

  transformer:
    image: rudderstack/rudder-transformer:latest
    depends_on:
      - metrics-exporter
    expose:
      - 9090
    environment:
      STATSD_SERVER_HOST: metrics-exporter
      STATSD_SERVER_PORT: 9125
    networks:
      - net

  aistor:
    image: quay.io/minio/aistor/minio:RELEASE.2025-12-20T04-58-37Z
    container_name: rudder-aistor-1
    restart: unless-stopped
    expose:
      - 9000
      - 9001
    volumes:
      - s3-data:/mnt/data
      - s3-certs:/etc/minio/certs
      - ./minio.license:/minio.license
    command: minio server /mnt/data --license /minio.license
    environment:
      MINIO_ROOT_USER: <hidden>
      MINIO_ROOT_PASSWORD: <hidden>
    networks:
      - net
      - proxyable

  metrics-exporter:
    image: prom/statsd-exporter:v0.22.4
    expose:
      - 9102
    networks:
      - net

volumes:
  s3-data: null
  s3-certs: null

networks:
  net: null
  proxyable:
    external: true

[gitcommitshow]

@zgjimgjonbalaj this is not the default docker compose file.
This is - https://raw.githubusercontent.com/rudderlabs/rudder-server/master/rudder-docker.yml
As you can see the minio configs are commented out there.
Adjust your config acordingly, this should fix the issue. Let me know if it doesn't.
This is the official step-by-step guide to set up RudderStack using Docker, please make sure to follow this to avoid unexpected results - https://www.rudderstack.com/docs/get-started/rudderstack-open-source/data-plane-setup/docker/

[zgjimgjonbalaj]

Hey @gitcommitshow ,

Thanks for your response, apologies on the delay just got a chance to look at this. So is the mere presence of a S3 container in the stack enough for RudderStack to throw those warnings? I don't see any other ENV vars that tie RudderStack to the S3 bucket aside from settings in the control plane for one of our destinations. We are using S3 only for the ClickHouse destination configured in the control plane. It seems to require a S3 staging area:

This is really the only reason why we added the container in the first place.

[zgjimgjonbalaj]

Quick Update:

Reconfigured everything exactly as they were with two changes. First, I noticed both of our ClickHouse destinations were using the same S3 staging bucket and access policy but a different database name so I separated those destinations into different buckets. I also updated the rudder-server image to the latest version.

I am not sure if either of those was the culprit, or if it was simply remnants of previous testing, but so far I have not seen those warnings come up in the container logs yet.

That said, the warnings did take a while to show up (estimate, 1 - 2 weeks) so I don't feel I am out of the woods yet. I will post another update in a week or so.

[gitcommitshow]

Thanks for the update. This information is useful.

[zgjimgjonbalaj]

Hey @gitcommitshow,

They are back with the same result

2026-02-24T02:59:36.628Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 11, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
2026-02-24T02:59:41.735Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
2026-02-24T02:59:41.735Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 1, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
2026-02-24T02:59:46.858Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
2026-02-24T02:59:46.858Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 10, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
2026-02-24T02:59:51.974Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
2026-02-24T02:59:51.975Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 5, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
2026-02-24T02:59:57.103Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

[gitcommitshow]

Thanks for reporting. Allow me some time to investigate it.

[zgjimgjonbalaj]

@gitcommitshow

Thanks for looking into it. It looks like they started about 5 days or so after booting up the stack. I am trying to recall if I did anything else that day but don't recall.

2026-02-19T02:59:26.432Z        ERROR   transformer-features    transformer/features_impl.go:135        error sending request   {"error": "Get \"http://transformer:9090/features\": dial tcp 172.19.0.3:9090: connect: connection refused"}

2026-02-19T02:59:26.437Z        ERROR   transformer-features    transformer/features_impl.go:135        error sending request   {"error": "Get \"http://transformer:9090/features\": dial tcp 172.19.0.3:9090: connect: connection refused"}

2026-02-19T02:59:26.440Z        ERROR   transformer-features    transformer/features_impl.go:135        error sending request   {"error": "Get \"http://transformer:9090/features\": dial tcp 172.19.0.3:9090: connect: connection refused"}                                                                                                                                          

2026-02-24T02:59:36.627Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}                                                                                                                                                                                       

2026-02-24T02:59:36.628Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 11, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}                                                                                                                                                                                       

2026-02-24T02:59:41.735Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}                                                                                                                                                                                       

2026-02-24T02:59:41.735Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 1, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}                                                                                                                                                                                        

2026-02-24T02:59:46.858Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}                                                                                                                                                                                       

2026-02-24T02:59:46.858Z        ERROR   runner.warehouse.archiver       archive/archiver.go:370 [Archiver]: Error backing up staging files for upload   {"uploadId": 10, "error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}                                                                                                                                                                                       

2026-02-24T02:59:51.974Z        ERROR   tablearchiver   tablearchiver/tablearchiver.go:124      [TableJSONArchiver]: Error uploading local file dump to object storage  {"error": "operation error S3: PutObject, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

This is a sample dataset, does not include all as they are pretty much occurring every 5 seconds or so.

[gitcommitshow]

@zgjimgjonbalaj just learned that

• By default warehouse archiving is enabled
• set following config to false to disable it

  Warehouse:
    archiveUploadRelatedRecords: false
  

[zgjimgjonbalaj]

@gitcommitshow

Ahh that makes sense now, thanks for following up. Is there an ENV equivalent we can set or is this set in the compose.yml?

[gitcommitshow]

@zgjimgjonbalaj I don't see such env in code.
Btw, the RudderStack team will now disable archiver by default as you can see in the above linked PRs.

[zgjimgjonbalaj]

Awesome @gitcommitshow, yes I saw that. Thank you gain for you continued efforts here. I have updated to the latest image docker image on my end and will monitor for any issues.

[gitcommitshow]

Those PRs are not merged yet so you would have to wait for another release. Until then, you can use the config.