Skip to content

RedownloadAvatarWorker silently fails when target directory doesn't exist #39118

Description

@Zendorea

Description

RedownloadAvatarWorker (and likely RedownloadHeaderWorker) silently fails to save downloaded avatar files when the target directory structure doesn't exist on disk. The worker downloads the file successfully from the remote server but the file is never written to the filesystem.

This occurs after a media cache has been cleared (e.g., via tootctl media remove or manual cleanup). The database retains avatar_file_name and avatar_remote_url but account.avatar.exists? returns false.

Root Cause

The Sidekiq background worker doesn't create the directory structure before writing. In contrast, direct assignment via Rails (account.avatar = tempfile; account.save!) works correctly because Paperclip/ActiveStorage auto-creates directories during the assignment process.

Steps to Reproduce

  1. Have a running instance with federated accounts that have cached avatars
  2. Clear the media cache: tootctl media remove --days 0 or delete files from /system/cache/accounts/avatars/
  3. Wait for RedownloadAvatarWorker jobs to process (or trigger via tootctl accounts refresh)
  4. Check: Account.where.not(domain: nil).where.not(avatar_file_name: nil).select { |a| !a.avatar.exists? }.count
  5. Result: Many accounts report avatar.exists? = false despite having avatar_file_name set

Evidence

account = Account.find_by(username: 'example', domain: 'remote.instance')
account.avatar_file_name  # => "abc123.png"
account.avatar.exists?    # => false
File.directory?(File.dirname(account.avatar.path))  # => false (directory missing!)

# Manual fix works:
tempfile = URI.open(account.avatar_remote_url)
account.avatar = tempfile
account.save!  # Creates directory + saves file
account.avatar.exists?  # => true

Workaround

Run periodically via cron:

Account.where.not(domain: nil).where.not(avatar_remote_url: [nil, '']).find_each do |account|
  next if account.avatar.exists?
  tempfile = URI.open(account.avatar_remote_url, open_timeout: 5, read_timeout: 10)
  account.avatar = tempfile
  account.save!
rescue
  nil
end

Expected Behavior

RedownloadAvatarWorker should ensure the target directory exists (via FileUtils.mkdir_p or equivalent) before attempting to write the downloaded file, matching the behavior of direct Paperclip assignment.

Environment

  • Mastodon version: 4.4.16
  • Storage: Local filesystem (no S3)
  • Triggered after: tootctl media remove --days 1 cleared avatar cache
  • Scale: 29,334 accounts with avatar_file_name set, only 3,717 files on disk

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions