Skip to content

[Feature]: unify config across backends #284

Description

@evacchi

Problem Statement

currently each backend uses different sets of flags that are partially overlapping.

  • Some backends allow config files, some backends have shortcuts to set flags on the CLI (e.g. SortedSet), etc.
  • gates are parsed differently across backends (e.g. sorted set uses a custom StringMap; GCP PubSub uses map[string]string; composite gate defines its own local struct)
  • config surfaces are fragmented: queue/topic configs, pool configs, runtime params, and gate configs are spread across separate files, CLI flags, and env vars with different loading patterns
  • only one transport can be active per async instance, but the config flags do not make this explicit

Proposed Solution

  • We introduce a single config flag to set the backend (e.g. --transport) and we pass either a --transport-config [string] or a --transport-config-file [path] + a --pool-config-file
  • we allow some env variables to override specific keys (e.g. redis url etc) to specify secrets
  • we extract shared parameter passing and pass structured data to the backend constructors

Alternatives Considered

n/a

Willingness to Contribute

Yes, I can submit a PR

Additional Context

this is a somewhat significant refactor and might be introduce some breakage, so we should be mindful about that and possibly introduce some temporary measure for backwards compatibility if necessary

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions