You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Updating a single Datadog config key in ADP currently requires touching several files. There are two
sources of truth with no sync mechanism between them.
All config metadata (which keys are supported, which pipelines they affect, what they're used for)
is encoded in hand-written Rust structs and is only visible at runtime.
The ledger and documentation exist as separate artifacts with no generation step.
Proposed Solution
Replace all existing per-key annotation sources with a single hand-maintained schema_overlay.yaml,
joined at build time with the vendored Datadog schema. The overlay will be the single source of
truth for Datadog config coverage, documentation, runtime classification, and smoke-test support.
lib/datadog-agent-commons/build.rs will read both files and:
Validate that every key in the vendored schema appears exactly once in the overlay.
Generate config registry annotation constants, replacing the hand-written datadog/*.rs files.
Generate the documentation markdown.
Generate a DatadogConfiguration struct (via typify or a custom generator, depending on the
schema language).
Generate a "witness trait" (DatadogConfigConsumer) trait with one method per supported key, plus
a drive function — forcing a human to explicitly handle every supported key at compile time.
A hand-written Translator will implement the trait, converting DatadogConfiguration into the
internal topology and dataplane config structs. Adding a key to the overlay without updating the
translator will result in a compilation error. Removing a key leaves an orphan impl method — also a
compilation error.
The audit ledger will be deleted. Documentation will be generated. The registry annotation files
will be deleted at the end of the process when the smoke tests switch over to using the new code.
Layout
lib/datadog-agent-commons/config/schema.yaml # vendored, untouched
lib/datadog-agent-commons/config/schema_overlay.yaml # hand-written, single source of truth
lib/datadog-agent-schema-model/ # types for deserializing the overlay
Anticipated PRs
If everything goes according to plan, then the following sequence makes sense to me. If it goes off
the rails, all bets are off!
(small): New datadog-agent-schema-model crate: Rust types for deserializing schema_overlay.yaml
(small): Infra scaffold: move schema to datadog-agent-commons/config/, add build.rs that
validates overlay structure (no coverage enforcement yet)
(medium): Generation pipeline with seed data: build.rs generates config registry constants
and documentation markdown, proved with a small set of synthetic overlay entries
(xtra-large): Migrate all keys to schema_overlay.yaml; enable full coverage enforcement;
replace the hand-written config_registry/datadog/*.rs with generated files (TBD exactly what,
but preserve the smoke tests for now). Delete the audit ledger. Generate the documentation file.
(small): Generate DatadogConfiguration struct from the vendored schema
(medium): Generate trait witness: DatadogConfigConsumer trait and drive function
Problem
Updating a single Datadog config key in ADP currently requires touching several files. There are two
sources of truth with no sync mechanism between them.
Three additional problems compound this:
allow(dead_code),cfg(test), andfeature = "config-test-support"guards throughout (see Separate prod from test/build concerns in the config registry. #1623).is encoded in hand-written Rust structs and is only visible at runtime.
Proposed Solution
Replace all existing per-key annotation sources with a single hand-maintained
schema_overlay.yaml,joined at build time with the vendored Datadog schema. The overlay will be the single source of
truth for Datadog config coverage, documentation, runtime classification, and smoke-test support.
lib/datadog-agent-commons/build.rswill read both files and:datadog/*.rsfiles.DatadogConfigurationstruct (via typify or a custom generator, depending on theschema language).
DatadogConfigConsumer) trait with one method per supported key, plusa
drivefunction — forcing a human to explicitly handle every supported key at compile time.A hand-written
Translatorwill implement the trait, convertingDatadogConfigurationinto theinternal topology and dataplane config structs. Adding a key to the overlay without updating the
translator will result in a compilation error. Removing a key leaves an orphan impl method — also a
compilation error.
The audit ledger will be deleted. Documentation will be generated. The registry annotation files
will be deleted at the end of the process when the smoke tests switch over to using the new code.
Layout
Anticipated PRs
If everything goes according to plan, then the following sequence makes sense to me. If it goes off
the rails, all bets are off!
datadog-agent-schema-modelcrate: Rust types for deserializingschema_overlay.yamldatadog-agent-commons/config/, addbuild.rsthatvalidates overlay structure (no coverage enforcement yet)
build.rsgenerates config registry constantsand documentation markdown, proved with a small set of synthetic overlay entries
schema_overlay.yaml; enable full coverage enforcement;replace the hand-written
config_registry/datadog/*.rswith generated files (TBD exactly what,but preserve the smoke tests for now). Delete the audit ledger. Generate the documentation file.
DatadogConfigurationstruct from the vendored schemaDatadogConfigConsumertrait anddrivefunctionTranslator; replace the config-reading entry point; smoke tests passGenericConfigurationaccess with this???