Skip to content

Commit 88c15cb

Browse files
committed
Merge branch 'main' into adelhajhassan/support-enabling-untaint-controller-in-datadog-chart
2 parents eb3e544 + ffe424d commit 88c15cb

193 files changed

Lines changed: 6211 additions & 838 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,9 @@ jobs:
2424
with:
2525
fetch-depth: 0
2626
- name: Set up Helm
27-
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
27+
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
2828
with:
29-
version: v4.2.0
29+
version: v4.2.2
3030
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
3131
with:
3232
python-version: 3.14

.github/workflows/go-test-datadog-csi-driver.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,9 @@ jobs:
2727
go-version: 1.26
2828
id: go
2929
- name: Set up Helm
30-
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
30+
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
3131
with:
32-
version: v4.2.0
32+
version: v4.2.2
3333
- name: Add Datadog Helm repo
3434
run: helm repo add datadog https://helm.datadoghq.com && helm repo update
3535
- name: Check out code into the Go module directory

.github/workflows/go-test-datadog.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,9 @@ jobs:
3333
go-version: 1.26
3434
id: go
3535
- name: Set up Helm
36-
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
36+
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
3737
with:
38-
version: v4.2.0
38+
version: v4.2.2
3939
- name: Add Datadog Helm repo
4040
run: helm repo add datadog https://helm.datadoghq.com && helm repo update
4141
- name: Add Prometheus Community Helm repo

.github/workflows/go-test-operator.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,9 @@ jobs:
2929
go-version: 1.26
3030
id: go
3131
- name: Set up Helm
32-
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
32+
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
3333
with:
34-
version: v4.2.0
34+
version: v4.2.2
3535
- name: Add Datadog Helm repo
3636
run: helm repo add datadog https://helm.datadoghq.com && helm repo update
3737
- name: Check out code into the Go module directory

.github/workflows/go-test-private-action-runner.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,9 @@ jobs:
2727
go-version: 1.26
2828
id: go
2929
- name: Set up Helm
30-
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
30+
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
3131
with:
32-
version: v4.2.0
32+
version: v4.2.2
3333
- name: Add Datadog Helm repo
3434
run: helm repo add datadog https://helm.datadoghq.com && helm repo update
3535
- name: Check out code into the Go module directory
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Go Test YAML Mapper
2+
on:
3+
push:
4+
paths:
5+
- 'test/datadog/**'
6+
- 'test/common/**'
7+
- 'charts/datadog/**'
8+
- '.github/workflows/go-test-yamlmapper.yaml'
9+
pull_request:
10+
paths:
11+
- 'test/datadog/**'
12+
- 'test/common/**'
13+
- 'charts/datadog/**'
14+
- '.github/workflows/go-test-yamlmapper.yaml'
15+
16+
# Permission forced by repo-level setting; only elevate on job-level
17+
permissions:
18+
contents: read
19+
# packages: read
20+
21+
env:
22+
GO111MODULE: "on"
23+
PROJECTNAME: "helm-charts"
24+
jobs:
25+
integ-tests:
26+
runs-on: ubuntu-latest
27+
timeout-minutes: 60
28+
steps:
29+
- name: Set up Go
30+
uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
31+
with:
32+
go-version: 1.26
33+
id: go
34+
- name: Set up Helm
35+
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
36+
with:
37+
version: v3.21.3
38+
- name: Add Datadog Helm repo
39+
run: helm repo add datadog https://helm.datadoghq.com && helm repo update
40+
- name: Add Prometheus Community Helm repo
41+
run: helm repo add prometheus-community https://prometheus-community.github.io/helm-charts && helm repo update
42+
- name: Check out code into the Go module directory
43+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
44+
with:
45+
fetch-depth: 0
46+
- name: Create K8s v1.33.1 cluster
47+
uses: helm/kind-action@0025e74a8c7512023d06dc019c617aa3cf561fde # v1.10.0
48+
with:
49+
version: v0.22.0
50+
node_image: kindest/node:v1.33.1
51+
cluster_name: datadog-ci
52+
config: .github/kind_config.yaml
53+
- name: Install Datadog CRDs
54+
run: |
55+
make setup-mapper-crds
56+
- name: Run yamlmapper integ tests
57+
run: |
58+
kubectl cluster-info
59+
kubectl get nodes
60+
helm dependency build ./charts/datadog
61+
make integ-test-mapper-strict
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# This workflow checks that the yaml-mapper mapping file is in sync with the keys
2+
# in charts/datadog/values.yaml. It fails if any key from values.yaml is missing
3+
# from charts/datadog/files/mapping_datadog_helm_to_datadogagent_crd.yaml.
4+
#
5+
# To update the mapping file, run from the yaml-mapper directory:
6+
# ./yaml-mapper -updateMap -sourceFile=../../charts/datadog/values.yaml
7+
#
8+
# This workflow is currently disabled (rename to .yaml to enable).
9+
name: Yaml-Mapper Tool
10+
11+
on:
12+
pull_request:
13+
paths:
14+
- "charts/datadog/values.yaml"
15+
- "charts/datadog/files/mapping_datadog_helm_to_datadogagent_crd.yaml"
16+
workflow_dispatch:
17+
# Permission forced by repo-level setting; only elevate on job-level
18+
permissions:
19+
contents: read
20+
21+
jobs:
22+
check-mapper-update:
23+
runs-on: ubuntu-latest
24+
25+
steps:
26+
- name: Checkout
27+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
28+
with:
29+
fetch-depth: 0
30+
- name: Set up Python
31+
uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4.7.1
32+
with:
33+
python-version: 3.12
34+
35+
- name: Install dependencies
36+
run: pip install pyyaml
37+
38+
- name: Compare values.yaml with mapping file
39+
run: |
40+
python <<EOF
41+
import yaml
42+
import sys
43+
44+
def flatten_dict(d, parent_key="", sep="."):
45+
"""Recursively flattens a nested dictionary."""
46+
items = []
47+
for k, v in d.items():
48+
new_key = f"{parent_key}{sep}{k}" if parent_key else k
49+
if isinstance(v, dict):
50+
items.extend(flatten_dict(v, new_key, sep=sep).items())
51+
else:
52+
items.append((new_key, v))
53+
return dict(items)
54+
55+
values_file = "charts/datadog/values.yaml"
56+
mapping_file = "charts/datadog/files/mapping_datadog_helm_to_datadogagent_crd.yaml"
57+
58+
with open(values_file, "r") as vf, open(mapping_file, "r") as mf:
59+
values_data = yaml.safe_load(vf) or {}
60+
mapping_data = yaml.safe_load(mf) or {}
61+
62+
# Flatten values.yaml structure
63+
flattened_values = flatten_dict(values_data)
64+
65+
# Extract keys from the mapping file
66+
mapping_keys = set(mapping_data.keys())
67+
68+
# Compare missing keys
69+
missing_keys = [key for key in flattened_values.keys() if key not in mapping_keys]
70+
71+
if missing_keys:
72+
print("The following keys are missing in the mapping file:")
73+
for key in missing_keys:
74+
print(f"- {key}")
75+
print("To update the mapping file, run from the yaml-mapper directory:")
76+
print(" ./yaml-mapper -updateMap -sourceFile=../../charts/datadog/values.yaml")
77+
sys.exit(1)
78+
else:
79+
print("Mapper file is correctly updated!")
80+
EOF

.github/workflows/pr-labeler.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717
pull-requests: write
1818
timeout-minutes: 5
1919
steps:
20-
- uses: actions/labeler@f27b608878404679385c85cfa523b85ccb86e213 # v6.1.0
20+
- uses: actions/labeler@b8dd2d9be0f68b860e7dae5dae7d772984eacd6d # v6.2.0
2121
with:
2222
repo-token: "${{ secrets.GITHUB_TOKEN }}"
2323
configuration-path: .github/workflows/labeler/labels.yaml

.github/workflows/stale.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ jobs:
2222
scope: DataDog/helm-charts
2323
policy: self.stale.manage-stale
2424

25-
- uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0
25+
- uses: actions/stale@1e223db275d687790206a7acac4d1a11bd6fe629 # v10.4.0
2626
with:
2727
repo-token: ${{ steps.octo-sts.outputs.token }}
2828

CONTRIBUTING.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,64 @@ make test
7171
```
7272
You can run tests from IDE too (tested with VScode) as long as the environment variables are configured properly.
7373

74+
#### YAML Mapper Integration Tests
75+
76+
The YAML mapper integration tests validate the migration path from the Datadog Helm chart to the DatadogAgent CRD (used by the Datadog Operator). Each test:
77+
1. Installs the `datadog` Helm chart with a values file.
78+
2. Runs the YAML mapper to produce a `DatadogAgent` CR from those same values.
79+
3. Installs the Datadog Operator and applies the generated CR.
80+
4. Compares the live agent configuration (`agent config --all`) between both installations to verify the mapper produces an equivalent result.
81+
82+
**Prerequisites**
83+
* A local Kubernetes cluster (e.g. Kind). **Do not run against a staging or production cluster.**
84+
* `kubectl` context pointing at the test cluster.
85+
* Helm repos added:
86+
```shell
87+
helm repo add datadog https://helm.datadoghq.com
88+
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
89+
```
90+
Run `helm repo update` if you have recently bumped the `datadog-operator` version in `charts/datadog/requirements.yaml` and need to pull the updated chart.
91+
* Datadog CRDs installed in the cluster:
92+
```shell
93+
make setup-mapper-crds
94+
```
95+
* The `datadog` chart dependencies built:
96+
```shell
97+
helm dependency build ./charts/datadog
98+
```
99+
* Environment variables (optional):
100+
* `API_KEY` and `APP_KEY` — not required; all test values files use hardcoded dummy keys. If set, an additional Datadog secret is created in each test namespace.
101+
102+
**Running the tests**
103+
104+
```shell
105+
# Standard mode: log agent config diffs but don't fail on them
106+
make integ-test-mapper
107+
108+
# Strict mode (used in CI): fail if helm vs operator agent config differs
109+
make integ-test-mapper-strict
110+
111+
# Run a specific test by name
112+
make integ-test-mapper GOTEST_RUN=TestBaseValues
113+
```
114+
115+
**Environment variables**
116+
117+
| Variable | Default | Description |
118+
|---|---|---|
119+
| `YAMLMAPPER_AGENT_CONF_STRICT` | `0` | Fail tests if the Helm and Operator agent configs differ |
120+
| `YAMLMAPPER_WARNINGS_STRICT` | `0` | Fail tests if the mapper emits any warnings |
121+
| `YAMLMAPPER_CLEANUP_STALE` | `0` | Clean up leftover test namespaces from previous interrupted runs (safe for local clusters only) |
122+
123+
**Cleanup**
124+
125+
Each test creates a uniquely named namespace and cleans it up on completion. If a test run is interrupted (e.g. Ctrl+C), stale namespaces prefixed with `datadog-agent-` may remain. Re-run with `YAMLMAPPER_CLEANUP_STALE=true` to automatically remove them, or delete manually with `kubectl delete namespace`.
126+
127+
To remove the CRDs installed by `setup-mapper-crds`:
128+
```shell
129+
make cleanup-mapper-crds
130+
```
131+
74132
#### End-to-End Tests
75133
The helm-charts end-to-end (E2E) tests run on [Pulumi][pulumi]-deployed test infrastructures, defined as "stacks". The test infrastructures are deployed using the [`e2e-framework`][e2e-framework-source] and [`datadog-agent`][agent-e2e-source] E2E frameworks.
76134

0 commit comments

Comments
 (0)