Skip to content

oleg-tkachuk/flamingo-hw

BranchesTags

Repository files navigation

flamingo-hw

End-to-end local deployment of FleetDM — server, MySQL and Valkey (Redis-compatible) — onto a minikube cluster, orchestrated by Helmfile and driven via thin Task wrappers.

The full stack is described declaratively in a single helmfile.yaml.gotmpl. Release ordering, namespaces, values files and the schema-bootstrap Jobs are all part of that spec — no custom imperative install script.

The deployment follows FleetDM's Option B — install MySQL and Redis/Valkey separately and uses official upstream Helm charts for every off-the-shelf component:

Component Source
MySQL Local chart wrapping the official mysql:8.4 Docker image — see charts/mysql/
Valkey valkey-io/valkey-helm
Fleet fleetdm/fleet
ingress-nginx kubernetes/ingress-nginx
cert-manager jetstack/cert-manager

Why a local chart for MySQL? Oracle's MySQL Operator only provisions InnoDBCluster resources with Group Replication enabled, which rejects Fleet's host_mdm_windows_profiles migration (it DROP PRIMARY KEYs without a paired ADD PRIMARY KEY in the same statement). The thin charts/mysql/ wrapper avoids that without giving up the upstream image.

TLS is mandatory, not optional: Fleet's UI bundle persists the session token in a __Host-token cookie marked Secure, which browsers refuse to set on plain HTTP. cert-manager + a local self-signed root CA (charts/cert-manager-issuers/) make HTTPS work offline on *.nip.io without external ACME — see TLS / cert-manager below.

Table of contents

Documentation

Document Scope
docs/ARCHITECTURE.md AWS Well-Architected solution design for Company Inc. (cloud accounts, VPC, EKS, CI/CD, MongoDB Atlas)
deploy/values/README.md Release graph + per-release values walkthrough for the local stack
charts/bootstrap/README.md Namespaces + shared secrets bootstrap chart
charts/mysql/README.md Standalone MySQL 8.x chart; why not Oracle's operator
charts/fleet-db-init/README.md init-db + prepare-db Helm post-install Jobs
charts/cert-manager-issuers/README.md Self-signed root CA + fleet-ca ClusterIssuer chain

Highlights

  • One-command cluster + stack: task cluster:create && task fleet:install (fleet:install is a wrapper for helmfile sync)
  • Standalone MySQL 8.x via a thin local chart on the official Docker image — no Group Replication, no operator
  • Fleet user/database bootstrapped by an init-db Job, then fleet prepare db runs as a follow-up prepare-db Job — both as Helm post-install hooks of the fleet-db-init chart
  • HTTPS by default: cert-manager + local self-signed root CA mint a leaf cert into the ingress automatically; task fleet:trust-ca imports the root into the host keychain so the browser shows a green lock (idempotent; skipped on CI=true)
  • ingress-nginx admission webhook serving cert is also provisioned by cert-manager (via the chart's certManager.enabled=true switch) — avoids the chart's default racy post-install patch job that can leave the ValidatingWebhookConfiguration with an empty caBundle
  • Exposure without /etc/hosts: ingress on a nip.io hostname derived from the minikube IP, plus NodePort and port-forward alternatives
  • End-to-end task fleet:verify health checks for pods, MySQL, Valkey, Fleet HTTP API and ingress
  • CI: yamllint + chart-testing on PRs; semantic-release + chart-releaser on merge to main (publishes any chart under charts/ to gh-pages)
  • Pre-commit hooks (lefthook): yamllint, helm lint, helm template, Conventional Commits enforcement

Repository layout

.
├── helmfile.yaml.gotmpl       # Declarative spec: releases, ordering, hooks
├── Taskfile.yaml              # Root task entrypoints + global defaults
├── tasks/
│   ├── cluster.task.yaml      # cluster:create / start / stop / delete
│   ├── chart.task.yaml        # chart:install / lint / uninstall (ad-hoc)
│   └── fleet.task.yaml        # thin wrappers around `helmfile sync` etc.
├── charts/                    # Local Helm charts
│   ├── bootstrap/             #   Namespaces + shared secrets (JWT, mysql-creds)
│   ├── mysql/                 #   Standalone MySQL 8.x (no operator)
│   ├── fleet-db-init/         #   init-db + prepare-db as Helm post-install Jobs
│   └── cert-manager-issuers/  #   SelfSigned -> root CA -> fleet-ca ClusterIssuer
├── deploy/
│   └── fleet/                 # Per-release values files for upstream charts
├── .github/workflows/
│   ├── ci.yaml                # PR: yamllint + helmfile lint + chart-testing
│   └── release.yaml           # semantic-release + chart-releaser
├── lefthook.yml               # Local pre-commit hooks (incl. helmfile lint)
├── .yamllint.yaml             # yamllint config
├── ct.yaml                    # chart-testing config
└── release.config.cjs         # semantic-release config

Prerequisites

Platform note: end-to-end tested on macOS only (Apple Silicon + Intel, Docker Desktop and vfkit drivers). The stack is expected to work on Linux as well — task definitions and Helm charts are platform-agnostic — but the Linux path (e.g. minikube with kvm2/docker driver, update-ca-certificates in fleet:trust-ca) has not been exercised yet and needs verification. Reports / PRs from a Linux run welcome.

Required tools

Tool Purpose Install
minikube Local Kubernetes cluster brew install minikube
kubectl Cluster CLI brew install kubectl
helm Chart install / lint / packaging brew install helm
helmfile Declarative orchestration of helm releases brew install helmfile
task Task runner brew install go-task/tap/go-task
openssl JWT key generation preinstalled on macOS/Linux
curl Health checks preinstalled
docker Container runtime for the docker driver Docker Desktop / OrbStack / Colima
vfkit Apple Virtualization driver (macOS only) brew install cfergeau/crc/vfkit

Optional tools

Tool Used for
yamllint Pre-commit YAML linting (lefthook)
lefthook Git pre-commit hooks (brew install lefthook)
fleetctl Building Fleet enrollment packages for agents

Cluster sizing

Comfortable minimum for the full stack: 4 vCPU / 6 GiB RAM / 30 GiB disk — these are the defaults in Taskfile.yaml. Override via CLUSTER_* env vars when needed (e.g. CLUSTER_MEMORY=8192).

Quickstart

# 1. Create a local cluster (default driver = docker)
task cluster:create

# 2. Deploy MySQL + Valkey + cert-manager + ingress-nginx + Fleet
task fleet:install

# 3. Expose via nip.io — no /etc/hosts edits, HTTPS by default
#    The browser will warn about the self-signed cert on first visit —
#    just click through to accept it (or run `task fleet:trust-ca` for
#    a green lock; that one is opt-in and prompts for sudo).
task fleet:expose

# 4. macOS + docker driver only: open a SECOND terminal and run the tunnel.
#    This routes the cluster's LoadBalancer IP onto 127.0.0.1 so the
#    ingress (and `task fleet:verify`'s ingress check) is reachable
#    from the host. Keep it running in the background — Ctrl+C tears
#    the tunnel down. On Linux / vfkit driver the minikube IP is already
#    host-routable, so this step is unnecessary.
sudo true                           # cache sudo timestamp ahead of the tunnel
task fleet:tunnel                   # leave this terminal open

# 5. Back in the main terminal: verify the full stack
task fleet:verify

# 6. Open the URL printed by fleet:url and create the initial admin user
task fleet:url

To use Apple Virtualization instead of Docker (skip the task fleet:tunnel step above — vfkit's minikube IP is host-routable directly):

CLUSTER_DRIVER=vfkit task cluster:create

kubectl setup

task cluster:create invokes minikube start --profile fleet, which automatically writes the fleet context into ~/.kube/config and marks it as current. There is nothing to copy or merge by hand — open a fresh shell and kubectl already points at the new cluster.

Verify:

kubectl config current-context             # → fleet
kubectl cluster-info                        # → control plane at https://127.0.0.1:<port>
kubectl get nodes                           # → fleet  Ready  control-plane

Switching between clusters

If you also have ~/.kube/config entries from other clusters, swap with:

kubectl config get-contexts                 # list everything kubectl knows about
kubectl config use-context fleet            # switch to this project's cluster
kubectl config use-context <other>          # switch back

A nicer UX:

brew install kubectx                        # also installs `kubens`
kubectx fleet                               # switch context
kubens fleet                                # default namespace for subsequent commands

The same idea via minikube:

minikube -p fleet kubectl -- get pods -A    # explicit, ignores ~/.kube/config
minikube profile list                       # see every minikube profile on the box
minikube profile fleet                      # set the default profile for `minikube` commands

Talking to a stopped or paused cluster

After task cluster:stop (or a host reboot) the API server is gone and kubectl calls hang or refuse the connection. Start the cluster first — the kubeconfig entry survives stop/start cycles:

task cluster:start                          # = minikube start -p fleet
kubectl get nodes                           # works again

Fixing a broken context

If ~/.kube/config drifts (e.g. after deleting the cluster manually), re-derive the entry from the live minikube profile:

minikube -p fleet update-context            # rewrites server URL + cert refs
kubectl config use-context fleet

Tear-down

task cluster:delete removes the cluster and the kubeconfig entry, so you don't end up with orphan stale contexts.

Task reference

$ task --list-all
cluster:create         Create local minikube cluster (CLUSTER_DRIVER=docker|vfkit)
cluster:start          Start a previously created minikube cluster
cluster:stop           Stop the running minikube cluster
cluster:delete         Delete local minikube cluster

chart:install          Install or upgrade a Helm chart           # ad-hoc
chart:lint             Lint a Helm chart                          # ad-hoc
chart:uninstall        Uninstall a Helm release                   # ad-hoc

fleet:install          helmfile sync — deploy the full FleetDM stack
fleet:uninstall        helmfile destroy + drop namespaces
fleet:diff             Preview what `helmfile sync` would change
fleet:lint             Lint the helmfile and all referenced values
fleet:template         Render manifests without applying
fleet:expose           Publish Fleet via HTTPS ingress on a nip.io hostname
fleet:forward          Quick UI access via kubectl port-forward (localhost:8080)
fleet:tunnel           Bridge cluster IP to host (macOS/docker driver, sudo)
fleet:verify           Run end-to-end health checks (pods/MySQL/Valkey/Fleet/ingress)
fleet:status           Show pods, services, PVCs across namespaces + helmfile list
fleet:url              Print the Fleet server URLs (ingress / port-forward)
fleet:logs             Tail Fleet server logs
fleet:prepare-db       Re-run init-db + prepare-db Helm hook Jobs
fleet:trust-ca         Opt-in: import the local root CA into the host trust store (sudo)
fleet:untrust-ca       Remove the local root CA from the host trust store

Generic chart:install

A provider-agnostic Helm wrapper used by the Fleet tasks and reusable for anything else:

task chart:install \
    CHART_REF=oci://registry-1.docker.io/bitnamicharts/redis \
    CHART_RELEASE=my-redis \
    CHART_NAMESPACE=cache \
    CHART_VERSION=20.6.2 \
    CHART_VALUES=path/to/values.yaml
Var Required Notes
CHART_REF yes Chart reference (repo, OCI URL, tgz, dir)
CHART_RELEASE yes Helm release name
CHART_NAMESPACE no Default default; --create-namespace
CHART_VERSION no Pin chart version
CHART_VALUES no Path to a values file

Configuration cheatsheet

All knobs are env vars consumed by Task — no flags needed.

Cluster

Var Default Description
CLUSTER_NAME fleet Minikube profile name
CLUSTER_DRIVER docker docker or vfkit
CLUSTER_CPUS 4 vCPUs
CLUSTER_MEMORY 6144 RAM (MiB)
CLUSTER_DISK 30g Disk size
CLUSTER_K8S_VERSION stable Kubernetes version

Fleet stack

The version pins below live in helmfile.yaml.gotmpl under environments.default.values. They are not env-var overridable today — edit the helmfile if you need to bump them.

Setting Value Source
fleetChartVersion v6.9.4 fleetdm/fleet
valkeyChartVersion 0.9.4 valkey/valkey
ingressNginxChartVersion 4.13.2 ingress-nginx/ingress-nginx
certManagerChartVersion v1.16.2 jetstack/cert-manager

Task-level env vars (consumed only by tasks/*.yaml, not by the helmfile):

Var Default Description
FLEET_NAMESPACE fleet Namespace for the Fleet server
DATABASE_NAMESPACE database Namespace for MySQL
CACHE_NAMESPACE cache Namespace for Valkey
INGRESS_NAMESPACE ingress-nginx Namespace for the ingress controller
INGRESS_DNS_SUFFIX nip.io Wildcard-DNS suffix for fleet:expose
INGRESS_HOST (unset) Explicit ingress hostname; overrides the auto-derived nip.io host
CI (unset) When true, fleet:trust-ca is a no-op (no sudo prompts on CI)

Examples:

CLUSTER_DRIVER=vfkit CLUSTER_MEMORY=8192 task cluster:create
task fleet:install
INGRESS_DNS_SUFFIX=sslip.io task fleet:expose

How-to

Enrol an osquery agent

host=$(kubectl -n fleet get ingress fleet -o jsonpath='{.spec.rules[0].host}')
secret=$(fleetctl get enroll_secret --json | jq -r '.spec.secrets[0].secret')

fleetctl package --type=deb \
    --fleet-url="https://$host" \
    --enroll-secret="$secret" \
    --insecure   # only for the local self-signed root CA; drop in production

The same $host works from any machine that can resolve public DNS. --insecure is needed only because the dev stack signs the leaf cert with a self-signed root; trust the root (task fleet:trust-ca on the host running fleetctl) and --insecure can be dropped.

Re-run database migrations

init-db (creates the fleet schema + user) and prepare-db (runs fleet prepare db --no-prompt) are both Helm post-install hooks of the charts/fleet-db-init chart, with a TCP-wait initContainer that polls MySQL until it's accepting connections. To re-run both on demand:

task fleet:prepare-db    # = helmfile -l name=fleet-db-init sync

Publish your own charts

Drop a chart under charts/<name>/. On the next push to main, semantic-release cuts a tag and chart-releaser-action packages and publishes it to gh-pages as a Helm repository:

helm repo add hw https://oleg-tkachuk.github.io/flamingo-hw
helm repo update
helm search repo hw

TLS / cert-manager

The stack runs HTTPS-only because Fleet's UI bundle stores its auth token in a __Host-token cookie marked Secure. Browsers silently drop that cookie on plain HTTP — login succeeds, but every follow-up API call goes out as Authorization: Bearer null and the UI bounces back to /login with Authentication required. There is no server-side or chart-level switch to disable that behaviour.

Chain produced by charts/cert-manager-issuers/:

ClusterIssuer/fleet-selfsigned   (SelfSigned, bootstrap only)
        │
        └─► Certificate/fleet-root-ca         (isCA=true, 10y, in cert-manager ns)
                │
                └─► ClusterIssuer/fleet-ca    (CA, signs all leaf certs)
                        │
                        ├─► Ingress fleet (annotation: cert-manager.io/cluster-issuer: fleet-ca)
                        │       → Secret fleet-tls (auto-managed)
                        │
                        └─► ingress-nginx admission webhook serving cert
                                → Secret ingress-nginx-admission (auto-managed)
                                → caBundle injected by cainjector

The Fleet ingress and the ingress-nginx admission webhook are both auto-rotated by cert-manager. The local root CA stays cluster-resident until you explicitly task fleet:untrust-ca (and delete the secret).

The browser will warn on first visit (NET::ERR_CERT_AUTHORITY_INVALID or equivalent) because the root CA is not in your trust store. Just click through to accept it — the warning sticks per-host and you won't see it again on that browser profile.

If you want a green lock (e.g. when demoing), trust the root on the host once:

task fleet:trust-ca       # macOS: System keychain; Linux: /usr/local/share/ca-certificates
task fleet:untrust-ca     # symmetric cleanup

fleet:trust-ca is opt-in — it prompts for sudo to write to the system keychain, so it deliberately stays out of fleet:install. The task is idempotent (no-op if the cert is already trusted) and short- circuits on CI=true so it never fires from CI by accident.

Tear everything down

task fleet:uninstall    # removes both releases + namespace + PVCs
task cluster:delete     # destroys the minikube VM

CI / release pipeline

On every PR (.github/workflows/ci.yaml)

  1. yamllint over the whole repo.
  2. chart-testing (ct lint) on changed charts under charts/.
  3. helm dependency update + helm template smoke test for each changed chart.

On push to main (.github/workflows/release.yaml)

  1. semantic-release analyses Conventional Commits, produces a GitHub release and a vX.Y.Z tag.
  2. If a release was cut, chart-releaser-action runs helm package for every chart and pushes the index + tarballs to gh-pages, exposing the repo as a public Helm repository.

Branch protection should require both yamllint and Helm chart lint & template checks to pass before merge.

Local pre-commit hooks (lefthook.yml)

brew install lefthook
lefthook install

Runs on every git commit:

  • yamllint on staged YAML files
  • helm lint on charts whose Chart.yaml is staged
  • helm template on charts whose templates are staged
  • Conventional Commits enforcement on the commit message

Troubleshooting

Symptom Likely cause / fix
Login returns 200 then UI immediately shows Authentication required Accessing the UI over plain HTTP — Fleet's __Host-token cookie needs HTTPS. Re-run task fleet:expose.
Browser warns Your connection is not private Run task fleet:trust-ca to import the local root CA into the host keychain; restart the browser.
task fleet:verify reports SKIP: cannot reach ingress ... port 443 Run task fleet:tunnel in a second terminal (macOS + docker driver only; minikube's LoadBalancer needs the tunnel to bridge cluster IPs onto 127.0.0.1).
curl: (7) Failed to connect to fleet.*.nip.io Same as above — the LoadBalancer Service has no host-routable IP. Start task fleet:tunnel.
kubectl apply on an Ingress fails with x509: signed by unknown authority Stale ingress-nginx admission webhook with empty caBundle. Re-run helmfile -l name=ingress-nginx sync.
Prepare-db Job stays Pending MySQL not yet Ready — Job has a TCP-wait initContainer; check kubectl -n fleet describe job/fleet-prepare-db.
nip.io hostname doesn't resolve Air-gapped network — fall back to task fleet:forward or NodePort.
helm dep update 401 on OCI helm registry login to docker.io may be required for OCI rate-limits.

Inspect anything with:

task fleet:status     # pods, svc, pvc, ingress
task fleet:logs       # follow Fleet server logs
kubectl -n fleet get events --sort-by=.lastTimestamp

Component versions

Component Version Source
FleetDM server v4.84.3 docker.io/fleetdm/fleet
FleetDM chart v6.9.4 https://fleetdm.github.io/fleet/charts (fleetdm/fleet)
MySQL 8.4 docker.io/library/mysql via charts/mysql/
Valkey chart 0.9.4 https://valkey.io/valkey-helm/ (valkey/valkey)
ingress-nginx chart 4.13.2 https://kubernetes.github.io/ingress-nginx (ingress-nginx/ingress-nginx)
cert-manager chart v1.16.2 https://charts.jetstack.io (jetstack/cert-manager)

To bump anything, edit the pins under environments.default.values in helmfile.yaml.gotmpl.

Further reading

License

See repository LICENSE (TBD).

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 30

v1.0.5 Latest
May 11, 2026
+ 29 releases

Packages

 
 
 

Contributors

Languages