Skip to content

README.md

Latest commit

 

History

History
185 lines (149 loc) · 8.98 KB

README.md

File metadata and controls

185 lines (149 loc) · 8.98 KB
Raw

FleetDM on minikube — release graph and values

End-to-end deployment of FleetDM with its required backing services inside a local minikube cluster, following FleetDM's Option B — install MySQL and Redis/Valkey separately.

Orchestrated by Helmfile. All off-the-shelf components use official upstream Helm charts; the cluster-bootstrap, MySQL, schema-init and TLS-issuer pieces live as local charts under ../../charts/.

The declarative spec lives at ../../helmfile.yaml.gotmpl. This directory only holds the values files it references.

Components

Layer Namespace Release Chart
Bootstrap (cluster-wide) bootstrap ../../charts/bootstrap/
Ingress ingress-nginx ingress-nginx ingress-nginx/ingress-nginx
TLS — issuers cert-manager cert-manager jetstack/cert-manager
TLS — root CA cert-manager cert-manager-issuers ../../charts/cert-manager-issuers/
MySQL database mysql ../../charts/mysql/ (official mysql:8.4 image)
Schema bootstrap fleet fleet-db-init ../../charts/fleet-db-init/
Valkey cache cache valkey valkey/valkey
Fleet server fleet fleet fleetdm/fleet

Cross-namespace references used by Fleet:

  • MySQL: mysql.database.svc.cluster.local:3306
  • Cache: valkey.cache.svc.cluster.local:6379

The mysql-credentials Secret is mirrored into both database (for the init-db Job) and fleet (for the prepare-db Job and the Fleet pod) by the bootstrap chart.

Release DAG (needs: from helmfile.yaml.gotmpl)

bootstrap ──┬─► ingress-nginx ◄─── needs cert-manager-issuers
            │
            ├─► cert-manager ─► cert-manager-issuers ──┐
            │                                          │
            ├─► mysql ──► fleet-db-init ──┐            │
            │                             ├──► fleet ◄─┘
            └─► valkey ───────────────────┘
  1. bootstrap — creates the four workload namespaces and the mysql-credentials + Fleet JWT Secrets. Auto-generates the JWT on first install and preserves it on upgrades via Helm lookup.
  2. cert-manager — official jetstack/cert-manager chart with CRDs.
  3. cert-manager-issuers — fleet-selfsigned ClusterIssuer bootstraps a self-issued root CA Certificate (fleet-root-ca, 10y, in the cert-manager namespace), which the fleet-ca ClusterIssuer then uses to sign every leaf cert in the cluster.
  4. ingress-nginx — LoadBalancer Service. Admission-webhook serving cert is provisioned by cert-manager via controller.admissionWebhooks.certManager.enabled=true; the chart's default -create / -patch Helm-hook Jobs are disabled to eliminate the race where the ValidatingWebhookConfiguration would briefly exist with an empty caBundle.
  5. mysql — single-pod StatefulSet running the official mysql:8.4 Docker image. PVC-backed, root password lives in mysql-credentials.
  6. fleet-db-init — init-db Job creates the fleet schema + user; prepare-db Job runs fleet prepare db --no-prompt. Both have a busybox TCP-wait initContainer and run as Helm post-install hooks.
  7. valkey — official chart, standalone, no auth (local dev only).
  8. fleet — official fleetdm/fleet chart. Ingress annotation cert-manager.io/cluster-issuer: fleet-ca makes cert-manager mint a leaf cert into Secret/fleet-tls automatically.

Prerequisites

  • A running minikube cluster — task cluster:create
  • helm, kubectl, curl on PATH
  • Cluster sized for ≥ 4 vCPU / 6 GiB / 30 GiB (defaults in Taskfile)
  • Internet access to pull official images and charts

Install

task fleet:install        # = helmfile sync + idempotent fleet:trust-ca

Or invoke helmfile directly:

helmfile sync             # apply
helmfile diff             # preview pending changes
helmfile template         # render manifests without applying
helmfile destroy          # tear down all releases

Expose to UI and agents

task fleet:expose      # nip.io HTTPS ingress (UI + agents)
task fleet:forward     # kubectl port-forward (UI only, HTTP localhost:8080)
task fleet:url         # show ingress / port-forward URLs

TLS is mandatory for the UI. Fleet's JS bundle stores the auth token in a __Host-token cookie marked Secure, which browsers refuse to set on plain HTTP. Run task fleet:trust-ca once to import the local root CA into the host trust store and the browser stops warning.

Verify

task fleet:verify
  1. Pods — every pod Running + Ready across database, cache, fleet.
  2. MySQL — mysqladmin ping from the StatefulSet; table count in the fleet schema confirms fleet prepare db ran.
  3. Valkey — valkey-cli PING (with redis-cli fallback); SET/GET round-trip.
  4. Fleet server — GET /healthz, GET /version, GET /api/v1/setup.
  5. Ingress (skipped if fleet:expose not run) — GET /healthz via the published host.

Re-run migrations

task fleet:prepare-db     # = helmfile -l name=fleet-db-init sync

Re-runs both Jobs (init-db and prepare-db) by upgrading the fleet-db-init release, which re-fires the post-install hooks.

Uninstall

task fleet:uninstall

Removes every helm release (fleet, fleet-db-init, valkey, mysql, cert-manager-issuers, cert-manager, ingress-nginx, bootstrap), the workload namespaces, and the PVCs.

Customisation

Version pins (chart versions) are set in ../../helmfile.yaml.gotmpl under environments.default.values — edit the helmfile to bump them. The task-level env vars below tweak only what Task itself does:

Variable Default Purpose
FLEET_NAMESPACE fleet Namespace for the Fleet server
DATABASE_NAMESPACE database Namespace for MySQL
CACHE_NAMESPACE cache Namespace for Valkey
INGRESS_NAMESPACE ingress-nginx Namespace for the ingress controller
INGRESS_DNS_SUFFIX nip.io Wildcard-DNS suffix used by fleet:expose
INGRESS_HOST (unset) Explicit ingress hostname; overrides the auto-derived nip.io
FLEET_SKIP_TRUST 0 1 skips the auto trust-ca step inside fleet:install
CI (unset) When true, fleet:trust-ca is a no-op (no sudo prompts)

Files

Notes

  • Single-pod MySQL is a dev profile. Production wants a real cluster (Vitess, an InnoDBCluster on a chart that lets us disable Group Replication's DROP PRIMARY KEY check, or a managed service).
  • The MySQL root password lives in Secret/mysql-credentials and is defaulted to a placeholder by the bootstrap chart — provision the Secret out of band for anything but local dev.
  • The local self-signed root CA never leaves the cluster, but it is imported into the host trust store by fleet:trust-ca. Use fleet:untrust-ca for symmetric cleanup before tearing down.
  • Fleet's server-side TLS is intentionally disabled here — TLS terminates at the ingress and the fleet-service ClusterIP is plain-HTTP inside the cluster.