Skip to content

fix(nodes): validate every certificate in the node mTLS trust bundle - #6188

Open
n0ctal wants to merge 1 commit into
MHSanaei:mainfrom
n0ctal:upstream-strict-mtls-ca-bundle
Open

fix(nodes): validate every certificate in the node mTLS trust bundle#6188
n0ctal wants to merge 1 commit into
MHSanaei:mainfrom
n0ctal:upstream-strict-mtls-ca-bundle

Conversation

@n0ctal

@n0ctal n0ctal commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Parse and validate every certificate in the node mTLS trust bundle instead of only the first one, and reject the bundle if any block is malformed.

Why

Two places accept the nodeMtlsClientCAPem trust bundle, and both are satisfied by a bundle that is only partly valid:

  • NodeMtlsClientCAPool builds the listener's ClientCAs with x509.CertPool.AppendCertsFromPEM, which returns true as soon as one certificate parses. Later entries that are damaged or truncated are skipped, and the pool is returned as if nothing were wrong.
  • SetNodeMtlsTrustCA validates the value before storing it, but pem.Decode returns only the first block and the remainder is discarded, so only the first certificate is ever checked.

Together these mean a bundle can be stored, reported as valid, and then produce a trust pool that is missing certificates nobody was told about.

This matters most during a CA rotation, which is the one time a bundle legitimately holds two certificates. The operator stores old CA + new CA, both calls report success, and the new CA is silently absent from the pool. Every node then rejects the panel's new client certificate at once — and the failure surfaces only when the panel next needs the node API, because data-plane traffic does not go through it.

Scope

  • New parseCertificateBundlePEM walks the whole bundle, requiring every block to be a parseable CERTIFICATE, and rejects an empty bundle, a non-certificate block, and trailing non-PEM bytes.
  • NodeMtlsClientCAPool builds the pool from the parsed certificates with AddCert, so a partially valid bundle is an error rather than a quietly reduced pool.
  • SetNodeMtlsTrustCA validates through the same helper, so what is accepted on write is exactly what will load on read.
  • No schema, API, or configuration change. A bundle that was fully valid before is still accepted and produces the same pool.

Validation

  • go build ./..., go vet, golangci-lint run clean on main at ece16559.
  • go test ./internal/web/service/ green.
  • New table-driven tests cover a single certificate, a valid two-certificate bundle, an empty and whitespace-only bundle, a valid certificate followed by a corrupt one, trailing non-PEM data, and a non-certificate PEM block.
  • The pool test was confirmed failing-first: against the previous AppendCertsFromPEM implementation it fails, reporting a pool built from one certificate while the second was dropped; it passes with this change.

Risk

Low. The change is strictly a tightening of validation on a value that is expected to contain only certificates. Existing single-certificate configurations are unaffected.

One behaviour worth calling out: a stored bundle with trailing text after the last certificate used to load, and now will not. Operators who paste output from tools that append human-readable text would see the trust pool fail to build. That failure is currently non-fatal — web.go logs a warning and starts the listener without mTLS, falling back to bearer-token auth for the node API — so a bad bundle degrades the node API from two factors to one rather than stopping the panel. I think that fallback deserves to fail closed instead, but it predates this change and I did not want to fold an unrelated behaviour change into a validation fix. Happy to send it separately if you agree.

AppendCertsFromPEM reports success once a single certificate parses, so a
trust bundle whose later entries are damaged or truncated was accepted with
those entries silently absent from the pool. Parse and validate every PEM
block instead, and reject the bundle if any of them is malformed.
@n0ctal
n0ctal force-pushed the upstream-strict-mtls-ca-bundle branch from e64ec1c to a738507 Compare August 8, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant