Secure multi-tenant secret provider integration in ArgoCD

[NiklasRosenstein]

Preamble

Nyl can inject secrets into manifests using the ${{ secrets.get("...") }} syntax. The secrets object is an implemenetation of the SecretProvider class. Usually it will require some sort of initial secret to initialize the provider, for example:

• A private key when using SOPS (e.g. SOPS_AGE_KEY)
• A VAULT_TOKEN for accessing Vault
• Access to a Kubernetes API server when reading from a Kubnernetes secret
• etc.

Environment variables like the SOPS_AGE_KEY are currently passed as global environment variables to the nyl-v1 CMP:

https://github.com/NiklasRosenstein/nyl/blob/385f79bfd01143df3ee408dbf2c3344a0084c572/argocd-with-nyl/argocd.yaml#L12-L20

Issues

Template evaluation

For one, there is an open issue with the ability to execute arbitrary code with the current implementation of structured-templates, which would allow anyone to write code to look up environment variables or peek into the system where Nyl is running as an ArgoCD plugin. It would mean that evaluating untrusted Kubernetes manifests in ArgoCD with Nyl could be used to extract the SOPS_AGE_KEY and vice versa.

• Improve template evaluation security to prevent arbitrary code execution #48

Shared initial secrets

Currently, only a single instance of environment variables like SOPS_AGE_KEY and vice versa are set in the Nyl CMP. This means that all parties whose Kubernetes manifests are evaluated by the Nyl plugin share the same secret, and thus either the same capability to decrypt or access the same set of secrets.

This renders it unusable if at least one party is not trusted.

Imagine a scenario with two teams, they both store their secrets in a SOPS file in their own repositories. They each need to encrypt their SOPS file with a public key that Nyl can decrypt, but because there is only one SOPS_AGE_KEY, they each need to encrypt it with the same key.

Now, they can simply copy & paste the encrypted SOPS file of the other team into their own repository and decrypt its secrets into a rendered Kubernetes manifests using Nyl inside ArgoCD.

Proposed solution

Nyl can look up the ArgoCD AppProject and Application that it is being invoked for in the local Kubernetes cluster (see ArgoCD Build Environment). These then have properties that inform it about the initial secret to unlock the SecretProvider. This is very compatible with how ArgoCD functions in other areas as well (e.g. an AppProject points to a Kubernetes cluster, and ArgoCD looks up the access credentials for it from a Kubernetes secret).

When invoked from ArgoCD, Nyl will lookup the AppProject's annotations to find its settings. This is secure, because AppProject resources must only be configurable by ArgoCD administrators.

For example:

---
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: sample-test-project
  annotations:
    nyl.io/environment: sample-test-project-nyl-env
spec:
  ...

---
apiVersion: v1
kind: Secret
metadata:
  name: sample-test-project-nyl-env
data:
  SOPS_AGE_KEY: ...