Description
Since KEDA 2.20.0, the release manifests bundled by keda-olm-operator (resources/keda.yaml) grant the keda-operator the events resource only on the events.k8s.io API group (kedacore/keda#7781). As a result, shortly after startup the keda-operator pod logs:
event.go:359 "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:keda:keda-operator\" cannot create resource \"events\" in API group \"\" in the namespace \"keda\"" event=...
The denial is genuine: the rendered keda-operator ClusterRole only grants the events resource on the events.k8s.io API group:
- apiGroups: [events.k8s.io]
resources: [events]
verbs: [create, patch]
There is no rule granting events on the core ("") API group.
Why this fails
This is the trap described in kubernetes/kubernetes#94857: client-go's event broadcaster (and controller-runtime's leader-election event recorder) still writes events to the legacy core "" API group even when configured for events.k8s.io. Granting only the new group is insufficient.
The upgrade notes for KEDA 2.20.0 (kedacore/keda#7781) migrated the event recorder to events.k8s.io, but the core group entry appears to have been dropped rather than retained. The internal dependencies (leader election) that still emit core events were not accounted for.
Impact
Native Kubernetes event recording is denied — including controller-runtime leader-election events and ScaledObject events such as ScaledObjectReady, KEDAScalersStarted and ScaledObjectCheckFailed. Scaling itself is unaffected; the effect is cosmetic (kubectl describe scaledobject shows fewer events than it should), but the logs are noisy with forbidden errors.
Reproduction
- Deploy KEDA via keda-olm-operator (KEDA 2.20.x manifests).
- Check
kubectl get clusterrole keda-operator -o json | jq '.rules[] | select(.resources[]? == "events")' — only events.k8s.io is granted.
- Watch
kubectl -n keda logs deploy/keda-operator | grep "events is forbidden".
Suggested fix
Restore the core "" API group alongside events.k8s.io in the operator ClusterRole rule:
- apiGroups: ["", "events.k8s.io"]
resources: [events]
verbs: [create, patch]
This is the same regression fixed on the Helm chart side in kedacore/charts#886 (reported in kedacore/charts#883). The equivalent fix for keda-olm-operator is proposed in #316.
Versions
- keda-olm-operator manifests: KEDA
2.20.1
- Operator image:
kedacore/keda:2.20.x
Description
Since KEDA 2.20.0, the release manifests bundled by keda-olm-operator (
resources/keda.yaml) grant thekeda-operatortheeventsresource only on theevents.k8s.ioAPI group (kedacore/keda#7781). As a result, shortly after startup the keda-operator pod logs:The denial is genuine: the rendered
keda-operatorClusterRole only grants theeventsresource on theevents.k8s.ioAPI group:There is no rule granting
eventson the core ("") API group.Why this fails
This is the trap described in kubernetes/kubernetes#94857: client-go's event broadcaster (and controller-runtime's leader-election event recorder) still writes events to the legacy core
""API group even when configured forevents.k8s.io. Granting only the new group is insufficient.The upgrade notes for KEDA 2.20.0 (kedacore/keda#7781) migrated the event recorder to
events.k8s.io, but the core group entry appears to have been dropped rather than retained. The internal dependencies (leader election) that still emit core events were not accounted for.Impact
Native Kubernetes event recording is denied — including controller-runtime leader-election events and ScaledObject events such as
ScaledObjectReady,KEDAScalersStartedandScaledObjectCheckFailed. Scaling itself is unaffected; the effect is cosmetic (kubectl describe scaledobjectshows fewer events than it should), but the logs are noisy with forbidden errors.Reproduction
kubectl get clusterrole keda-operator -o json | jq '.rules[] | select(.resources[]? == "events")'— onlyevents.k8s.iois granted.kubectl -n keda logs deploy/keda-operator | grep "events is forbidden".Suggested fix
Restore the core
""API group alongsideevents.k8s.ioin the operator ClusterRole rule:This is the same regression fixed on the Helm chart side in kedacore/charts#886 (reported in kedacore/charts#883). The equivalent fix for keda-olm-operator is proposed in #316.
Versions
2.20.1kedacore/keda:2.20.x