Skip to content

fix(apiserver): probe SA permissions before loading admission-policy plugins#1842

Open
neel5481 wants to merge 1 commit into
carvel-dev:developfrom
neel5481:fix/1841-admission-policy-rbac-probe
Open

fix(apiserver): probe SA permissions before loading admission-policy plugins#1842
neel5481 wants to merge 1 commit into
carvel-dev:developfrom
neel5481:fix/1841-admission-policy-rbac-probe

Conversation

@neel5481

@neel5481 neel5481 commented Jul 8, 2026

Copy link
Copy Markdown

Fixes #1841

On Kubernetes 1.36+ the MutatingAdmissionPolicy admission plugin ships enabled by default. Its SharedInformer requires list+watch permission on mutatingadmissionpolicies and mutatingadmissionpolicybindings. If the deployed ClusterRole for the kapp-controller ServiceAccount lacks those permissions, the informer's cache never syncs, HasSynced() never returns true, and every mutating admission request into the aggregated apiserver fails permanently with 'not yet ready to handle request (reason: Forbidden)'.

Signed-off-by: Neel Patel

What this PR does / why we need it:

The existing serverResourceExists guard only confirms the API resource is present in cluster discovery, which is always true on k8s 1.36+ regardless of RBAC. This change extends the guard with a SelfSubjectAccessReview probe for both list and watch on the policy resource and its bindings resource before loading each admission-policy plugin. When the probe fails the plugin is skipped and an actionable warning is logged naming the missing (resource, verb) pair.

The probe uses subjectaccessreviews/create, which is already granted in the shipped ClusterRole, no new RBAC required.

Also applied to ValidatingAdmissionPolicy for symmetry; the same failure mode is possible on k8s 1.30+ if RBAC drifts.

Which issue(s) this PR fixes:

Fixes #1841

Does this PR introduce a user-facing change?


Additional Notes for your reviewer:

Review Checklist:
  • Follows the developer guidelines
  • Relevant tests are added or updated
  • Relevant docs in this repo added or updated
  • Relevant carvel.dev docs added or updated in a separate PR and there's
    a link to that PR
  • Code is at least as readable and maintainable as it was before this
    change

Additional documentation e.g., Proposal, usage docs, etc.:


@carvel-bot carvel-bot added this to Carvel Jul 8, 2026
…plugins

Fixes carvel-dev#1841

On Kubernetes 1.36+ the MutatingAdmissionPolicy admission plugin ships enabled
by default. Its SharedInformer requires list+watch permission on
mutatingadmissionpolicies and mutatingadmissionpolicybindings. If the deployed
ClusterRole for the kapp-controller ServiceAccount lacks those permissions,
the informer's cache never syncs, HasSynced() never returns true, and every
mutating admission request into the aggregated apiserver fails permanently
with 'not yet ready to handle request (reason: Forbidden)'.

The existing serverResourceExists guard only confirms the API resource is
present in cluster discovery, which is always true on k8s 1.36+ regardless
of RBAC. This change extends the guard with a SelfSubjectAccessReview probe
for both list and watch on the policy resource and its bindings resource
before loading each admission-policy plugin. When the probe fails the
plugin is skipped and an actionable warning is logged naming the missing
(resource, verb) pair.

The probe uses subjectaccessreviews/create, which is already granted in the
shipped ClusterRole, no new RBAC required.

Also applied to ValidatingAdmissionPolicy for symmetry; the same failure
mode is possible on k8s 1.30+ if RBAC drifts.

Signed-off-by: Neel Patel <neel5481@gmail.com>
@neel5481 neel5481 force-pushed the fix/1841-admission-policy-rbac-probe branch from b8bac11 to 4b45307 Compare July 8, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Aggregated apiserver rejects all writes with "not yet ready to handle request" on k8s 1.36+ when SA lacks RBAC for MutatingAdmissionPolicy

2 participants