Skip to content

fix(dpf): start without the site-wide BMC root credential - #4172

Merged
nv-dmendoza merged 1 commit into
NVIDIA:release/v2.0from
wminckler:backport/v2.0/dpf-init-without-bmc-root-cred
Jul 27, 2026
Merged

fix(dpf): start without the site-wide BMC root credential#4172
nv-dmendoza merged 1 commit into
NVIDIA:release/v2.0from
wminckler:backport/v2.0/dpf-init-without-bmc-root-cred

Conversation

@wminckler

Copy link
Copy Markdown
Contributor

Backport of #4167 onto release/v2.0.

Operators set the site-wide BMC root credential through the API, so on a fresh
site it does not exist when nico-api first boots. DPF SDK initialization read
that credential unconditionally and propagated the failure out of setup,
aborting the boot before the listener started — which left no way to set the
credential that would have let the next boot succeed. A fresh site with DPF
enabled could not be brought up at all.

This makes the initial read best-effort when a BMC password refresh interval is
configured: initialization continues without the Secret, and the existing 60s
refresh task writes it as soon as the credential appears, so no restart is
needed. With no refresh task configured nothing would ever retry, so there the
failure stays fatal.

The refresh task could not have recovered as written: create_secret returned
early when the Secret already existed, so a rotated password was never
propagated to the cluster. It is renamed to apply_secret and implemented as a
server-side apply, matching apply_configmap.

All ten source files cherry-picked onto the release/v2.0 base with no
conflicts. The documentation changes carried by #4167 are omitted here, as
release branches do not carry docs.

Related issues

#4130

Type of Change

  • Add - New feature or capability
  • Change - Changes in existing functionality
  • Fix - Bug fixes
  • Remove - Removed features or deprecated functionality
  • Internal - Internal changes (refactoring, tests, docs, etc.)

Breaking Changes

  • This PR contains breaking changes

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • Manual testing performed
  • No testing required (docs, internal refactor, etc.)

cargo test -p carbide-dpf on this branch: 98 passed, 0 failed. That includes
the four new tests, each confirmed by name — the refresh task writing the Secret
once the password appears, staying unwritten while it is unavailable, and build
succeeding vs. failing when the password is unset with and without a refresh
interval configured.

The equivalent change on main was additionally verified against a local kind
cluster running the DPF operator, with nico-api pointed at a Vault mount holding
the UEFI site defaults but no machines/bmc/site/root. Startup reached DPF SDK
initialization instead of aborting at the credential read:

level=ERROR  The Vault server returned an error (status code 404)
level=WARN   site-wide BMC root not set; deferring lockdown IKM seed until it is configured
level=INFO   Initializing DPF SDK

That verification was performed on the main-based branch, not on this
release/v2.0 backport.

Additional Notes

The repository pre-commit hook and pre-commit-verify-workspace both invoke
cargo xtask lint-error-messages, which does not exist on release/v2.0, so
neither could run against this branch. CI is the first lint signal here.

🤖 Generated with Claude Code

Operators set the site-wide BMC root credential through the API, so on a
fresh site it does not exist when nico-api first boots. DPF SDK init read
it unconditionally and propagated the failure out of setup, aborting the
boot before the listener started -- leaving no way to set the credential
that would let the next boot succeed.

Make the initial read best-effort when a BMC password refresh interval is
configured: initialization continues without the Secret and the existing
60s refresh task writes it as soon as the credential appears, so no
restart is needed. With no refresh task nothing would ever retry, so
there the failure stays fatal.

The refresh task could not have recovered as written: create_secret
returned early when the Secret already existed, so a rotated password was
never propagated to the cluster. Rename it to apply_secret and implement
it as a server-side apply, matching apply_configmap.

Backport of 2d2eaea onto release/v2.0. The documentation changes from
that commit are omitted; release branches do not carry docs.

Fixes NVIDIA#4130

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wminckler
wminckler requested a review from a team July 26, 2026 16:41
@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1a8b6eb6-d7f7-4e72-8785-4b3742a5b514

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

🔐 TruffleHog Secret Scan

No secrets or credentials found!

Your code has been scanned for 700+ types of secrets and credentials. All clear! 🎉

🔗 View scan details

🕐 Last updated: 2026-07-26 16:44:38 UTC | Commit: b6ef000

@github-actions

github-actions Bot commented Jul 26, 2026

Copy link
Copy Markdown

🔍 Container Scan Summary

Service Total Critical High Medium Low Other
boot-artifacts-aarch64 3 0 0 3 0 0
boot-artifacts-x86_64 3 0 0 3 0 0
forge-admin-cli-x86_64 266 16 33 80 8 129
machine-validation-runner 558 41 125 192 18 182
machine_validation 558 41 125 192 18 182
machine_validation-aarch64 558 41 125 192 18 182
nvmetal-carbide 558 41 125 192 18 182
TOTAL 2504 180 533 854 80 857

Per-CVE detail lives in the per-service grype-* artifacts (JSON + SARIF). Severity counts only — no CVE IDs published here.

@nv-dmendoza
nv-dmendoza merged commit d77950a into NVIDIA:release/v2.0 Jul 27, 2026
120 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants