Skip to content

fix: pin otelcol build, rest-api CLI tools in support of hermetic builds#4129

Open
ianderson-nvidia wants to merge 1 commit into
NVIDIA:mainfrom
ianderson-nvidia:hermetic_build_part1
Open

fix: pin otelcol build, rest-api CLI tools in support of hermetic builds#4129
ianderson-nvidia wants to merge 1 commit into
NVIDIA:mainfrom
ianderson-nvidia:hermetic_build_part1

Conversation

@ianderson-nvidia

@ianderson-nvidia ianderson-nvidia commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

The docker-build-otelcol Dockerfile and the build-otelcol cargo-make task previously ran ocb without --skip-get-modules, causing go mod tidy to execute inside every build. Because go mod tidy queries and possibly updates Go modules, two builds from the same commit could resolve different indirect dependency versions and produce different binaries.

Both build paths now run OCB with --skip-compilation --skip-get-modules to generate Go source only, then overwrite the incomplete OCB-generated go.mod/go.sum with the committed files from bluefield/otel/ocb-generated/ before calling go build -mod=mod. The include-otelcol task condition is updated to invalidate the cached binary when the committed module files change.

The trade-off: developers must run bluefield/otel/update-ocb-modules.sh and commit the outputs whenever the OCB config or component versions change. This is a manual step, but it makes the build reproducible and prevents silent dependency drift.

rest-api/tools/tools.go (new) pins buf, protoc-gen-go, protoc-gen-go-grpc, golangci-lint, and revive in go.mod. rest-api/Makefile is updated to invoke these tools via $(BUF), $(GOLANGCI_LINT), and $(REVIVE) using go run.

protoc-gen-go and protoc-gen-go-grpc must be real executables on PATH because buf generate spawns them as subprocesses — go run cannot be used directly. A .tools-stamp target builds them from the pinned go.mod versions into .tools/ and injects that directory into PATH for core-protogen and flow-protogen. The manual version-check guards previously in core-protogen are removed; the version is now enforced by go.mod rather than runtime assertions.

The Install buf and protoc plugins steps in ci.yaml and rest-lint-and-test.yml are removed — make core-proto now handles tool installation via .tools-stamp. The go install github.com/mgechev/revive@v1.15.0 call in the style CI job is replaced with go run github.com/mgechev/revive so the version from go.mod is used.

Type of Change

  • Add - New feature or capability
  • Change - Changes in existing functionality
  • Fix - Bug fixes
  • Remove - Removed features or deprecated functionality
  • Internal - Internal changes (refactoring, tests, docs, etc.)

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • Manual testing performed
  • No testing required (docs, internal refactor, etc.)

@ianderson-nvidia
ianderson-nvidia requested review from a team and lauragra-y as code owners July 24, 2026 20:07
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3cc65bb2-4107-49d6-8de5-96c52d61a638

📥 Commits

Reviewing files that changed from the base of the PR and between 4b708bd and 36701f4.

⛔ Files ignored due to path filters (2)
  • bluefield/otel/ocb-generated/go.sum is excluded by !**/*.sum
  • rest-api/go.sum is excluded by !**/*.sum
📒 Files selected for processing (14)
  • .github/workflows/ci.yaml
  • .github/workflows/rest-lint-and-test.yml
  • .gitignore
  • bluefield/Makefile.toml
  • bluefield/containers/otelcol-contrib/Dockerfile
  • bluefield/otel/README.md
  • bluefield/otel/ocb-generated/go.mod
  • bluefield/otel/otelcol_builder_config_yaml.txt
  • bluefield/otel/update-ocb-modules.sh
  • rest-api/.gitignore
  • rest-api/Makefile
  • rest-api/go.mod
  • rest-api/tools/README.md
  • rest-api/tools/tools.go
💤 Files with no reviewable changes (1)
  • .github/workflows/ci.yaml
🚧 Files skipped from review as they are similar to previous changes (12)
  • rest-api/.gitignore
  • rest-api/tools/README.md
  • bluefield/otel/otelcol_builder_config_yaml.txt
  • rest-api/tools/tools.go
  • bluefield/otel/README.md
  • bluefield/otel/update-ocb-modules.sh
  • .gitignore
  • .github/workflows/rest-lint-and-test.yml
  • bluefield/otel/ocb-generated/go.mod
  • bluefield/Makefile.toml
  • rest-api/go.mod
  • rest-api/Makefile

Summary by CodeRabbit

  • New Features

    • Added reproducible OpenTelemetry Collector builds for Linux ARM64 using pinned module definitions.
    • Added local, version-pinned tooling for protobuf generation and Go linting.
    • Added commands to build and clean local development tools.
  • Documentation

    • Documented OpenTelemetry module regeneration and development tool version management.
  • Bug Fixes

    • Improved consistency between local builds and CI checks by using pinned tooling and dependencies.

Walkthrough

The PR standardizes pinned REST tooling and protobuf generation, removes redundant CI tool installation, and introduces a committed OpenTelemetry Collector module graph used by local and Docker builds. It also adds regeneration documentation and ignores local development metadata.

Changes

REST tooling reproducibility

Layer / File(s) Summary
REST tool pinning contract
rest-api/go.mod, rest-api/tools/tools.go, rest-api/tools/README.md
Tool versions are pinned through Go dependencies and blank imports, with documentation for updates and Makefile usage.
Pinned lint and protobuf execution
rest-api/Makefile, .github/workflows/ci.yaml, .github/workflows/rest-lint-and-test.yml, rest-api/.gitignore
Lint commands use pinned tools, protobuf generators are built into .tools/, and CI no longer installs duplicate tool binaries.

OpenTelemetry module reproducibility

Layer / File(s) Summary
Generated module graph
bluefield/otel/otelcol_builder_config_yaml.txt, bluefield/otel/ocb-generated/go.mod, bluefield/otel/update-ocb-modules.sh, bluefield/otel/README.md
The Collector module is named, dependency-pinned, locally replaced where required, regenerated by script, and documented.
Pinned Collector compilation
bluefield/Makefile.toml, bluefield/containers/otelcol-contrib/Dockerfile
Builds generate sources first, copy committed module files, and explicitly cross-compile the arm64 Collector binary.

Repository hygiene

Layer / File(s) Summary
Ignore local metadata
.gitignore
Adds ignore patterns for session, environment, direnv, pre-commit, and tool artifacts.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

REST tool execution

sequenceDiagram
  participant Makefile
  participant Buf
  participant ProtobufPlugins
  Makefile->>ProtobufPlugins: Build protoc-gen-go and protoc-gen-go-grpc
  Makefile->>Buf: Run pinned buf generate
  Buf->>ProtobufPlugins: Invoke local protobuf plugins
Loading

OpenTelemetry Collector build

sequenceDiagram
  participant BuildTask
  participant OCB
  participant GoBuild
  BuildTask->>OCB: Generate sources without compilation
  BuildTask->>GoBuild: Copy committed go.mod and go.sum
  GoBuild->>GoBuild: Cross-compile otelcol-contrib for linux/arm64
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes pinning OTel builds and REST API CLI tools for hermetic builds.
Description check ✅ Passed The description is detailed and directly aligned with the repository changes, especially reproducible builds and pinned tooling.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

🔐 TruffleHog Secret Scan

No secrets or credentials found!

Your code has been scanned for 700+ types of secrets and credentials. All clear! 🎉

🔗 View scan details

🕐 Last updated: 2026-07-24 20:10:30 UTC | Commit: d205ef9

@ianderson-nvidia ianderson-nvidia changed the title fix: pin otelcol build, rest-api CLI tools fix: pin otelcol build, rest-api CLI tools in support of hermetic builds Jul 24, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.gitignore:
- Around line 63-64: Remove the .pre-commit-config.yaml entry from .gitignore so
the repository-wide pre-commit configuration remains tracked. Do not add a
replacement ignore rule; local-only exclusions should use a personal ignore
filename or .git/info/exclude.

In `@bluefield/Makefile.toml`:
- Around line 297-300: Update the Go build commands in bluefield/Makefile.toml
lines 297-300 and bluefield/containers/otelcol-contrib/Dockerfile lines 80-85 to
use -mod=readonly instead of -mod=mod. Apply the same change at both sites so
builds fail when the pinned module graph is incomplete without modifying go.mod
or go.sum.

In `@bluefield/otel/ocb-generated/go.mod`:
- Line 389: Update the generator inputs or dependency override controlling
google.golang.org/grpc, requiring version 1.82.1 or later, then regenerate the
generated go.mod and go.sum files. Do not manually edit
bluefield/otel/ocb-generated/go.mod; verify the generated dependency no longer
uses v1.81.1.

In `@bluefield/otel/update-ocb-modules.sh`:
- Line 65: Correct the user-facing commit command in update-ocb-modules.sh by
changing “Updatingd” to “Updating” and adding the missing closing single quote
after ${VERSION}, while preserving the surrounding echo behavior and shell
quoting.

In `@rest-api/tools/tools.go`:
- Line 15: Align all Buf version references to v1.72.0: update the stale version
comment in rest-api/tools/tools.go at line 15 and the README example in
rest-api/tools/README.md at lines 10-16. Leave the already-correct
rest-api/go.mod reference at line 18 unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 975a2198-5f53-4975-b37e-652b593fd9f8

📥 Commits

Reviewing files that changed from the base of the PR and between a991005 and d205ef9.

⛔ Files ignored due to path filters (2)
  • bluefield/otel/ocb-generated/go.sum is excluded by !**/*.sum
  • rest-api/go.sum is excluded by !**/*.sum
📒 Files selected for processing (16)
  • .github/workflows/ci.yaml
  • .github/workflows/rest-lint-and-test.yml
  • .gitignore
  • bluefield/Makefile.toml
  • bluefield/containers/otelcol-contrib/Dockerfile
  • bluefield/otel/README.md
  • bluefield/otel/ocb-generated/go.mod
  • bluefield/otel/otelcol_builder_config_yaml.txt
  • bluefield/otel/update-ocb-modules.sh
  • rest-api/.tools-stamp
  • rest-api/.tools/protoc-gen-go
  • rest-api/.tools/protoc-gen-go-grpc
  • rest-api/Makefile
  • rest-api/go.mod
  • rest-api/tools/README.md
  • rest-api/tools/tools.go
💤 Files with no reviewable changes (1)
  • .github/workflows/ci.yaml

Comment thread .gitignore
Comment thread bluefield/Makefile.toml Outdated
Comment thread bluefield/otel/ocb-generated/go.mod Outdated
Comment thread bluefield/otel/update-ocb-modules.sh Outdated
Comment thread rest-api/tools/tools.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
bluefield/otel/ocb-generated/go.mod (1)

389-389: ⚠️ Potential issue | 🟠 Major

Regenerate with google.golang.org/grpc 1.82.1 or later.

The generated graph still pins google.golang.org/grpc v1.81.1. The advisory marks versions below 1.82.1 as affected and 1.82.1 as patched, including xDS RBAC authorization-bypass and HTTP/2 denial-of-service risks. (github.com)

Update the OCB dependency inputs or override, then regenerate both go.mod and go.sum; do not hand-edit this generated file. This duplicates the unresolved previous review finding.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@bluefield/otel/ocb-generated/go.mod` at line 389, Update the OCB dependency
inputs or override to require google.golang.org/grpc v1.82.1 or later, then
regenerate the generated go.mod and go.sum files so the dependency graph no
longer contains v1.81.1; do not hand-edit the generated output.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@bluefield/otel/ocb-generated/go.mod`:
- Line 389: Update the OCB dependency inputs or override to require
google.golang.org/grpc v1.82.1 or later, then regenerate the generated go.mod
and go.sum files so the dependency graph no longer contains v1.81.1; do not
hand-edit the generated output.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 66e45d49-6151-4c0a-b594-9b565bc3de45

📥 Commits

Reviewing files that changed from the base of the PR and between d205ef9 and 4b708bd.

⛔ Files ignored due to path filters (2)
  • bluefield/otel/ocb-generated/go.sum is excluded by !**/*.sum
  • rest-api/go.sum is excluded by !**/*.sum
📒 Files selected for processing (14)
  • .github/workflows/ci.yaml
  • .github/workflows/rest-lint-and-test.yml
  • .gitignore
  • bluefield/Makefile.toml
  • bluefield/containers/otelcol-contrib/Dockerfile
  • bluefield/otel/README.md
  • bluefield/otel/ocb-generated/go.mod
  • bluefield/otel/otelcol_builder_config_yaml.txt
  • bluefield/otel/update-ocb-modules.sh
  • rest-api/.gitignore
  • rest-api/Makefile
  • rest-api/go.mod
  • rest-api/tools/README.md
  • rest-api/tools/tools.go
💤 Files with no reviewable changes (1)
  • .github/workflows/ci.yaml
🚧 Files skipped from review as they are similar to previous changes (9)
  • bluefield/otel/README.md
  • bluefield/otel/otelcol_builder_config_yaml.txt
  • .gitignore
  • bluefield/otel/update-ocb-modules.sh
  • .github/workflows/rest-lint-and-test.yml
  • bluefield/Makefile.toml
  • bluefield/containers/otelcol-contrib/Dockerfile
  • rest-api/Makefile
  • rest-api/go.mod

The `docker-build-otelcol` Dockerfile and the `build-otelcol`
cargo-make task previously ran `ocb` without `--skip-get-modules`,
causing `go mod tidy` to execute inside every build. Because `go mod
tidy` queries and possibly updates go modules, two builds from the
same commit could resolve different indirect dependency versions and
produce different binaries.

Both build paths now run OCB with `--skip-compilation
--skip-get-modules` to generate Go source only, then overwrite the
incomplete OCB-generated `go.mod`/`go.sum` with the committed files
from `bluefield/otel/ocb-generated/` before calling `go build
-mod=mod`. The `include-otelcol` task condition is updated to
invalidate the cached binary when the committed module files change.

The trade-off: developers must run `bluefield/otel/update-ocb-modules.sh`
and commit the outputs whenever the OCB config or component versions
change. This is a manual step, but it makes the build reproducible
and prevents silent dependency drift.

`rest-api/tools/tools.go` (new) pins `buf`, `protoc-gen-go`,
`protoc-gen-go-grpc`, `golangci-lint`, and `revive` in `go.mod`.
`rest-api/Makefile` is updated to invoke these tools via `$(BUF)`, `$(GOLANGCI_LINT)`,
and `$(REVIVE)` using `go run`.

`protoc-gen-go` and `protoc-gen-go-grpc` must be real executables on
PATH because `buf generate` spawns them as subprocesses — `go run`
cannot be used directly. A `.tools-stamp` target builds them from the
pinned `go.mod` versions into `.tools/` and injects that directory
into PATH for `core-protogen` and `flow-protogen`. The manual
version-check guards previously in `core-protogen` are removed; the
version is now enforced by `go.mod` rather than runtime assertions.

The `Install buf and protoc plugins` steps in `ci.yaml` and
`rest-lint-and-test.yml` are removed — `make core-proto` now handles
tool installation via `.tools-stamp`. The `go install
github.com/mgechev/revive@v1.15.0` call in the `style` CI job is
replaced with `go run github.com/mgechev/revive` so the version from
`go.mod` is used.
go install github.com/mgechev/revive@v1.15.0
pkgs=$(go list ./... | grep -v $(go env GOMODCACHE))
go list $pkgs | grep -v $(go env GOMODCACHE) | xargs -L1 revive -config .revive.toml -set_exit_status
go list $pkgs | grep -v $(go env GOMODCACHE) | xargs -L1 go run github.com/mgechev/revive -config .revive.toml -set_exit_status

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runs a random github thing?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’s just running the version of ‘revive’ that is pinned in go.mod instead of downloading it on every CI run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants