formatter: add REQ_ALL_HEADERS/RESP_ALL_HEADERS extension formatter

[willianpaixao]

Commit Message:
formatter: add REQ_ALL_HEADERS / RESP_ALL_HEADERS extension formatter

Additional Description:

Adds a new extension formatter (envoy.formatter.all_headers) exposing two access log command operators:

• %REQ_ALL_HEADERS% — serializes all HTTP request headers as a JSON object
• %RESP_ALL_HEADERS% — same for response headers

This eliminates the need to enumerate each header individually via %REQ(name)% / %RESP(name)%. When used with json_format, the output is a nested JSON object (via Protobuf::Struct, same pattern as DYNAMIC_METADATA) rather than a flat string, so downstream consumers (SIEM, Elasticsearch, etc.) can query individual headers without extra deserialization.

Motivation (see #45244):

• Security observability — unexpected or malicious headers (e.g. the Log4Shell spring.cloud.function.routing-expression header) are silently dropped today because they were never enumerated in config. This lets threat-detection pipelines see headers that were already on the wire.
• Operational overhead — adding a single header to logs no longer requires a config change + rolling redeploy.

Shipped as an opt-in extension formatter (following the req_without_query precedent) to directly address the "configurable obfuscation" concern raised by @phlax on the original request (#40584).

Safety knobs:

• exclude_headers — case-insensitive list of headers to omit. Docs recommend excluding authorization, cookie, set-cookie.
• max_value_bytes — truncate individual header values to bound log line size (0 = unlimited).

Behavior details:

• Repeated headers are comma-joined per RFC 7230 (the set-cookie caveat is documented).
• Pseudo-headers (:method, :path, :authority, :status) are included by default since they are naturally present in the HeaderMap.

Performance note: HeaderMap::iterate() is O(n headers) with string copies into a protobuf struct, which is non-trivial on high-throughput paths. The extension-formatter design makes it fully opt-in; docs recommend pairing it with an access log filter (e.g. sampling, or only on non-- %RESPONSE_FLAGS%) and using max_value_bytes to bound size.

Risk Level: Low — new opt-in extension; no change to existing behavior when the formatter is not configured.

Testing: Unit tests added in test/extensions/formatter/all_headers/all_headers_test.cc (covers request and response serialization, exclude_headers case-insensitivity, max_value_bytes truncation, comma-joined repeated headers, and pseudo-header inclusion).

Docs Changes: Added formatter documentation to docs/root/configuration/observability/access_log/advanced/substitution_formatter.rst including the recommendation to exclude sensitive headers.

Release Notes: Added changelogs/current/.../formatter__added-req-all-headers-resp-all-headers.rst.

Platform Specific Features: N/A

Fixes: #45244 (also partially addresses #13454 — multi-value header support in access logs)

[repokitteh-read-only]

Hi @willianpaixao, welcome and thank you for your contribution.

We will try to review your Pull Request as quickly as possible.

In the meantime, please take a look at the contribution guidelines if you have not done so already.

🐱

Caused by: #45526 was opened by willianpaixao.

see: more, trace.

[repokitteh-read-only]

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @mattklein123
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).

🐱

Caused by: #45526 was opened by willianpaixao.

see: more, trace.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Introduces a new envoy.formatter.all_headers substitution-formatter extension that serializes all HTTP request/response headers into JSON for access logging, with configuration options to truncate values and exclude sensitive headers.

Changes:

• Adds %REQ_ALL_HEADERS% / %RESP_ALL_HEADERS% formatter implementation + factory wiring and build integration.
• Adds API proto + extension metadata/build config registration.
• Adds comprehensive unit tests, docs, and changelog entry for the new formatters.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
source/extensions/formatter/all_headers/all_headers.h	Declares formatter + command parser for serializing headers to JSON/google.protobuf.Value.
source/extensions/formatter/all_headers/all_headers.cc	Implements header extraction, exclusion, truncation, and multi-value handling.
source/extensions/formatter/all_headers/config.h	Declares command parser factory for the extension.
source/extensions/formatter/all_headers/config.cc	Implements factory creation logic from proto config and registers the factory.
source/extensions/formatter/all_headers/BUILD	Adds Bazel targets for the formatter library and extension config.
api/envoy/extensions/formatter/all_headers/v3/all_headers.proto	Defines configuration proto for max_value_bytes and exclude_headers.
api/envoy/extensions/formatter/all_headers/v3/BUILD	Adds Bazel API package for the new proto.
api/BUILD	Registers the new API package in the overall API build.
api/versioning/BUILD	Registers the new API package for versioning checks.
source/extensions/extensions_metadata.yaml	Adds metadata entry for envoy.formatter.all_headers.
source/extensions/extensions_build_config.bzl	Registers the extension for build inclusion.
docs/root/configuration/advanced/substitution_formatter.rst	Documents the new substitution commands and extension configuration.
changelogs/current/new_features/formatter__added-req-all-headers-resp-all-headers.rst	Adds release note entry for the formatter.
test/extensions/formatter/all_headers/all_headers_test.cc	Adds unit tests for string/JSON formatting, exclusion, truncation, and parser errors.
test/extensions/formatter/all_headers/BUILD	Adds Bazel test target for the new tests.