Add --profile-json to run/run-c/run-cu: machine-readable run output#764
Open
SiliconState wants to merge 1 commit into
Open
Add --profile-json to run/run-c/run-cu: machine-readable run output#764SiliconState wants to merge 1 commit into
SiliconState wants to merge 1 commit into
Conversation
Replaces the Result:/stats lines with one compact JSON object on
stdout: {"backend", "result", "stats":{...}} with the HVM '- KEY:
value' lines parsed into typed fields (keys lowercased, TIME's unit
stripped into time_s, unknown keys passed through so new runtime stats
appear without a Bend release). Backend uses Bend-side naming
(run-rs/run-c/run-cu). Values are emitted as JSON numbers only when
they match the strict JSON number grammar (HVM prints '- MIPS: inf'
when the reported duration is 0.00; inf/NaN/+5/.5 are quoted instead).
No new dependencies; without the flag, output is byte-for-byte
unchanged. Unit tests included; full suite passes.
f6b940b to
00ee605
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every tool that measures Bend today (CI benchmarks, profilers, editors) re-parses the human output:
That format is undocumented and implicitly frozen: any change to the
Result:prefix, the- KEY: valuestat lines, or the0.06sunit breaks every downstream parser silently. Each consumer writes (and re-debugs) the same fragile scraper. The scraping contract already exists — it is just informal; this PR makes it explicit and cheap to keep, instead of frozen-by-fear.Design
A
--profile-jsonflag onrun,run-c, andrun-cuthat replaces theResult:/stats lines with one compact JSON object on stdout:$ bend run-c prog.bend --profile-json {"backend": "run-c", "result": "16744448", "stats": {"itrs": 5767113, "time_s": 0.06, "mips": 98.01}}backend: which runner produced the measurement, in Bend's own naming (run-rs,run-c,run-cu; HVM internally calls run-rs justrun).result: the readback term, exactly the text that followedResult:.stats: the HVM- KEY: valuelines parsed into typed fields. Keys are lowercased; numeric values become JSON numbers;TIME's trailingsunit is stripped and the field namedtime_s. Unknown keys pass through, so new runtime stats appear automatically without a Bend release.- MIPS: infwhen the reported duration is 0.00, and Rust's float parser acceptsinf/NaN/+5/.5, none of which are valid JSON, so those are quoted instead.Patch (src/main.rs only)
CliRunOptsgains--profile-json.profile_json_line(backend, &term.to_string(), &stats)instead of theResult:/-slines when the flag is set (-s/-pare subsumed: stats are always included, and the result is the plain term text).profile_json_line+is_json_number+json_escapehelpers with unit tests (typed stats parsing, strict-number quoting, escaping, non-stat lines skipped).Backward compatible: without the flag, output is byte-for-byte unchanged.
Testing
itrs/time_s/mips/unknown-key passthrough), JSON number strictness, string escaping, non-stat-line skipping.bend run-c parallel_sum.bend --profile-jsonemits one valid JSON line (validated withpython3 -m json.tool) whoseitrsmatches the-srun;run-rsworks identically; full test suite passes.Composition
With a runtime thread count (#763 + HigherOrderCO/HVM2#441), a complete machine-readable scaling experiment is one shell line: