Skip to content

Add --profile-json to run/run-c/run-cu: machine-readable run output#764

Open
SiliconState wants to merge 1 commit into
HigherOrderCO:mainfrom
SiliconState:pr/profile-json
Open

Add --profile-json to run/run-c/run-cu: machine-readable run output#764
SiliconState wants to merge 1 commit into
HigherOrderCO:mainfrom
SiliconState:pr/profile-json

Conversation

@SiliconState

@SiliconState SiliconState commented Jun 10, 2026

Copy link
Copy Markdown

Problem

Every tool that measures Bend today (CI benchmarks, profilers, editors) re-parses the human output:

Result: 16744448
- ITRS: 5767113
- TIME: 0.06s
- MIPS: 98.01

That format is undocumented and implicitly frozen: any change to the Result: prefix, the - KEY: value stat lines, or the 0.06s unit breaks every downstream parser silently. Each consumer writes (and re-debugs) the same fragile scraper. The scraping contract already exists — it is just informal; this PR makes it explicit and cheap to keep, instead of frozen-by-fear.

Design

A --profile-json flag on run, run-c, and run-cu that replaces the Result:/stats lines with one compact JSON object on stdout:

$ bend run-c prog.bend --profile-json
{"backend": "run-c", "result": "16744448", "stats": {"itrs": 5767113, "time_s": 0.06, "mips": 98.01}}
  • backend: which runner produced the measurement, in Bend's own naming (run-rs, run-c, run-cu; HVM internally calls run-rs just run).
  • result: the readback term, exactly the text that followed Result: .
  • stats: the HVM - KEY: value lines parsed into typed fields. Keys are lowercased; numeric values become JSON numbers; TIME's trailing s unit is stripped and the field named time_s. Unknown keys pass through, so new runtime stats appear automatically without a Bend release.
  • Program output (from IO) still streams to stdout during the run, so tools take the last line of stdout. Diagnostics stay on stderr.
  • No new dependencies: emission is a small formatter with a JSON string escaper. Values are emitted as JSON numbers only when they match the strict JSON number grammar — HVM prints - MIPS: inf when the reported duration is 0.00, and Rust's float parser accepts inf/NaN/+5/.5, none of which are valid JSON, so those are quoted instead.

Patch (src/main.rs only)

  1. CliRunOpts gains --profile-json.
  2. The run arm prints profile_json_line(backend, &term.to_string(), &stats) instead of the Result:/-s lines when the flag is set (-s/-p are subsumed: stats are always included, and the result is the plain term text).
  3. profile_json_line + is_json_number + json_escape helpers with unit tests (typed stats parsing, strict-number quoting, escaping, non-stat lines skipped).

Backward compatible: without the flag, output is byte-for-byte unchanged.

Testing

  • Unit tests for stats parsing (itrs/time_s/mips/unknown-key passthrough), JSON number strictness, string escaping, non-stat-line skipping.
  • End-to-end: bend run-c parallel_sum.bend --profile-json emits one valid JSON line (validated with python3 -m json.tool) whose itrs matches the -s run; run-rs works identically; full test suite passes.

Composition

With a runtime thread count (#763 + HigherOrderCO/HVM2#441), a complete machine-readable scaling experiment is one shell line:

for n in 1 2 4 8; do bend run-c prog.bend --threads $n --profile-json; done

Replaces the Result:/stats lines with one compact JSON object on
stdout: {"backend", "result", "stats":{...}} with the HVM '- KEY:
value' lines parsed into typed fields (keys lowercased, TIME's unit
stripped into time_s, unknown keys passed through so new runtime stats
appear without a Bend release). Backend uses Bend-side naming
(run-rs/run-c/run-cu). Values are emitted as JSON numbers only when
they match the strict JSON number grammar (HVM prints '- MIPS: inf'
when the reported duration is 0.00; inf/NaN/+5/.5 are quoted instead).
No new dependencies; without the flag, output is byte-for-byte
unchanged. Unit tests included; full suite passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant