diff --git a/hypaware-core/plugins-workspace/claude/hypaware.plugin.json b/hypaware-core/plugins-workspace/claude/hypaware.plugin.json index 3d3dd1d3..a9d26403 100644 --- a/hypaware-core/plugins-workspace/claude/hypaware.plugin.json +++ b/hypaware-core/plugins-workspace/claude/hypaware.plugin.json @@ -37,6 +37,9 @@ { "name": "hypaware-ignore", "clients": ["claude"] }, { "name": "hypaware-unignore", "clients": ["claude"] }, { "name": "hypaware-privacy", "clients": ["claude"] }, + { "name": "hypaware-graph", "clients": ["claude"] }, + { "name": "hypaware-sensitive-scan", "clients": ["claude"] }, + { "name": "hypaware-ai-usage-report", "clients": ["claude"] }, { "name": "hypaware-ai-adoption-report", "clients": ["claude"] }, { "name": "hypaware-ai-improvement-report", "clients": ["claude"] }, { "name": "hypaware-ai-security-report", "clients": ["claude"] }, diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-adoption-report/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-adoption-report/SKILL.md index bac61fab..578440b8 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-adoption-report/SKILL.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-adoption-report/SKILL.md @@ -1,6 +1,6 @@ --- name: hypaware-ai-adoption-report -description: AI Adoption Profile for a HypAware server: descriptive "who's using the fleet and how": per-gateway utilization (volume + focus: top models, tools, repos, themes) and parallelism/fan-out (multi-agent adoption, concurrency, main-vs-subagent split, payoff). +description: "SUPERSEDED by hypaware-ai-usage-report (merged with the spend review, 2026-07-13) — invoke that skill instead unless the user explicitly asks for a standalone adoption profile. Original scope: AI Adoption Profile for a HypAware server: descriptive \"who's using the fleet and how\": per-gateway utilization (volume + focus: top models, tools, repos, themes) and parallelism/fan-out (multi-agent adoption, concurrency, main-vs-subagent split, payoff)." --- # AI Adoption Profile @@ -23,13 +23,31 @@ list it once). Present the options, ask which one (or more) to profile, then proceed against the chosen source. **Descriptive only.** Route every *action* out: "fan out more/less" and tooling go to -**hypaware-ai-improvement-report**, token waste to **hypaware-ai-spend-report**. Query mechanics -live in the **hypaware-query** skill; reuse hypaware-ai-spend-report's token spine for -any token figure. For descriptive who-used-what rollups (distinct sessions per -repo/model/tool/file), prefer the **hypaware-graph** skill, which reads them from the projected -graph instead of scanning messages; keep token figures on messages. +**hypaware-ai-improvement-report**, token waste to **hypaware-ai-spend-report**. Reuse +hypaware-ai-spend-report's token spine for any token figure. Query mechanics are step 0 below — +a mandatory read, not a reference. ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before you run any `hyp query sql`, read the **hypaware-query** skill (invoke it or + Read its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes and recall of past runs do NOT substitute: stale notes have cost real + runs failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` + scans that 504'd and then OOM'd the prod server). Then route every question by shape, per + hypaware-query's "when the graph answers it cheaper" boundary: + - **Graph first (`node`/`edge` — tiny, join-safe) for every entity/connection question:** + which sessions used a repo/model/tool/file, skill and program rollups (these two exist ONLY + in the graph — SQL reconstructions disagree with the projection), client mix, co-occurrence, + and gateway→person attribution — take `min/max(session_id)` per gateway from messages (an + ID-only aggregate), then look those session_ids up in graph Session nodes (`props.cwd`, + `props.client_name`). **Never GROUP BY / DISTINCT / row-fetch wide content columns (`cwd`, + `content_text`) on the messages table at scale** — that is the query shape that kills servers. + - **Messages (`ai_gateway_messages`) only for per-message measures:** token sums, distinct + part/session counts, timestamps and ordering, `is_sidechain`/`agent_id`, content sampling. + Slice long windows into server-sized date ranges; capture stderr and check it even on + success (truncation and server-cap notices land there). + If you skipped this step and a query fails, the fix is to come back here — not to iterate on + the failing SQL. 1. **Scope + coverage.** Distinct `gateway_id` (the unit; `user_id` is ~always null, so don't measure reach by it); window; coverage of `gateway_id`, token usage, and subagent provenance (`agent_id` / `is_sidechain` / `parent_thread_id`, transcript-enriched, may not survive diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-improvement-report/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-improvement-report/SKILL.md index 09b1512f..c9aa8ba1 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-improvement-report/SKILL.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-improvement-report/SKILL.md @@ -16,9 +16,7 @@ and let the user choose which to query: **local logs** (this machine's own recor `query_sql` / `graph_neighbors` tool in your toolset); the same server can appear both ways, list it once). Present the options, ask which one (or more) to run the review against, then proceed against the chosen source. -Query mechanics live in the **hypaware-query** skill; read it first. For descriptive -who-used-what rollups (distinct sessions per repo/model/tool/file), prefer the **hypaware-graph** -skill: it reads relationships from the projected graph instead of scanning messages. +Query mechanics are step 0 of the procedure — a mandatory read, not a reference. Focus on these signals: @@ -31,6 +29,23 @@ Focus on these signals: subagent, a context-hygiene rule, a skill that avoids the redo). ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). Route by shape, per hypaware-query's "when the graph + answers it cheaper" boundary: entity/connection questions go to the graph's tiny `node`/`edge` + tables — sessions per repo/model/tool/file, **skill and program rollups (graph-only facets — + SQL reconstructions measurably disagree with the projection)**, client mix, shared-file + clustering via `touched` edges, and gateway→person attribution (`min/max(session_id)` per + gateway from messages — an ID-only aggregate — then look those session_ids up in graph + Session nodes' `props.cwd`). The messages table is for per-message measures only — token + sums (the spend spine), `is_error`/retry loops, stop-reasons, ordering, prompt sampling — + date-sliced to server-sized ranges, and **never GROUP BY / DISTINCT / row-fetch wide content + columns (`cwd`, `content_text`) at scale** (that query shape kills servers). Capture stderr + and check it even on success — truncation and server-cap notices land there. If a query + fails, come back to this step; don't iterate on the failing SQL. 1. **Footprint + basis.** Window; distinct contributors (`gateway_id`), repos, sessions; claude/codex mix. State N and breadth; flag if single-contributor. 2. **Scan the signals for possible improvements.** Work the three signals above; each turns @@ -67,26 +82,56 @@ Focus on these signals: ## Output - SAVE A SHORT MAIN FILE + ONE FILE PER SECTION A **progressively-disclosed one-pager** is the deliverable: a reader can stop after the thesis -sentence, glance the key-numbers table, skim the findings, or follow any link into a detail -**section**, and each section is its own markdown file. Build the one-pager top-to-bottom so each -layer is shorter and more glanceable than the one it summarizes. +sentence, skim the numbered change list, or follow any link into a detail **section**, and each +section is its own markdown file. Build the one-pager top-to-bottom so each layer is shorter and +more glanceable than the one it summarizes. + +**Report language (applies to every file).** The reader may be a supervisor with no HypAware +knowledge. Plain literal language only: no pipeline vocabulary (ingest, sidechain, part, +projection), no metaphors or report-coined shorthand ("marathon sessions" was flagged as vague - +write "sessions kept open across days re-read their entire history each morning"); coined names +(a harness, an internal tool) get a one-line gloss at first mention on each page; when a skill or +tool is named as a fix, add one clause saying what it literally does ("`/handoff` writes a short +end-of-day note of where the work stands so the next day starts a fresh session"), never just its +name; absolute dates, never "last week". Findings equip, never direct: state who/size/driver/what +a change returns, never "ask X" / "talk to Y". In section files, any list item carrying several +facts is a bold topic line + 2-3 short sub-bullets, not a multi-sentence prose bullet. - **Main one-pager:** `hypaware-reports/-improvement-review.md` (create the dir if - needed). Dated so reviews accumulate. Lay it out in exactly these blocks, separated by `---` rules: - 1. **Title + scope** - `# AI Improvement Review`, then a `## · ` subtitle. - 2. **Thesis** - ONE **bold** sentence: "Create/edit these N things and what it will improve." - 3. **`### Key numbers`** - a small metric→readout table, ~4-6 rows, each one glanceable fact with - no link (# improvements, new-vs-edit split, the single biggest token prize, the basis: - gateways / sessions / repos). This is the ONLY table on the one-pager. - 4. **`## What this shows`** - 3-5 numbered findings = the top improvements, highest-leverage - first; each is a `### N. ` naming **what + its form** (skill / subagent / AGENTS.md - edit), followed by 2-3 plain-language sentences with the evidence and the **bold** token prize, - then a single `[
→](/.md)` link on its own line. Show AGENTS.md edits as - real diffs/code blocks in their section file, not prose. - 5. **`## Caveat`** - token prizes are floors and capture is partial, in plain language + a - `[caveats →]` link. - 6. **`## Report details`** - a one-line footer linking **every** section file written this run - (not just the cited ones), so nothing is orphaned - e.g. + needed). Dated so reviews accumulate. It is a **terse bullet brief** (user feedback 2026-07-14: + dense multi-clause sentences and prose paragraphs were both rejected as hard to read). Its only + headings are **Proposed changes / Data limitations / Supporting analysis** - standard + business-report vocabulary, never coinages like "What this shows" / "Key numbers" / "Report + details", and no metrics or findings section (the evidence rides inside the change list). Blocks: + 1. **Title + scope** - an eyebrow line ` · `, then `# AI Improvement Review`. + 2. **Thesis** - one SHORT bold sentence (the create/edit-these-N-things claim). If it needs + supporting clauses, put them in 2-3 plain sub-bullets under it, never packed into the sentence. + 3. **`## Proposed changes`** - the changes themselves lead, NOT a metrics block, and there is + NO separate findings/evidence section (user feedback 2026-07-14: a Key findings section + alongside the change list is redundant - the strongest evidence rides with each change). A + **numbered list**, one item per improvement, highest-leverage first, each exactly: + - the **what**: a short bold imperative naming ONE action (mechanics in parens after the + bold), nothing else on the line. Never join two actions with ";" or "+" in the bold line + (user feedback: "sounds like two suggestions, unclear") - when a change pairs a skill + move with a companion AGENTS.md rule, the bold names the primary action and the companion + rides in a sub-bullet; + - sub-bullet 1, the **why**: one short sentence with the token prize or headline number; + - sub-bullet 2, the **evidence**: one short line with the 1-2 strongest supporting numbers, + ending with the section link. + Never pack what+why+prize into the bold line (rejected as too long). Close with one line + linking the ready-to-apply artifacts (which live in section files, not here). No tables on + the one-pager. + **Every proposed change ships its artifact in its section file** (user feedback 2026-07-14 + - not only AGENTS.md diffs): AGENTS.md/CLAUDE.md edit → a real diff; new skill or subagent + → the full proposed file (frontmatter + body) in a code block, ready to save; move of an + existing artifact → concrete source → destination paths, flagging any machine-specific + content to review (if the source file lives on another machine, say so - name the move, + don't fake the file); tool/config change → the exact proposed text. + 4. **`## Data limitations`** - ~3 bullets: the basis (gateways / sessions / window), token + prizes are floors, capture is partial, plus any window-specific data-quality incident + a + `[caveats]` link. + 5. **`## Supporting analysis`** - a one-line footer linking **every** section file written this + run (not just the cited ones), so nothing is orphaned - e.g. `[basis](/basis.md) · [skill candidates](/skill-candidates.md) · [subagent candidates](/subagent-candidates.md) · [AGENTS.md edits](/agents-md-edits.md) · [efficiency → setup](/efficiency-setup.md) · [caveats](/caveats.md)`. - **Section files are analysis, not inventory.** Each detail section is its own `/.md`, held to the same standard as the one-pager: it argues one claim, opens with that takeaway, and ties every number to what it means for the reader. A section file that is only a stat table has failed - fold it back into the one-pager rather than shipping it as a page. Cut table narration ("how to read the table") and standing bookkeeping prose; compress source/window/method to a few lines. - **No scope apologies (in any file).** Scope rules (what routes to which report) are authoring guidance, never report copy. Don't write "descriptive only", "no recommendations here", or routing disclaimers in the one-pager or any section file; state findings plainly, and where a sibling report owns the action a plain cross-link is enough. diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-security-report/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-security-report/SKILL.md index 79a85d38..b6d8f4e1 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-security-report/SKILL.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-security-report/SKILL.md @@ -20,7 +20,8 @@ ask which one (or more) to audit, then proceed against the chosen source. **REDACT everything.** Never echo a secret, token, key, or credential value in the report, even one you found in `tool_args`; a raw secret appearing in args is itself a capture finding -(flag it, don't reproduce it). Query mechanics live in the **hypaware-query** skill; read it first. +(flag it, don't reproduce it). Query mechanics are step 0 of the procedure — a mandatory read, +not a reference. ## Risk classes (match over the FULL command text, not just the first token) | Class | Match signals | Why it matters | @@ -36,6 +37,22 @@ Risk is **amplified by autonomy**: weight anything that ran under `bypassPermiss than the same command in a gated session. ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). Route by shape, per hypaware-query's "when the graph + answers it cheaper" boundary: entity/connection questions go to the graph's tiny `node`/`edge` + tables — which sessions ran a tool/program, per-repo/client inventories, and gateway→person + attribution (`min/max(session_id)` per gateway from messages — an ID-only aggregate — then + look those session_ids up in graph Session nodes' `props.cwd`). This audit's core scans DO + read wide columns (`tool_args`, `content_text`) — that's their job — but keep them + aggregate-shaped regex matches (`regexp_extract` over `CAST(tool_args AS VARCHAR)`) over + **date-sliced, server-sized ranges**, never GROUP BY / DISTINCT / bulk row-fetches of raw + content columns. Capture stderr and check it even on success — truncation and server-cap + notices land there, and a clipped scan reads as "no findings". If a query fails, come back to + this step; don't iterate on the failing SQL. 1. **Coverage + autonomy baseline.** Window; coverage of `tool_args` on Bash calls and of `permission_mode` (bound the read if sparse); distinct `gateway_id` (the unit; `user_id` is ~always null). Total tool calls, % Bash, % conversations in `bypassPermissions`. State N; if @@ -51,26 +68,39 @@ than the same command in a gated session. ## Output - SAVE A SHORT MAIN FILE + ONE FILE PER SECTION A **progressively-disclosed one-pager** is the deliverable: a reader can stop after the thesis -sentence, glance the key-numbers table, skim the findings, or follow any link into a detail -**section**, and each section is its own markdown file. Build the one-pager top-to-bottom so each -layer is shorter and more glanceable than the one it summarizes. **All examples redacted in every file.** +sentence, skim the findings bullets, or follow any link into a detail **section**, and each +section is its own markdown file. Build the one-pager top-to-bottom so each layer is shorter and +more glanceable than the one it summarizes. **All examples redacted in every file.** + +**Report language (applies to every file; user feedback 2026-07-14 on the sibling reports).** The +reader may be a supervisor with no HypAware knowledge. Plain literal language: no pipeline +vocabulary, metaphors, or report-coined shorthand (write the literal behavior); coined names get +a one-line gloss at first mention per page; when a skill/tool/guardrail is named as a fix, add +one clause saying what it literally does; absolute dates only. Findings equip, never direct - +never "ask X" / "tell Y". Any bullet carrying several facts is a bold topic line + 2-3 short +sub-bullets; no multi-sentence prose bullets, no dense multi-clause lines. - **Main one-pager:** `hypaware-reports/-security-review.md` (create the dir if - needed). Dated so reports accumulate. Lay it out in exactly these blocks, separated by `---` rules: - 1. **Title + scope** - `# Security & Risk Review`, then a `## · ` subtitle. - 2. **Thesis** - ONE **bold** sentence: the posture - how autonomous the fleet is - (% `bypassPermissions`) and whether anything needs urgent attention. - 3. **`### Key numbers`** - a small metric→readout table, ~4-6 rows, each one glanceable fact with - no link and all redacted (% `bypassPermissions`, count of HIGH findings, the top risk class, - total tool calls + % Bash, coverage of `tool_args` / `permission_mode`). This is the ONLY - table on the one-pager. - 4. **`## What this shows`** - 3-5 numbered findings, highest-severity first; each is a - `### N. ` followed by 2-3 plain-language sentences with **bold** severities (redacted), - then a single `[
→](/.md)` link on its own line (autonomy posture, the top - risk findings, the single most important guardrail). - 5. **`## Caveat`** - the standing capture caveat in plain language + a `[capture caveats →]` link. - 6. **`## Report details`** - a one-line footer linking **every** section file written this run - (not just the cited ones), so nothing is orphaned - e.g. + needed). Dated so reports accumulate. It is a **terse bullet brief** with standard + business-report headings - **Key metrics / Key findings / Data limitations / Supporting + analysis** - never coinages like "What this shows" / "Key numbers" / "Report details". Blocks: + 1. **Title + scope** - an eyebrow line ` · `, then `# Security & Risk Review`. + 2. **Thesis** - one SHORT bold sentence: the posture - how autonomous the fleet is + (% `bypassPermissions`) and whether anything needs urgent attention. Supporting clauses go + in 2-3 plain sub-bullets under it, never packed into the sentence. + 3. **`## Key metrics`** - 4-6 **bold topic** bullets, each with 1-2 short sub-bullets carrying + the redacted numbers (% `bypassPermissions`, count of HIGH findings, the top risk class, + total tool calls + % Bash, coverage of `tool_args` / `permission_mode`). No tables on the + one-pager. + 4. **`## Key findings`** - 3-5 bullets, highest-severity first (autonomy posture, the top risk + findings, the single most important guardrail); each is a **bold headline** with **bold** + severity, then 2-3 short sub-bullets with the redacted evidence, the last ending with the + section link. Proposed guardrails ship ready to apply in their section file: settings/policy + → the exact config block; AGENTS.md rule → a real diff; redaction → the exact pattern. + 5. **`## Data limitations`** - ~2 bullets: the standing capture caveat in plain language + a + `[capture caveats]` link. + 6. **`## Supporting analysis`** - a one-line footer linking **every** section file written this + run (not just the cited ones), so nothing is orphaned - e.g. `[autonomy baseline](/autonomy-baseline.md) · [command profile](/command-profile.md) · [risk findings](/risk-findings.md) · [guardrails](/guardrails.md) · [trends](/trends.md) · [capture caveats](/capture-caveats.md)`. - **Section files are analysis, not inventory.** Each detail section is its own `/.md`, held to the same standard as the one-pager: it argues one claim, opens with that takeaway, and ties every number to what it means for the reader. A section file that is only a stat table has failed - fold it back into the one-pager rather than shipping it as a page. Cut table narration ("how to read the table") and standing bookkeeping prose; compress source/window/method to a few lines. - **No scope apologies (in any file).** Scope rules (what routes to which report) are authoring guidance, never report copy. Don't write "descriptive only", "no recommendations here", or routing disclaimers in the one-pager or any section file (redaction still applies); state findings plainly, and where a sibling report owns the action a plain cross-link is enough. diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-spend-report/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-spend-report/SKILL.md index 33599948..883ab16e 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-spend-report/SKILL.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-spend-report/SKILL.md @@ -1,6 +1,6 @@ --- name: hypaware-ai-spend-report -description: AI Spend Review for a HypAware server: where tokens go (by user/repo/model/gateway), what work they go on (graph-clustered), and how to spend less (waste scorecard + cost levers). Token volume, never dollars. Saves a dated report under hypaware-reports/; first asks which HypAware source to query (local logs or a remote server) via the hypaware-query skill. +description: "SUPERSEDED by hypaware-ai-usage-report (merged with the adoption profile, 2026-07-13) — invoke that skill instead unless the user explicitly asks for a standalone spend review. Original scope: AI Spend Review for a HypAware server: where tokens go (by user/repo/model/gateway), what work they go on (graph-clustered), and how to spend less (waste scorecard + cost levers). Token volume, never dollars. Saves a dated report under hypaware-reports/; first asks which HypAware source to query (local logs or a remote server) via the hypaware-query skill." --- # AI Spend Review @@ -19,8 +19,7 @@ list it once). Present the options, ask which one (or more) to run the review against, then proceed against the chosen source. **Tokens, never dollars.** Capture is partial, so stop at token volume and behavioral proxies. -Query mechanics (`--remote`, date-pruning the 30s timeout, the SQL dialect, sampling) live in -the **hypaware-query** skill; read it first. +Query mechanics are step 0 of the procedure — a mandatory read, not a reference. ## Token math (get this right; every breakdown reconciles to it) Usage is in `attributes.usage` (NOT `raw_frame`): `input_tokens`, `output_tokens`, @@ -46,6 +45,22 @@ WHERE date BETWEEN '' AND '' ``` ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). Route by shape, per hypaware-query's "when the graph + answers it cheaper" boundary: entity/connection questions go to the graph's tiny `node`/`edge` + tables — sessions per repo/model/tool/file, skill and program rollups (graph-only facets), + client mix, work-type clustering by shared-file `touched` edges, and gateway→person + attribution (`min/max(session_id)` per gateway from messages — an ID-only aggregate — then + look those session_ids up in graph Session nodes' `props.cwd`). The **token spine stays on + `ai_gateway_messages`**: token sums, counts, ordering — date-sliced to server-sized ranges, + and **never GROUP BY / DISTINCT / row-fetch wide content columns (`cwd`, `content_text`) at + scale** (that query shape kills servers). Capture stderr and check it even on success — + truncation and server-cap notices land there. If a query fails, come back to this step; don't + iterate on the failing SQL. 1. **Coverage + footprint.** Window; usage coverage, `model`-column coverage (token-weighted), user/repo coverage; distinct gateways, sessions, claude/codex mix. `user_id` is ~always null → attribute by `gateway_id` (≈ one machine/user) and always show an explicit **unattributed** diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-usage-report/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-usage-report/SKILL.md new file mode 100644 index 00000000..06a938eb --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-ai-usage-report/SKILL.md @@ -0,0 +1,227 @@ +--- +name: hypaware-ai-usage-report +description: Team AI Usage Review for a HypAware server, written for whoever supervises the team (eng manager, lead, CTO — no HypAware knowledge assumed): who's using AI and how much, what work it goes on and whether it pays off, which way it's trending, where tokens are wasted, and the levers to improve — the former adoption profile and spend review merged onto one token spine. Token volume, never dollars. Saves a dated report under hypaware-reports/; first asks which HypAware source to query (local logs or a remote server) via the hypaware-query skill. Supersedes hypaware-ai-adoption-report and hypaware-ai-spend-report — run this instead of either. +--- + +# Team AI Usage Review + +One report, one window, one token spine, answering the four questions a supervisor +actually has: + +1. **Who is using AI, how much, and on what?** — per-person utilization, and allocation by + person / repo / model. +2. **What does the work look like, and is the sophistication paying off?** — each person's + focus, recurring work-types sized by tokens, multi-agent fan-out and whether it earns + its cost, code that actually landed (GitHub reach, where enriched). +3. **Which way is it trending?** — weekly volume/output, deltas vs the last review. +4. **Where is it wasted, and what can I do about it this week?** — waste scorecard + + ranked levers, each with an estimated weekly token saving. + +The old adoption-profile and spend-review reports each told half of this and repeated the +other half; this report replaces both. Don't run those skills alongside it. + +## Audience contract (this is the point of the merge — enforce it everywhere) + +The reader is **the person supervising the team** — an engineering manager, team lead, +CTO, or ops owner. Assume they have never seen a HypAware table, don't know the schema +vocabulary, and are reading to make decisions about their team, not to audit queries. +Concretely: + +- **People, not infrastructure.** A `gateway_id` is ≈ one person's machine: attribute it + to a named person (via session `cwd`) and say the person. "phil's laptop", never + "gateway a3f9". Keep an explicit **unattributed** row when attribution fails. +- **Every number carries a judgment and a "so what".** Not "cache-read ratio 99.5%" but + what that means for them (context reuse is healthy / this is where the bill comes from). + Terms the report can't avoid (cache-read, subagent, output tokens) get a one-line + plain-language gloss the first time they appear — once, then use them freely. +- **Plain literal language, everywhere.** Pipeline vocabulary (ingest, sidechain, + provenance, projector, org-scoped, part) never appears in report copy — translate to + what it means for the numbers ("the logs record which tokens belong to helper + agents"). Coined names (a harness like "gastown") get a one-line gloss at first + mention on EACH page that uses them. Headings state the literal fact, never a + metaphor ("Message counts for 07-09 → 07-14 don't reflect real activity", not "The + volume meter broke"). This bans report-coined shorthand too: "marathon sessions" was + flagged by the user as vague (2026-07-14) — write the literal behavior, "sessions + kept open across days re-read their entire history each morning" / "the 60 longest + sessions". When a skill or tool is named as a fix, add one clause on what it + literally does ("`/handoff` writes a short end-of-day note of where the work stands + so the next day starts a fresh session"), never just its name. Time references are + absolute dates ("07-09 → 07-14", "wk of 07-13"), never relatives like "final week" + or "this week" that decay after the run date. +- **Equip the reader; never direct them.** The report's job is to surface what a + supervisor needs to act — who is involved, the size of the effect, what would change + it, and the expected result — stated as findings and sized options, NOT as + instructions ("ask X about Y", "talk to kenny", "set an expectation", "sanction Z"). + The reader knows how to manage their team; telling them how reads as presumptuous and + buries the information. Write "the 60 longest sessions are 32 phil / 24 kenny; a 25% + context trim returns ~290M/wk", never "ask phil and kenny to trim their sessions". + Deep tooling work (skills to build, harness changes) still routes to + **hypaware-ai-improvement-report** as a plain cross-link — surface it, don't spec it + here. +- **Comparisons over absolutes.** A supervisor can't judge "69.5M output tokens" in a + vacuum; they can judge shares, trends vs the last review, and spread across the team. + Lead with those. +- **Tokens, never dollars.** Capture is partial, so stop at token volume and behavioral + proxies; say so once in the caveat, not in every section. + +IMPORTANT: Don't assume which logs to read: **ask first.** Start by listing the data +sources and let the user choose which to query: **local logs** (this machine's own +recordings, `hyp query sql …`, no `--remote`) and **each remote HypAware server** (every +target from `hyp remote list`, plus any hypaware MCP server already available to you as +MCP tools (a `query_sql` / `graph_neighbors` tool in your toolset); the same server can +appear both ways, list it once). Present the options, ask which one (or more) to review, +then proceed against the chosen source. + +## Token math (get this right; every breakdown reconciles to it) + +Usage is in `attributes.usage` (NOT `raw_frame`): `input_tokens`, `output_tokens`, +`cache_read_tokens`, `cache_write_tokens` (+ `reasoning_tokens` for Codex). Usage rides +exactly one row per response (the last assistant part; non-carrier parts are null), so a +plain `SUM` over assistant rows is correct with no dedup (the one-carrier rule, LLP 0035); +`input_tokens` is net of cache, so it never double-counts. Report the four types +separately (cache-read is usually the bulk; output the scarce slice). + +```sql +SELECT + sum(CAST(JSON_EXTRACT(attributes,'$.usage.input_tokens') AS BIGINT)) t_in, + sum(CAST(JSON_EXTRACT(attributes,'$.usage.output_tokens') AS BIGINT)) t_out, + sum(CAST(JSON_EXTRACT(attributes,'$.usage.cache_write_tokens') AS BIGINT)) t_cw, + sum(CAST(JSON_EXTRACT(attributes,'$.usage.cache_read_tokens') AS BIGINT)) t_cr +FROM ai_gateway_messages +WHERE date BETWEEN '' AND '' + AND role='assistant' AND JSON_EXTRACT(attributes,'$.usage') IS NOT NULL; +-- One carrier row per response (LLP 0035): a plain SUM is correct, no dedup. +-- Slice by adding gateway_id / model / repo_root / date to SELECT + GROUP BY. +-- Defensive equivalent: max(...) GROUP BY session_id, message_id -- session_id is the +-- uniform key; conversation_id is null for Claude and only separates Codex threads. +``` + +## Procedure + +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user + picks a source and before any `hyp query sql`, read the **hypaware-query** skill + (invoke it or Read its SKILL.md), and the **hypaware-graph** skill if `hyp query + status` lists `node`/`edge` datasets. Memory notes from past runs do NOT substitute — + stale notes have cost real runs failed queries and server crashes (a phantom "100-row + output cap"; message-table `cwd` scans that 504'd then OOM'd the prod server). Route + by shape, per hypaware-query's "when the graph answers it cheaper" boundary: + - **Graph first (`node`/`edge` — tiny, join-safe) for every entity/connection + question:** which sessions used a repo/model/tool/file, skill and program rollups + (graph-only facets — SQL reconstructions disagree with the projection), client mix, + work-type clustering by shared-file `touched` edges, co-occurrence, and + gateway→person attribution (`min/max(session_id)` per gateway from messages — an + ID-only aggregate — then look those session_ids up in graph Session nodes' + `props.cwd`, `props.client_name`). + - **Messages (`ai_gateway_messages`) only for per-message measures:** token sums, + distinct part/session counts, timestamps and ordering, `is_sidechain`/`agent_id`, + content sampling. Slice long windows into server-sized date ranges. **Never GROUP + BY / DISTINCT / row-fetch wide content columns (`cwd`, `content_text`) on the + messages table at scale** — that query shape kills servers. Capture stderr and + check it even on success (truncation and server-cap notices land there). + If a query fails, come back to this step; don't iterate on the failing SQL. + +1. **Scope + coverage.** Window; distinct `gateway_id` (the unit; `user_id` is ~always + null, so never measure reach by it) mapped to named people; usage coverage, + `model`-column coverage (token-weighted), user/repo coverage; claude/codex mix; + subagent provenance (`agent_id` / `is_sidechain` / `parent_thread_id`, + transcript-enriched, may not survive ingest). Decide cost-capable vs volume-only and + which parallelism dimensions are real vs proxied (the `Task`-call proxy). State N; if + it's effectively one person / dogfood, say so. If usage is thin, fall back to + behavioral proxies (turns, tool calls, length), labeled as estimates. + **ALWAYS verify GitHub enrichment before deciding it's out of scope: probe, never + assume.** Run the probe every run: `hyp query sql "SELECT node_type, projector, + count(*) AS n, max(first_seen) AS newest FROM node GROUP BY node_type, projector" + --remote ` (and check `edge` exists via `hyp query status`). If a `github.t0` + projector with `PullRequest` / `Review` nodes is present, **GitHub reach is IN SCOPE + and MUST be computed in step 3**: record the node counts and max `first_seen` per type + as the graph's as-of date, and treat every reach figure as a floor. If the probe finds + nothing, state **"checked - no GitHub enrichment present"** explicitly. Never write + "not assessed" for reach — that phrasing means the probe was skipped. + +2. **Who, how much, and which way it's moving.** One row per person: volume (messages, + sessions, active days, first/last seen), tokens + cache-read ratio + `cache_read/(cache_read+input)`. Slice the token spine by person / repo / model + (→ `(unknown)` bucket) with shares; weekly trend with WoW deltas vs the last review; + top-spend outlier sessions. This one spine feeds every later section — reconcile, + don't re-derive. + +3. **What the work is, and whether it pays off.** Per person: top models, tools (Bash + dominance + top commands), repos, client, and 2–4 recurring work themes (sampled, + redacted), distilled into a one-line **focus label** a supervisor can repeat. Cluster + sessions into recurring **work-types** (shared-file overlap for code work, tool-set + signature for no-file work; context graph if projected, else SQL), sized by tokens. + Parallelism as a payoff question: % of sessions that fan out to subagents (incl. the + zero bucket), breadth/depth, true concurrency vs serial, main-loop-vs-subagent token + split, fan-out vs tokens-to-resolution — say plainly whether the sophisticated + pattern is earning its cost and who on the team has the habit worth spreading. When + step 1 found `github.t0` enrichment, add each person's real *reach*: repos and PRs + their AI-assisted work landed in (`Session -at-> Commit <-references- PullRequest`) + and whether it drew review (`… PullRequest <-on- Review <-submitted- Actor`), dated to + the graph's freshness. This is the "did the tokens become shipped code" evidence a + supervisor most wants and the messages cannot show — not optional when the graph + supports it. + +4. **Where to save, and the information to act.** Score the waste: cache-read ratio + (the biggest lever, feature it), long-session concentration (sessions kept open + across days), retry loops, + abandoned costly sessions, model over-spec, context bloat. Turn the top few into + **ranked levers, each with an estimated weekly token saving and the facts a + supervisor needs to act on it** — who is involved, what behavior or default drives + it, and what a stated change would return. Per the audience contract: findings and + sized options, never instructions to the manager. Hand expensive recurring + work-types to **hypaware-ai-improvement-report** as packaging candidates (surface + them; don't build them here). + +## Output - SAVE A SHORT MAIN FILE + ONE FILE PER SECTION + +A **short bullet brief** is the main deliverable (~40 lines of content): a reader gets +the whole story from scannable bullets, and every detail lives in a linked section file. +Headings are standard business-report vocabulary — never AI-flavored coinages like "The +numbers", "What this shows", or "Where the leverage is". + +- **Main brief:** `hypaware-reports/-usage-review.md` (create the dir if + needed). Dated so reviews accumulate. Lay it out in exactly these blocks: + 1. **Title + scope** - an eyebrow line ` · `, then + `# Team AI Usage Review`. + 2. **Headline** - ONE short **bold** sentence a supervisor could repeat in a meeting: + the trend, the biggest concentration, the top leverage point. Facts, not + instructions. + 3. **`## Key metrics`** - grouped bullets, each a **bold topic line + 2-3 short + sub-bullets** (topics ≈ Volume / Who / Trend / The work / Fan-out / Health): + glanceable facts with bold numbers, no prose sentences. Each topic line ends with + ` · [
](/.md)` linking its detail section. + 4. **`## Key findings`** - 3-5 ranked findings as the same bold-topic + sub-bullets + shape: each names the finding, who is involved, the driver, and the estimated + weekly token effect where one exists, with the topic line linking its detail + section like Key metrics. This is data reporting, not consulting — sized facts, + never instructions to the manager (audience contract) and never pitch-flavored + headings ("Opportunities", "Recommendations"). + 5. **`## Data limitations`** - 2-3 bullets: the caveats that most change how to read + the report (tokens-never-dollars + partial capture; whether subagent identity + survived ingest; any capture anomalies this window). + 6. **`## Supporting analysis`** - a one-line footer linking **every** section file + written this run (not just the cited ones), so nothing is orphaned - e.g. + `[scope & coverage](/scope-coverage.md) · [team usage](/team-usage.md) · [trends](/trends.md) · [focus & reach](/focus-and-reach.md) · [work-types](/work-types.md) · [parallelism payoff](/parallelism-payoff.md) · [waste scorecard](/waste-scorecard.md) · [levers](/levers.md) · [caveats](/caveats.md)`. +- **Chart the who-breakdowns.** Keep the one-row-per-person tables as the record, and + pair each with a breakdown chart following the HTML renderer's authoring contract + (`hypaware-report-to-html/authoring.md`; component snippets in `components.md` next to + it): share of messages and tokens by person on the team-usage page, main-vs-subagent + token split on the parallelism page, waste levers as a ranked bar set. Where a real + team grouping exists (a user-supplied mapping, or cwd naming), add a by-team rollup; + never invent teams the data doesn't show. +- **Section files are analysis, not inventory.** Each detail section is its own + `/.md`, held to the same standard as the main brief: it argues one claim, + opens with a SHORT bold thrust line (a few clauses, not a paragraph — optionally + followed by 2-4 bullets), and ties every number to what it means for the reader. + Body lists use the same bold-topic + short-sub-bullets shape as the main brief; + multi-sentence prose bullets are hard to scan and not allowed. A + section file that is only a stat table has failed - fold it back into the main brief + rather than shipping it as a page. Cut table narration and standing bookkeeping prose; + compress source/window/method to a few lines. +- **No scope apologies (in any file).** Scope rules (what routes to which report) are + authoring guidance, never report copy. Don't write "descriptive only" or routing + disclaimers; state findings plainly, and where a sibling report owns the action a plain + cross-link is enough. +- **Capture-health note:** if subagent provenance doesn't reach the server, the standing + #1 caveat is "subagent identity must survive ingest"; run fan-out adoption off the + sub-agent-invocation proxy (the tool calls that spawn subagents) and flag it. diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-graph/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-graph/SKILL.md new file mode 100644 index 00000000..e15da7b1 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-graph/SKILL.md @@ -0,0 +1,170 @@ +--- +name: hypaware-graph +description: Explore the HypAware context graph: the activity graph projected from ai_gateway_messages (Sessions, Apps, Models, Tools, Files, Skills, Programs, and git Repos/Commits, and how they connect), plus optional server-side GitHub enrichment (Repos, PullRequests, Commits, Reviewers) that bridges AI sessions to code review. Use when the user asks what connects to a file/session/tool/model, which sessions ran a skill or invoked a program, wants co-occurrence or N-hop traversal, wants to join AI sessions to GitHub repos/PRs/reviewers, or wants to build/refresh the graph. Covers `hyp graph project` and `hyp graph neighbors`. +--- + +# HypAware Graph + +`hyp graph` turns HypAware recordings into a queryable **activity graph** and walks it. The graph is a derived projection of the `ai_gateway_messages` dataset (the same data the `hypaware-query` skill reads), read as *relationships* instead of rows. + +## Availability: is the graph enabled here? + +If `hyp graph` comes back as an unknown command, or `node`/`edge` are missing from the datasets in `hyp query status`, the graph plugins are not in the active config. They ship bundled with the package but activate only when the config names them, and on fleet-joined hosts the plugin set comes from the central config layer, which may omit them. + +The fix is additive and works even on fleet-locked installs (the central layer locks only the plugin names it declares; the local layer may contribute the rest): add `{"name": "@hypaware/context-graph"}` and `{"name": "@hypaware/ai-gateway-graph"}` to the `plugins` array of the local `/hypaware-config.json`, verify with `hyp config validate`, then build the graph with `hyp graph project`. **This edits the user's config - propose the change and get their go-ahead rather than editing it silently.** If the central config later adds the same plugins, the local duplicates are dropped benignly (recorded as `collides_with_central`, visible in `hyp status`). + +A fleet server may have the graph enabled even when this machine does not (or vice versa): `hyp graph neighbors ... --remote ` and `hyp query sql "... from node ..." --remote ` use the server's graph and the server's projection schedule, not the local ones. Local and remote graphs answer at different scopes (this machine vs the fleet) and can legitimately disagree; treat the difference as coverage, not error. + +## Build or refresh the graph + +The projection runs **on demand**; it does not auto-update. Run it before querying, and again after new sessions are recorded: + +```bash +hyp graph project # project ai_gateway_messages -> node/edge tables (idempotent) +hyp graph project --dry-run # show what would be written +hyp graph compact # merge duplicate rows, rewrite partitions sorted (maintenance) +``` + +`hyp graph project` prints `N node(s), M edge(s) - wrote ...`. If `node`/`edge` come back empty, the projection has not been run yet (or there are no recordings). For recency-sensitive questions, check `max(first_seen)` in `node`, or just re-project first: it is idempotent and cheap. + +## The graph model + +Deterministic T0 projection: exact-key, no models. The core is **Session-rooted**, plus a small git-provenance sub-web (`Repo`, `Commit`) that doubles as the bridge to GitHub enrichment (see below). + +- **Node types:** `Session` (one per session), `App` (client), `Model`, `Tool`, `File`, `Skill`, `Program`, and, when the session's git context is captured, `Repo` and `Commit`. +- **Edge types:** `via` (Session->App), `used_model` (Session->Model), `used` (Session->Tool), `touched` (Session->File), `ran` (Session->Skill), `invoked` (Session->Program), `in` (Session->Repo), `at` (Session->Commit, the HEAD the session sat on), and `in` (Commit->Repo). +- **natural_key** is the human key per type: Session = `session_id` (the always-present session container; `conversation_id` is a nullable thread identity, null for Claude), App = `client_name`, Model = model id, Tool = tool name, File = full path (or `owner/repo:relpath` when the repo remote is known; its `label` is the basename), Skill = bare skill name, Program = lowercased basename, Repo = `owner/repo`, Commit = full 40-hex sha. Every row carries inline provenance (`source_dataset`, `projector`, ...). +- **Skills and programs are derived facets.** They have no column in `ai_gateway_messages`: `ran` edges come from multi-surface skill-activation detection (and carry `dispatch_*` boolean props saying how the skill was activated), `invoked` edges from argv[0] extraction with wrapper unwrapping. Ad hoc reconstruction from raw rows measurably disagrees with the canonical derivation (a 3-surface LIKE approximation returned 52 sessions where the strict rules give 44; a first-token approximation of "programs" returned 470 garbage tokens against the graph's 86 clean ones). Always answer skill/program questions from the graph. +- **Keys converge where raw spellings diverge.** Skill and Program nodes are keyed identically across claude and codex, so per-skill and per-program questions span both clients for free. Repo nodes normalize remote-URL forms that a raw `git_remote LIKE` misses (in one measurement the graph found 312 sessions in a repo where the LIKE found 240). +- **Bridge-ready keys.** `Repo`, `Commit`, and `File` use shared, content-addressed natural keys, so a node minted from a session's git context and the same node minted by the GitHub source converge on one id. This is what lets an AI session and a pull request meet at the same commit. + +## GitHub enrichment (server-side) + +The base graph above comes from `ai_gateway_messages` and is available anywhere, including a local install. A **server** can additionally run the `@hypaware/github` source, which captures repo / commit / PR / issue / review events and projects a second contract into the **same** `node` / `edge` tables. This is the graph's biggest payoff, and none of it is answerable from the message data alone. + +**Caveats first, so you don't query nodes that aren't there:** + +- **Server-only and opt-in.** GitHub nodes exist only on a host where the `@hypaware/github` source is configured and has captured events (normally the central server, reached with `--remote`). A plain local projection has none of them. An empty GitHub query usually means the source is not configured on that host, or the graph has not been re-projected since capture, not that the true answer is zero. +- **`Actor` is a GitHub login, not the AI user.** The identity that authored a commit or opened a PR is the git actor, not the `user_id` of whoever ran the agent. Cross-domain identity merge is later work (T1/T2); never equate an `Actor` with an AI operator. +- **Freshness applies here too.** The GitHub domain is only as current as the last projection on that host, and you cannot project through the read-only query token (projection is admin-side). On a stale central graph, recent PRs and reviews are simply missing. One subtlety: freshly projected rows sit in a spool until a settling read runs **on the server** (an admin-side query; the remote query surface never settles), and `node` and `edge` settle independently, so a graph can briefly show fresh nodes joined by stale edges. If a cross-domain join returns implausibly few rows against fresh-looking nodes, suspect an unsettled `edge` dataset before doubting the data. + +**What it adds:** + +- **Nodes:** `Actor` (login), `Issue`, `PullRequest`, `Review`, plus enriched `Repo` / `Commit` / `File`. +- **Edges:** `authored` (Actor->Commit), `opened` and `commented` (Actor->Issue | PullRequest), `submitted` (Actor->Review), `on` (Review->PullRequest), `references` (PullRequest->Commit), `touched` (Commit->File and PullRequest->File), `in` (Commit | File | Issue | PullRequest->Repo). + +**Why it matters, the possibility.** Because `Repo`, `Commit`, and `File` are bridge-ready, the AI-session web and the GitHub web are *one graph*. The commit a session sat on (`Session -at-> Commit`) is the same node GitHub knows through `PullRequest -references-> Commit` and `Actor -authored-> Commit`. So you can walk from an agent's activity into the code-review reality around it, questions `ai_gateway_messages` cannot express: + +- **AI work to the PR that shipped it:** `Session -at-> Commit <-references- PullRequest`. +- **AI work to who reviewed it:** continue `PullRequest <-on- Review <-submitted- Actor`. +- **Coverage, honestly:** which repos and PRs an agent's work actually reached, not just which cwd it ran in. +- **Reverse:** start from a `PullRequest` or `Repo` and walk inbound to every AI session that touched it. + +```bash +# Sessions whose HEAD commit is referenced by a PR (AI work that reached code review). +# Note: no --refresh with --remote; the server owns its freshness. +hyp query sql "select distinct s.natural_key session + from edge a join node s on a.src_id = s.node_id + join edge r on r.dst_id = a.dst_id and r.edge_type = 'references' + where a.edge_type = 'at'" --remote HYP_CENTRAL + +# From a PR, walk out to its reviews, actors, and referenced commits. +hyp graph neighbors owner/repo#123 --type PullRequest --depth 2 --direction both --remote HYP_CENTRAL +``` + +Reach for the GitHub domain whenever a question spans **both** AI activity and code collaboration (sessions to PRs, agents to reviewers, work to repos). If it is purely one side, the base graph or plain message SQL is enough. + +## Two ways to query + +### 1. SQL over `node` / `edge`: counts, filters, aggregates + +Flat SQL through the normal query surface (no recursion). Use for "how many", "top N", "group by", one- or two-hop joins: + +```bash +hyp query sql "select node_type, count(*) n from node group by node_type" --refresh always +hyp query sql "select t.natural_key tool, count(*) n from edge e join node t on e.dst_id = t.node_id + where e.edge_type = 'used' group by tool order by n desc" --refresh always +``` + +`node`/`edge` are ordinary datasets; see the `hypaware-query` skill for `hyp query` mechanics (read stderr, freshness, `--format json`). Use `--refresh always` so the read settles freshly-projected rows. + +### 2. Traversal with `hyp graph neighbors`: relationships and depth + +For "what connects to X", co-occurrence, and N-hop walks, what flat SQL can't express. + +```bash +hyp graph neighbors [--depth N] [--type T] [--edge-type T] [--direction out|in|both] [--limit N] [--json] +``` + +- **``** resolves by `node_id`, then `natural_key`, then `label`, so pass a session id, a file path (or basename), a model id, or a tool/skill/program/app name. If it's ambiguous the command lists the candidates; narrow with `--type Session|App|Model|Tool|File|Skill|Program|Repo|Commit|Actor|Issue|PullRequest|Review` (the last four require GitHub enrichment). +- **`--direction`** is load-bearing because edges are Session-rooted: + - `out` from a **Session** -> its app, model, tools, files, skills, programs. + - `in` from a **File / Tool / Skill / Program / Model / App** -> the **Sessions** that touched/used/ran/invoked it. + - `both` at `--depth 2` from a **File** -> **co-occurrence**: the other files/tools/models reached through the sessions that share it. +- **`--edge-type`** restricts which relations are walked (e.g. `--edge-type used`); repeatable or comma-separated. +- **`--limit`** caps output in BFS order and reports the true total when it truncates (`... - truncated; raise --limit`). +- **`--json`** emits `{ seed, neighbors: [{ hop, edge_type, direction, node }], reachable, truncated }` for follow-up reasoning, with **full node ids**. Note the flag is `--json`, not `--format json` (the latter is silently ignored and you get the text table). The bracketed id in the text output (`[c1446c4f2b01]`) is display-truncated - pasting it into SQL matches nothing; take full ids from `--json`, or resolve them from the `node` table by `natural_key`. + +Examples: + +```bash +hyp graph neighbors sess-abc123 --direction out # what a session used / touched / ran +hyp graph neighbors src/auth.py --direction in # which sessions touched a file +hyp graph neighbors src/auth.py --depth 2 --direction both # files/tools that co-occur with it +hyp graph neighbors Bash --direction in --edge-type used # sessions that ran a tool +hyp graph neighbors dataviz --type Skill --direction in # sessions that ran a skill +hyp graph neighbors aws --type Program --direction in # sessions that invoked a program +hyp graph neighbors claude-opus-4-8 --type Model --direction in # sessions that used a model +``` + +**File-node identity is split**: the same physical file can exist as both a repo-scoped node (`owner/repo:src/x.js`) and one or more absolute-path nodes (worktree and tmp-dir copies included). For a complete "who touched this file" answer, enumerate the keys first, then walk each: + +```bash +hyp query sql "select node_id, natural_key from node where node_type='File' and natural_key like '%src/core/runtime/bundled.js'" --format json +hyp graph neighbors --direction in +``` + +## Choosing the right tool + +Ask: does answering require *reading* rows, or only knowing they *exist and connect*? Route to the graph when the question is any of: (1) the answer is a set of identifiers, not text (membership/reachability); (2) the predicate is derived, not stored (skills, programs - see the model section); (3) it crosses two or more relationships (co-occurrence, indirect association) - the graph pre-materialized the join the raw route would express as a brittle correlated subquery; (4) it is an inventory/existence question - the node table is a pre-computed DISTINCT over all history; (5) identity needs normalizing across raw spellings (repos, cross-client skills). + +Then pick the surface: + +- Counting, ranking, grouping, "how often" -> **`hyp query sql`** over `node`/`edge`. Distinct-session counts key on the edge (`count(distinct src_id)`), far fewer rows than `count(distinct session_id)` over messages (benchmarked ~12x fewer for the repo rollup): sessions per tool = `used`, per model = `used_model`, per file = `touched`, per skill = `ran`, per program = `invoked`, per app = `via`, per repo = `in`, per commit = `at`. +- "What connects to X", paths, neighborhoods, co-occurrence, depth -> **`hyp graph neighbors`**. + +Stay on `ai_gateway_messages` when the measure lives on the message, not the relationship: + +- token sums and cache-read ratios; `count(*)` call totals (the graph keeps one edge per (session, entity) pair - an edge means "at least once", never a count); `is_error` / `is_sidechain` / stop-reason; ordering and time inside a session; and `content_text` classification. +- per-`gateway_id` or per-`user_id` rollups: there are no Gateway or User nodes yet. + +**Default strategy is two-stage**: the graph decides WHICH sessions or entities matter, then raw SQL reads WHAT happened inside them - a `session_id`-scoped messages query is as fast as the graph (~0.15s), while an unscoped one grows with history. The join is direct: a `Session` node's `natural_key` IS the `session_id` column in `ai_gateway_messages`. + +```bash +# 1. Which sessions used the tool (take full ids and session UUIDs from --json) +hyp graph neighbors --type Tool --direction in --json +# 2. The calls themselves, with args +hyp query sql "select message_index, tool_call_id, json_extract(tool_args,'\$.') a + from ai_gateway_messages where session_id='' and part_type='tool_call' and tool_name=''" --format json +# 3. Each call's result: tool_result rows point back via tool_result_for +hyp query sql "select content_text from ai_gateway_messages + where session_id='' and tool_result_for=''" --format json +# 4. Surrounding conversation: window on message_index +hyp query sql "select message_index, role, part_type, content_text from ai_gateway_messages + where session_id='' and message_index between and order by message_index, part_index" --format json +``` + +Coverage of graph and messages is near-identical when the graph is freshly projected, but they can drift (the graph only updates on `hyp graph project`; message rows can be pruned by retention), so treat an empty drill-down as "check freshness", not "no data". + +## SQL performance over node/edge + +SQL over `node`/`edge` has sharp performance tiers (measured): `graph neighbors` traversal ~0.2s; an edge self-join anchored on a **literal node_id** ~3s; the same join with a scalar subquery (`e1.dst_id = (select node_id from node where ...)`) ~33s. Resolve seed node_ids first (via `--json` or a separate lookup query) and inline them as literals. Use SQL only when you need per-edge weights (`count(distinct e.src_id)`) that `neighbors` (a deduplicating BFS) cannot report. + +The join planner has intermittently failed non-trivial edge self-joins with `Column ... not found`. If that happens, keep the edge self-join adjacent and early, or materialize it as a subquery and join `node` in the outer query; resolve a second node lookup in a separate query. + +## Guardrails + +- **Project first.** The graph is only as fresh as the last `hyp graph project`; empty results usually mean it has not run. +- **Read stderr for errors.** `graph neighbors` writes not-found / ambiguity notes (and the large-graph note) to stderr; exit `1` is a resolution error, `2` is a usage error. **Truncation is part of the result**, so it goes to **stdout** (`... - truncated; raise --limit`) and the `--json` `truncated` field, not stderr. +- The graph is **derived and rebuildable**, never the source of truth. To change what it contains, fix capture/projection and re-project; don't hand-edit `node`/`edge`. +- Basic traversal loads the graph in memory per call, fine at activity-graph scale; a very large graph prints a note pointing at the future indexed path. diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-query/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-query/SKILL.md index 2a549999..dab4d8ea 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-query/SKILL.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-query/SKILL.md @@ -58,8 +58,16 @@ Never read a smaller CLI row count as "fewer rows matched" — it is the display ## SQL dialect notes -- `json_extract_scalar()` does not exist. `JSON_EXTRACT` does, but it errors on rows where a JSON-typed column (notably `tool_args`) holds a plain string instead of a JSON object ("first argument must be JSON string or object, got string"). +The engine is SELECT-only with a deliberately small SQL surface. Every bullet below is a rejection observed in recorded sessions; when a query fails, the error message echoes the available columns, so read it before retrying. + +- SELECT-only: `SHOW`, `DESCRIBE`, DDL, and `information_schema` are parse errors. Discover a table's columns with `hyp query schema ` or `SELECT * FROM
LIMIT 1`, never introspection statements. Dataset names come from `hyp query status` (on a standard install: `ai_gateway_messages`, `node`, `edge`); never guess a table name. +- Boolean predicates: `IS NOT TRUE` / `IS TRUE` are not parsed (`NOT` must be followed by `NULL`). Compare directly: `col = true`, `col = false`, or `col IS NULL`. +- Cast types are only STRING, INT, BIGINT, FLOAT, BOOL. `TRY_CAST`, `CAST(... AS TIMESTAMP)`, and `TIMESTAMP '...'` literals do not exist. Filter time ranges on the STRING `date` column (`date >= 'YYYY-MM-DD'`); the event-time column is `message_created_at` (there is no `timestamp` column). +- `ANY_VALUE` does not exist: use `MAX`/`MIN`. `regexp_like` does not exist: use `REGEXP_MATCHES` for a boolean match, `REGEXP_SUBSTR` to extract, or plain `LIKE`. `LIKE ... ESCAPE` is not parsed. +- Regexp position arguments are 1-based: `regexp_extract(str, pattern, 1)`, never `0`. +- `json_extract_scalar()` does not exist. `JSON_EXTRACT` does, but it errors on rows where a JSON-typed column (notably `tool_args`) holds a plain string instead of a JSON object ("first argument must be JSON string or object, got string"). Dotted identifiers (`usage.output_tokens`) are not columns; extract JSON fields explicitly. - The robust pattern for extracting fields from `tool_args` is a regex over the raw text, e.g. `regexp_extract(CAST(tool_args AS VARCHAR), '"command":"([^"]+)', 1)`. +- Never invent a `--remote` target name: discover configured targets with `hyp remote list` first. ## AI gateway message model diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/SKILL.md index f5b31e40..23435b4e 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/SKILL.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/SKILL.md @@ -15,8 +15,9 @@ Two moving parts: 1. **`build.sh`** (in the repo, pandoc-based) converts each `.md` — plus any `/` sections — into a self-contained `html//` folder: `index.html` for the - one-pager, one `
.html` per section with a "← Back to the report" nav, - `assets/style.css`, and a `.nojekyll`. It rewrites inter-file `.md` links to `.html` + one-pager with a "← All reports" nav back to the top-level landing page + (`../../index.html`), one `
.html` per section with a "← Back to the report" + nav, `assets/style.css`, and a `.nojekyll`. It rewrites inter-file `.md` links to `.html` on the **emitted HTML** (`href` attributes), which catches Markdown-syntax links and links inside raw-HTML components (`rec` cards, callouts) in one pass: it flattens the one-pager's `/section.md` links to `section.html` and maps cross-report links @@ -66,17 +67,20 @@ Work relative to the repo root `~/hypaware-reports`. system; do not hand-tune per-page CSS. 3. **Enrich the report Markdown (the step that makes it a data report).** For each - top-level `.md`, check whether it already uses the component vocabulary — - **both** halves of it, because partial enrichment is the common failure: + top-level `.md`, check whether it already uses the component vocabulary: ```bash - grep -L 'class="metric-grid"' *.md # reports with no headline metric strip at all - grep -L 'class="rec"' *.md # reports whose findings are still prose-only + grep -L 'class="rec"' *.md # reports whose findings/changes are still prose-only + grep -L 'class="metric-grid"' *.md # reports with no headline metric strip (see below) ``` - A one-pager that has a `metric-grid` but no `rec` cards is **half-done, not done**: - its findings sit as `###` heading + paragraph + trailing link, so every number is - buried in a sentence and the reader has to open the section pages to see anything at - a glance. Finish it — convert the findings to cards (and rename any scaffolding - headings) rather than skipping it. + `rec` cards are required on every one-pager (findings or proposed changes). A + `metric-grid` is required **only where the source has a headline-numbers section** + (the usage and security reviews' "Key metrics"). The improvement review deliberately + has none — it opens with a numbered **Proposed changes** list (user-approved shape, + 2026-07-14); each numbered change becomes one `rec` card (bold what = card title, + why-sentence = body, evidence numbers = stat row, section link = the card) and **no + metric strip is added above or instead of the change list**. A one-pager with a + metric-grid but no `rec` cards is **half-done, not done** — finish the cards rather + than skipping it. For every report needing work, proceed in **two phases — inventory first, markup second**: @@ -94,15 +98,25 @@ Work relative to the repo root `~/hypaware-reports`. existing structure. Apply the recipe in [`authoring.md`](authoring.md) using ONLY the Phase A inventory, with [`example-enrichment.md`](example-enrichment.md) as a *shape* reference — and take a designer's liberties: - - **Restructure freely.** Reorder sections for narrative strength, merge or retitle - weak headings, delete decorative `---` rules and boilerplate scaffolding - ("Key numbers", "What this shows" → headings that say something). The document's - source order is not sacred; its facts are. + - **Restructure within the approved skeleton.** Merge or retitle weak headings inside + sections, delete decorative `---` rules — but the one-pager's top-level order is + **user-approved structure, not scaffolding** (2026-07-14): proposed changes / findings + lead, limitations and links follow. Never move a metric strip above a change list, + never split the evidence back out into a separate findings section, and keep the + standard heading vocabulary (Proposed changes / Key metrics / Key findings / Data + limitations / Supporting analysis) — retitle only headings that aren't part of that + skeleton. - **Rewrite for the surface.** Metric labels, card titles, stat labels, tag words, chart titles, and notes are *display copy* — write them fresh (2–4 word labels, one - punchy "so what" note), never paste sentence fragments from the prose. Body - paragraphs — the analysis itself — stay intact apart from trims where a visual now - carries the point. + plain "so what" note), never paste sentence fragments from the prose. Display copy + obeys the report-language rules: literal words, no metaphors or coined shorthand + (write "sessions open across days", never compress to a coinage like "marathon + sessions"), no pipeline vocabulary, absolute dates. Body paragraphs — the analysis + itself — stay intact apart from trims where a visual now carries the point. + - **Ready-to-apply artifacts are verbatim.** Proposed diffs, full skill/subagent file + drafts, tool-description text, and source→destination move tables render as the + code blocks / tables they are — never trimmed, carded, summarized, or reworded. They + are the deliverable, not display copy. - **Give every headline number the big treatment.** Any number the report leads with belongs in a `metric`, `gauge`, `rec-stat`, or chart — large, colored by judgment, with a note — not bolded inline in a sentence. After the pass, a number that matters @@ -149,6 +163,25 @@ Work relative to the repo root `~/hypaware-reports`. It prints `Built html/ : N report(s) …`. `html/` is wiped and rebuilt, so deleted or renamed reports never leave stale HTML behind. + **Every report page must offer a way back to the landing page.** `build.sh` is + responsible for this: it prepends a `topnav` to each one-pager before pandoc runs. If + the repo's `build.sh` predates this (no `All reports` string in it — + `grep -q 'All reports' build.sh`), add the injection where the one-pager is built, then + re-run it: + ```bash + { + printf '\n\n' + cat "$src" + } | pandoc -f gfm -t html5 -s \ + --css assets/style.css \ + --metadata pagetitle="$(page_title "$src" "$slug")" \ + -o "$out/index.html" + ``` + (i.e. pipe the nav + source into pandoc instead of passing `"$src"` as the input + file.) Section pages already chain back: "← Back to the report" → one-pager → + "← All reports" → landing page. The nav goes in **build.sh, not the source `.md`** — + the Markdown must stay renderer-agnostic. + 5. **Regenerate the top-level `index.html` as an at-a-glance dashboard, not a table of contents.** Write a fresh landing page (template in [`components.md`](components.md) → *Landing-page template*). One card per built report, newest first (slugs are @@ -158,11 +191,13 @@ Work relative to the repo root `~/hypaware-reports`. - **Kicker** - the report's scope line (the italic `*Source: … · Window: … *` line for adoption profiles, or the `## · ` subtitle for the others), trimmed to a short phrase, as the card's `rec-kind` eyebrow. - - **Stats** - the report's top 3-4 `metric-grid` tiles (step 3 guarantees they exist), - re-rendered as `rec-stat`s on the card: same values, same crit/warn/good judgments, - labels compressed to 2-4 words, notes dropped. Rules in components.md. This hoists - each report's key results and progress onto the landing page, so a reader gets the - fleet's state without opening a report. + - **Stats** - the report's top 3-4 headline numbers as `rec-stat`s on the card: from + its `metric-grid` tiles where it has one, otherwise (change-list reports like the + improvement review) from the `rec` cards' stat rows — same values, same + crit/warn/good judgments, labels compressed to 2-4 plain words (no coined + shorthand), notes dropped. Rules in components.md. This hoists each report's key + results and progress onto the landing page, so a reader gets the fleet's state + without opening a report. Link each report by its explicit `html//index.html`, **not** a bare `html//` directory URL. A trailing-slash directory link relies on server-side @@ -177,16 +212,17 @@ Work relative to the repo root `~/hypaware-reports`. ls html/ # one dir per report grep -o '[^<]*' html/*/index.html grep -rlo 'href="[^"]*\.md"' html/ || echo "no leftover .md links ✓" - grep -L 'metric-grid' html/*/index.html # should print nothing - grep -L 'class="rec"' html/*/index.html # should print nothing: findings are carded + grep -L 'class="rec"' html/*/index.html # should print nothing: findings/changes are carded grep -c 'rec-stat' index.html # ≥ number of reports: landing cards carry stats + grep -L 'All reports' html/*/index.html # should print nothing: every report links back ``` `href="….md"` in any built page means a link wasn't rewritten — remember links live both in Markdown syntax **and** inside raw-HTML components (`rec` card and callout `href`s), and may point across reports; investigate before publishing. A page missing - `metric-grid` means step 3 was skipped for that report; one missing `rec` cards means - step 3 stopped halfway (headline strip done, findings left as prose); a landing page - without `rec-stat`s means step 5 produced a bare link list. + `rec` cards means step 3 was skipped or stopped halfway; a `metric-grid` is only + expected where the source report has a headline-numbers section (do NOT add one to a + change-list report to satisfy a check); a landing page without `rec-stat`s means + step 5 produced a bare link list. Optionally open `index.html` (or `html//index.html`) in a browser to eyeball it (check both light and dark — the stylesheet supports both). diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/authoring.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/authoring.md index 80b8ec4c..f9a5230d 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/authoring.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/authoring.md @@ -44,12 +44,16 @@ failures … and let the whole team run a flow only phil has.** The scope/date line becomes an `eyebrow` above the title. The thesis stays one bold paragraph — the stylesheet turns it into the hero box. -## 2. "Key numbers" — metric grid, not a table +## 2. Headline numbers — metric grid, not a table (only where the report has them) -Every report's headline numbers section (the 2-column Metric/Readout table) becomes a -`metric-grid` of 3–6 cards. Color carries judgment: `is-crit` = problem, `is-warn` = -exposure, `is-good` = healthy/solved, no class = neutral. Keep any *detail* tables that -follow; only the headline strip converts. +A report's headline-numbers section (the usage and security reviews' **Key metrics**) +becomes a `metric-grid` of 3–6 cards. Color carries judgment: `is-crit` = problem, +`is-warn` = exposure, `is-good` = healthy/solved, no class = neutral. Keep any *detail* +tables that follow; only the headline strip converts. + +**Not every report has one.** The improvement review opens with a numbered **Proposed +changes** list and has no metrics section by design (user-approved, 2026-07-14) — do NOT +add a metric strip to it; its changes become `rec` cards (§3) and lead the page. ```markdown

The numbers that set the agenda

@@ -71,16 +75,25 @@ follow; only the headline strip converts. Every card needs a `note` that says why the number matters — a bare number is not a finding. -## 3. Findings index — rec cards, not `###` + link +## 3. Findings / proposed changes — rec cards, not `###` + link -On the one-pager, each finding/recommendation that links to a section page becomes an +On the one-pager, each finding or proposed change that links to a section page becomes an `` card (number badge, kind eyebrow, title, 1–2 sentence body, 2–3 stat row, go-link). See the full snippet in `components.md`. The `###` heading + trailing `section →` pattern is replaced by the card — don't emit both. +For the improvement review's numbered change list, the mapping is fixed: bold imperative += card title, the why-sentence = body, the evidence numbers = stat row, the section link += the card itself; card order = list order (highest leverage first — never resequenced). + Stat-row discipline: 2–3 stats per card, each `valuelabel`, color class only when it carries judgment. +**Ready-to-apply artifacts stay verbatim.** Proposed AGENTS.md diffs, full skill/subagent +file drafts, tool-description text, and source→destination move tables are deliverables, +not display copy: render them as the code blocks / tables they are, never trimmed, +carded, or reworded. + ## 4. Section pages: every claim gets a visual, breakdowns get charts Each section page opens with its own bold thesis directly under its `# ` title (hero @@ -125,9 +138,14 @@ literal): - **Tag words** — a single judgment noun: Exposure, Risk, Solved, Caveat, Basis. - **Chart titles** — name the axis and scope ("Edit-tool errors by message · 30 days"); **chart-foot** — the takeaway, one line. -- **Section headings** — replace scaffolding ("Key numbers", "What this shows") with - headings that carry the message ("The numbers that set the agenda", "The four - recommendations"). +- **Language rules bind display copy too** (user feedback 2026-07-14): literal words + only — no metaphors, pipeline vocabulary, or coined shorthand (write "sessions open + across days", never a coinage like "marathon sessions"); when a card names a skill or + tool as a fix, the body says in one clause what it literally does; dates absolute. +- **Section headings** — the one-pager's skeleton headings (Proposed changes / Key + metrics / Key findings / Data limitations / Supporting analysis) are user-approved + standard vocabulary: keep them. Inside section pages, retitle weak headings to state + the literal fact ("Worker lanes default to Opus"), never a punchy coinage or metaphor. Conversion is the floor, not the bar: a page that preserves the Markdown's structure and phrasing with components sprinkled in is a failed pass. The Markdown supplies facts, @@ -138,8 +156,12 @@ are designed. - [ ] Eyebrow + `# title` + bold thesis, nothing between them — on the one-pager AND every section page. -- [ ] Headline numbers are a `metric-grid` with judgment colors and notes. -- [ ] Findings on the one-pager are `rec` cards with stat rows. +- [ ] Headline numbers are a `metric-grid` with judgment colors and notes — only on + reports that have a Key metrics section; none added to change-list reports. +- [ ] Findings / proposed changes on the one-pager are `rec` cards with stat rows, in + source order. +- [ ] Diffs, proposed files, and move tables are verbatim code blocks/tables — nothing + trimmed or reworded. - [ ] Each section page's visuals each carry a distinct claim (typically 2-3 per page); source tables kept where numbers matter. - [ ] Every per-entity rollup (by user/gateway, team, repo, model) has a companion diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/example-enrichment.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/example-enrichment.md index 48590819..9f10fdb7 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/example-enrichment.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-report-to-html/example-enrichment.md @@ -13,6 +13,14 @@ the reference for SKILL.md step 3: same moves, same class names, numbers taken v from the plain version. Component reference: [`components.md`](components.md); rules: [`authoring.md`](authoring.md). +> ⚠ **The BEFORE below is the improvement review's OLD source shape.** Since 2026-07-14 +> that report emits a numbered **Proposed changes** list with no "Key numbers" table and +> no findings section (authoring.md §2–3). For today's improvement review: no +> `metric-grid` anywhere on its one-pager — map each numbered change to one `rec` card +> (bold what = title, why = body, evidence = stat row) in source order. The +> metric-grid moves below still apply to reports that HAVE a headline-numbers section +> (usage, security). The class names and width/judgment mechanics are unchanged. + ## BEFORE — plain Markdown as the report skills emit it ```markdown diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-sensitive-scan/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-sensitive-scan/SKILL.md new file mode 100644 index 00000000..e47c830c --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-sensitive-scan/SKILL.md @@ -0,0 +1,105 @@ +--- +name: hypaware-sensitive-scan +description: Scan HypAware recordings for sensitive content (secrets, credentials, secret-file reads, PII, private-repo code) and recommend what to opt out of recording - a folder-scoped `.hypignore` / `hyp ignore ` for directory-bound exposure, or a per-session opt-out for one-off sensitive sessions. Use when the user says "scan my logs for secrets", "what should I hypignore", "did I record anything sensitive", or wants ignore recommendations grounded in recorded data. REDACTS every secret (never echoes a value). Recommends PROSPECTIVE opt-outs only - it does NOT purge already-recorded rows. Asks which source to scan (local logs or a remote server) first. +--- + +# Scan recordings for sensitive data, recommend ignores + +Turn a window of recorded AI-gateway exchanges into a short, ranked answer to one +question: **what sensitive content did HypAware capture, and where should the user +turn recording off so it stops happening.** The deliverable is copy-paste ignore +commands grouped by scope, not a saved report. + +IMPORTANT: Both ignore mechanisms are **prospective only** - a `.hypignore` and the +session opt-out gate *future* recording; neither purges rows already in the cache +(LLP 0049 §prospective-only). So the sensitive rows you find stay captured. Say this +plainly and never imply the scan removes anything. + +IMPORTANT: **REDACT everything.** Never echo a secret, token, key, credential, or raw +PII value - not in a finding, not from `content_text`, not from `tool_args`. A secret +appearing in the recording *is* the finding; name the scope and class, quote nothing. + +IMPORTANT: **Ask which source first.** List the data sources and let the user choose: +**local logs** (this machine, `hyp query sql …`, no `--remote`) and each remote +HypAware server (every `hyp remote list` target, plus any hypaware MCP `query_sql` +tool already in your toolset; the same server can appear both ways - list it once). +Query mechanics are step 0 of the procedure — a mandatory read, not a reference. + +## What counts as sensitive (match over `content_text` AND `CAST(tool_args AS VARCHAR)`) +| Class | Match signals | Why it matters | +| --- | --- | --- | +| Secret in payload | `AKIA…`, `sk-…`, `ghp_…`/`gho_…`, `xox[baprs]-…`, `-----BEGIN … PRIVATE KEY-----`, `Bearer `, `password=`/`api_key=`, `postgres://user:pass@` | a live credential now sits in the cache | +| Secret-file read | reads of `.env`, `id_rsa`, `.ssh/`, `.aws/credentials`, `.netrc`, `*.pem`, `*.key`, `credentials.json` | key/config contents captured verbatim | +| PII | email, phone, national-id and card-like number runs | personal data captured | +| Private code / infra | private `git_remote`, internal hostnames, connection strings, non-public source paths | proprietary IP in the cache | + +## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). This scan's core job IS reading wide columns + (`content_text`, `tool_args`) — keep those queries aggregate-shaped regex matches over + **date-sliced, server-sized ranges** (especially against a remote server), never bulk + row-fetches of raw content; the step-2 rollup GROUP BY is on scope keys, not content. Use the + graph's tiny `node`/`edge` tables for the scope inventory where projected (which + sessions/cwds/repos exist, session→directory attribution via Session nodes' `props.cwd`) + instead of DISTINCT-scanning message columns. Capture stderr and check it even on success — + truncation and server-cap notices land there, and a clipped scan reads as "no hits". If a + query fails, come back to this step; don't iterate on the failing SQL. +1. **Pick the source** (above), then a **baseline**: window; total rows; distinct + `repo_root` / `cwd` / `session_id`; coverage of `content_text` and `tool_args` (a + sparse column bounds what the scan can see - say so). State N. +2. **Scan.** Match each class over `content_text` and `CAST(tool_args AS VARCHAR)` + (`tool_args` is JSON that may hold a plain string - never `JSON_EXTRACT` it; regex + the cast text per the hypaware-query dialect notes). For every hit, capture the + **scope keys** `repo_root` (fallback `cwd`), `session_id`, and `class` - **not** the + matched value. Roll up hit counts per `(scope, class)` and per `(session_id, class)`. + + ```sql + SELECT repo_root, cwd, session_id, + count(*) AS hits + FROM ai_gateway_messages + WHERE regexp_matches(coalesce(content_text,'') || ' ' || CAST(tool_args AS VARCHAR), + '(AKIA[0-9A-Z]{16}|sk-[A-Za-z0-9]{20,}|ghp_[A-Za-z0-9]{20,}|-----BEGIN [A-Z ]*PRIVATE KEY-----|xox[baprs]-|postgres://[^:]+:[^@]+@)') + GROUP BY repo_root, cwd, session_id + ORDER BY hits DESC + ``` +3. **Route each scope to a recommendation.** Two shapes, matched to how the exposure is + bound: + - **Directory-bound** (a `repo_root`/`cwd` that keeps producing hits across sessions, + or is inherently sensitive - a secrets repo, an infra tree) -> a **folder + `.hypignore`**, durable and committable/team-wide. + - **One-off** (hits isolated to a single `session_id` in a directory the user would + not want permanently ignored) -> a **per-session opt-out** at session start. + +## Recommendations (the output) +Print findings **most sensitive first** (weight: live-credential > secret-file > PII > +private-code; break ties by hit count and by how many distinct sessions a scope spans). +All examples redacted. Two groups, then the caveat: + +- **Folder ignores** - one line per scope: a redacted headline (` in , + N hits across M sessions`) followed by the copy-paste command on its own line: + ```bash + hyp ignore # writes a committable .hypignore; covers all future sessions in this tree + ``` + Note that an empty `.hypignore` at the repo root does the same for the whole repo, and + that folder matching keys on `cwd`/`repo_root` (Claude + Codex only; a no-op for + raw-proxy / OTEL sources, which carry no `cwd` - LLP 0049 §non-goals). +- **Session opt-outs** - for one-off sensitive work in a directory the user won't + permanently ignore: run `/hypaware-ignore` at the *start* of that session (in-memory, + current session forward, reverse with `/hypaware-unignore`). +- **Caveat (always).** The sensitive rows above are **already recorded**; ignoring is + prospective and does not delete them. Show the residual with + `hyp ignore --check ` (reports the governor and how many cached rows from the + scope remain). Retroactive purge is out of scope. + +## Notes +- Redaction is absolute; a raw secret in `tool_args` is itself worth flagging (the + gateway captured it) - name it, don't reproduce it. +- Read stderr on every query (staleness / row-budget notices land there); an empty + stdout is not the same as zero hits. +- Keep the pattern set honest: a high-recall regex over free text produces false + positives (an example string, a doc). Spot-check counts before recommending an ignore, + and prefer recommending the *scope* over asserting a specific leak you cannot verify. diff --git a/hypaware-core/plugins-workspace/claude/src/index.js b/hypaware-core/plugins-workspace/claude/src/index.js index b1543ced..fae0d68c 100644 --- a/hypaware-core/plugins-workspace/claude/src/index.js +++ b/hypaware-core/plugins-workspace/claude/src/index.js @@ -251,6 +251,9 @@ export async function activate(ctx) { 'hypaware-ignore', 'hypaware-unignore', 'hypaware-privacy', + 'hypaware-graph', + 'hypaware-sensitive-scan', + 'hypaware-ai-usage-report', 'hypaware-ai-adoption-report', 'hypaware-ai-improvement-report', 'hypaware-ai-security-report', diff --git a/hypaware-core/plugins-workspace/codex/hypaware.plugin.json b/hypaware-core/plugins-workspace/codex/hypaware.plugin.json index 2d32fa7f..f42586f2 100644 --- a/hypaware-core/plugins-workspace/codex/hypaware.plugin.json +++ b/hypaware-core/plugins-workspace/codex/hypaware.plugin.json @@ -2,7 +2,7 @@ "schema_version": 1, "name": "@hypaware/codex", "version": "2.0.0", - "description": "OpenAI Codex client adapter for HypAware. Registers the OpenAI-compatible upstream preset on the local AI gateway, configures Codex's config.toml to route through the gateway, and ships the hypaware-query helper skill plus the AI adoption/improvement/security/spend report skills and the report-to-html renderer.", + "description": "OpenAI Codex client adapter for HypAware. Registers the OpenAI-compatible upstream preset on the local AI gateway, configures Codex's config.toml to route through the gateway, and ships the hypaware-query, hypaware-graph, and hypaware-sensitive-scan helper skills plus the AI report skills and the report-to-html renderer.", "hypaware_api": "^1.0.0", "runtime": "node", "node_engine": ">=20", @@ -33,6 +33,9 @@ { "name": "hypaware-query", "clients": ["codex"] }, { "name": "hypaware-reference", "clients": ["codex"] }, { "name": "hypaware-privacy", "clients": ["codex"] }, + { "name": "hypaware-graph", "clients": ["codex"] }, + { "name": "hypaware-sensitive-scan", "clients": ["codex"] }, + { "name": "hypaware-ai-usage-report", "clients": ["codex"] }, { "name": "hypaware-ai-adoption-report", "clients": ["codex"] }, { "name": "hypaware-ai-improvement-report", "clients": ["codex"] }, { "name": "hypaware-ai-security-report", "clients": ["codex"] }, diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-adoption-report/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-adoption-report/SKILL.md index bac61fab..578440b8 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-adoption-report/SKILL.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-adoption-report/SKILL.md @@ -1,6 +1,6 @@ --- name: hypaware-ai-adoption-report -description: AI Adoption Profile for a HypAware server: descriptive "who's using the fleet and how": per-gateway utilization (volume + focus: top models, tools, repos, themes) and parallelism/fan-out (multi-agent adoption, concurrency, main-vs-subagent split, payoff). +description: "SUPERSEDED by hypaware-ai-usage-report (merged with the spend review, 2026-07-13) — invoke that skill instead unless the user explicitly asks for a standalone adoption profile. Original scope: AI Adoption Profile for a HypAware server: descriptive \"who's using the fleet and how\": per-gateway utilization (volume + focus: top models, tools, repos, themes) and parallelism/fan-out (multi-agent adoption, concurrency, main-vs-subagent split, payoff)." --- # AI Adoption Profile @@ -23,13 +23,31 @@ list it once). Present the options, ask which one (or more) to profile, then proceed against the chosen source. **Descriptive only.** Route every *action* out: "fan out more/less" and tooling go to -**hypaware-ai-improvement-report**, token waste to **hypaware-ai-spend-report**. Query mechanics -live in the **hypaware-query** skill; reuse hypaware-ai-spend-report's token spine for -any token figure. For descriptive who-used-what rollups (distinct sessions per -repo/model/tool/file), prefer the **hypaware-graph** skill, which reads them from the projected -graph instead of scanning messages; keep token figures on messages. +**hypaware-ai-improvement-report**, token waste to **hypaware-ai-spend-report**. Reuse +hypaware-ai-spend-report's token spine for any token figure. Query mechanics are step 0 below — +a mandatory read, not a reference. ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before you run any `hyp query sql`, read the **hypaware-query** skill (invoke it or + Read its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes and recall of past runs do NOT substitute: stale notes have cost real + runs failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` + scans that 504'd and then OOM'd the prod server). Then route every question by shape, per + hypaware-query's "when the graph answers it cheaper" boundary: + - **Graph first (`node`/`edge` — tiny, join-safe) for every entity/connection question:** + which sessions used a repo/model/tool/file, skill and program rollups (these two exist ONLY + in the graph — SQL reconstructions disagree with the projection), client mix, co-occurrence, + and gateway→person attribution — take `min/max(session_id)` per gateway from messages (an + ID-only aggregate), then look those session_ids up in graph Session nodes (`props.cwd`, + `props.client_name`). **Never GROUP BY / DISTINCT / row-fetch wide content columns (`cwd`, + `content_text`) on the messages table at scale** — that is the query shape that kills servers. + - **Messages (`ai_gateway_messages`) only for per-message measures:** token sums, distinct + part/session counts, timestamps and ordering, `is_sidechain`/`agent_id`, content sampling. + Slice long windows into server-sized date ranges; capture stderr and check it even on + success (truncation and server-cap notices land there). + If you skipped this step and a query fails, the fix is to come back here — not to iterate on + the failing SQL. 1. **Scope + coverage.** Distinct `gateway_id` (the unit; `user_id` is ~always null, so don't measure reach by it); window; coverage of `gateway_id`, token usage, and subagent provenance (`agent_id` / `is_sidechain` / `parent_thread_id`, transcript-enriched, may not survive diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-improvement-report/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-improvement-report/SKILL.md index 09b1512f..c9aa8ba1 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-improvement-report/SKILL.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-improvement-report/SKILL.md @@ -16,9 +16,7 @@ and let the user choose which to query: **local logs** (this machine's own recor `query_sql` / `graph_neighbors` tool in your toolset); the same server can appear both ways, list it once). Present the options, ask which one (or more) to run the review against, then proceed against the chosen source. -Query mechanics live in the **hypaware-query** skill; read it first. For descriptive -who-used-what rollups (distinct sessions per repo/model/tool/file), prefer the **hypaware-graph** -skill: it reads relationships from the projected graph instead of scanning messages. +Query mechanics are step 0 of the procedure — a mandatory read, not a reference. Focus on these signals: @@ -31,6 +29,23 @@ Focus on these signals: subagent, a context-hygiene rule, a skill that avoids the redo). ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). Route by shape, per hypaware-query's "when the graph + answers it cheaper" boundary: entity/connection questions go to the graph's tiny `node`/`edge` + tables — sessions per repo/model/tool/file, **skill and program rollups (graph-only facets — + SQL reconstructions measurably disagree with the projection)**, client mix, shared-file + clustering via `touched` edges, and gateway→person attribution (`min/max(session_id)` per + gateway from messages — an ID-only aggregate — then look those session_ids up in graph + Session nodes' `props.cwd`). The messages table is for per-message measures only — token + sums (the spend spine), `is_error`/retry loops, stop-reasons, ordering, prompt sampling — + date-sliced to server-sized ranges, and **never GROUP BY / DISTINCT / row-fetch wide content + columns (`cwd`, `content_text`) at scale** (that query shape kills servers). Capture stderr + and check it even on success — truncation and server-cap notices land there. If a query + fails, come back to this step; don't iterate on the failing SQL. 1. **Footprint + basis.** Window; distinct contributors (`gateway_id`), repos, sessions; claude/codex mix. State N and breadth; flag if single-contributor. 2. **Scan the signals for possible improvements.** Work the three signals above; each turns @@ -67,26 +82,56 @@ Focus on these signals: ## Output - SAVE A SHORT MAIN FILE + ONE FILE PER SECTION A **progressively-disclosed one-pager** is the deliverable: a reader can stop after the thesis -sentence, glance the key-numbers table, skim the findings, or follow any link into a detail -**section**, and each section is its own markdown file. Build the one-pager top-to-bottom so each -layer is shorter and more glanceable than the one it summarizes. +sentence, skim the numbered change list, or follow any link into a detail **section**, and each +section is its own markdown file. Build the one-pager top-to-bottom so each layer is shorter and +more glanceable than the one it summarizes. + +**Report language (applies to every file).** The reader may be a supervisor with no HypAware +knowledge. Plain literal language only: no pipeline vocabulary (ingest, sidechain, part, +projection), no metaphors or report-coined shorthand ("marathon sessions" was flagged as vague - +write "sessions kept open across days re-read their entire history each morning"); coined names +(a harness, an internal tool) get a one-line gloss at first mention on each page; when a skill or +tool is named as a fix, add one clause saying what it literally does ("`/handoff` writes a short +end-of-day note of where the work stands so the next day starts a fresh session"), never just its +name; absolute dates, never "last week". Findings equip, never direct: state who/size/driver/what +a change returns, never "ask X" / "talk to Y". In section files, any list item carrying several +facts is a bold topic line + 2-3 short sub-bullets, not a multi-sentence prose bullet. - **Main one-pager:** `hypaware-reports/-improvement-review.md` (create the dir if - needed). Dated so reviews accumulate. Lay it out in exactly these blocks, separated by `---` rules: - 1. **Title + scope** - `# AI Improvement Review`, then a `## · ` subtitle. - 2. **Thesis** - ONE **bold** sentence: "Create/edit these N things and what it will improve." - 3. **`### Key numbers`** - a small metric→readout table, ~4-6 rows, each one glanceable fact with - no link (# improvements, new-vs-edit split, the single biggest token prize, the basis: - gateways / sessions / repos). This is the ONLY table on the one-pager. - 4. **`## What this shows`** - 3-5 numbered findings = the top improvements, highest-leverage - first; each is a `### N. ` naming **what + its form** (skill / subagent / AGENTS.md - edit), followed by 2-3 plain-language sentences with the evidence and the **bold** token prize, - then a single `[
→](/.md)` link on its own line. Show AGENTS.md edits as - real diffs/code blocks in their section file, not prose. - 5. **`## Caveat`** - token prizes are floors and capture is partial, in plain language + a - `[caveats →]` link. - 6. **`## Report details`** - a one-line footer linking **every** section file written this run - (not just the cited ones), so nothing is orphaned - e.g. + needed). Dated so reviews accumulate. It is a **terse bullet brief** (user feedback 2026-07-14: + dense multi-clause sentences and prose paragraphs were both rejected as hard to read). Its only + headings are **Proposed changes / Data limitations / Supporting analysis** - standard + business-report vocabulary, never coinages like "What this shows" / "Key numbers" / "Report + details", and no metrics or findings section (the evidence rides inside the change list). Blocks: + 1. **Title + scope** - an eyebrow line ` · `, then `# AI Improvement Review`. + 2. **Thesis** - one SHORT bold sentence (the create/edit-these-N-things claim). If it needs + supporting clauses, put them in 2-3 plain sub-bullets under it, never packed into the sentence. + 3. **`## Proposed changes`** - the changes themselves lead, NOT a metrics block, and there is + NO separate findings/evidence section (user feedback 2026-07-14: a Key findings section + alongside the change list is redundant - the strongest evidence rides with each change). A + **numbered list**, one item per improvement, highest-leverage first, each exactly: + - the **what**: a short bold imperative naming ONE action (mechanics in parens after the + bold), nothing else on the line. Never join two actions with ";" or "+" in the bold line + (user feedback: "sounds like two suggestions, unclear") - when a change pairs a skill + move with a companion AGENTS.md rule, the bold names the primary action and the companion + rides in a sub-bullet; + - sub-bullet 1, the **why**: one short sentence with the token prize or headline number; + - sub-bullet 2, the **evidence**: one short line with the 1-2 strongest supporting numbers, + ending with the section link. + Never pack what+why+prize into the bold line (rejected as too long). Close with one line + linking the ready-to-apply artifacts (which live in section files, not here). No tables on + the one-pager. + **Every proposed change ships its artifact in its section file** (user feedback 2026-07-14 + - not only AGENTS.md diffs): AGENTS.md/CLAUDE.md edit → a real diff; new skill or subagent + → the full proposed file (frontmatter + body) in a code block, ready to save; move of an + existing artifact → concrete source → destination paths, flagging any machine-specific + content to review (if the source file lives on another machine, say so - name the move, + don't fake the file); tool/config change → the exact proposed text. + 4. **`## Data limitations`** - ~3 bullets: the basis (gateways / sessions / window), token + prizes are floors, capture is partial, plus any window-specific data-quality incident + a + `[caveats]` link. + 5. **`## Supporting analysis`** - a one-line footer linking **every** section file written this + run (not just the cited ones), so nothing is orphaned - e.g. `[basis](/basis.md) · [skill candidates](/skill-candidates.md) · [subagent candidates](/subagent-candidates.md) · [AGENTS.md edits](/agents-md-edits.md) · [efficiency → setup](/efficiency-setup.md) · [caveats](/caveats.md)`. - **Section files are analysis, not inventory.** Each detail section is its own `/.md`, held to the same standard as the one-pager: it argues one claim, opens with that takeaway, and ties every number to what it means for the reader. A section file that is only a stat table has failed - fold it back into the one-pager rather than shipping it as a page. Cut table narration ("how to read the table") and standing bookkeeping prose; compress source/window/method to a few lines. - **No scope apologies (in any file).** Scope rules (what routes to which report) are authoring guidance, never report copy. Don't write "descriptive only", "no recommendations here", or routing disclaimers in the one-pager or any section file; state findings plainly, and where a sibling report owns the action a plain cross-link is enough. diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-security-report/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-security-report/SKILL.md index 79a85d38..b6d8f4e1 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-security-report/SKILL.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-security-report/SKILL.md @@ -20,7 +20,8 @@ ask which one (or more) to audit, then proceed against the chosen source. **REDACT everything.** Never echo a secret, token, key, or credential value in the report, even one you found in `tool_args`; a raw secret appearing in args is itself a capture finding -(flag it, don't reproduce it). Query mechanics live in the **hypaware-query** skill; read it first. +(flag it, don't reproduce it). Query mechanics are step 0 of the procedure — a mandatory read, +not a reference. ## Risk classes (match over the FULL command text, not just the first token) | Class | Match signals | Why it matters | @@ -36,6 +37,22 @@ Risk is **amplified by autonomy**: weight anything that ran under `bypassPermiss than the same command in a gated session. ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). Route by shape, per hypaware-query's "when the graph + answers it cheaper" boundary: entity/connection questions go to the graph's tiny `node`/`edge` + tables — which sessions ran a tool/program, per-repo/client inventories, and gateway→person + attribution (`min/max(session_id)` per gateway from messages — an ID-only aggregate — then + look those session_ids up in graph Session nodes' `props.cwd`). This audit's core scans DO + read wide columns (`tool_args`, `content_text`) — that's their job — but keep them + aggregate-shaped regex matches (`regexp_extract` over `CAST(tool_args AS VARCHAR)`) over + **date-sliced, server-sized ranges**, never GROUP BY / DISTINCT / bulk row-fetches of raw + content columns. Capture stderr and check it even on success — truncation and server-cap + notices land there, and a clipped scan reads as "no findings". If a query fails, come back to + this step; don't iterate on the failing SQL. 1. **Coverage + autonomy baseline.** Window; coverage of `tool_args` on Bash calls and of `permission_mode` (bound the read if sparse); distinct `gateway_id` (the unit; `user_id` is ~always null). Total tool calls, % Bash, % conversations in `bypassPermissions`. State N; if @@ -51,26 +68,39 @@ than the same command in a gated session. ## Output - SAVE A SHORT MAIN FILE + ONE FILE PER SECTION A **progressively-disclosed one-pager** is the deliverable: a reader can stop after the thesis -sentence, glance the key-numbers table, skim the findings, or follow any link into a detail -**section**, and each section is its own markdown file. Build the one-pager top-to-bottom so each -layer is shorter and more glanceable than the one it summarizes. **All examples redacted in every file.** +sentence, skim the findings bullets, or follow any link into a detail **section**, and each +section is its own markdown file. Build the one-pager top-to-bottom so each layer is shorter and +more glanceable than the one it summarizes. **All examples redacted in every file.** + +**Report language (applies to every file; user feedback 2026-07-14 on the sibling reports).** The +reader may be a supervisor with no HypAware knowledge. Plain literal language: no pipeline +vocabulary, metaphors, or report-coined shorthand (write the literal behavior); coined names get +a one-line gloss at first mention per page; when a skill/tool/guardrail is named as a fix, add +one clause saying what it literally does; absolute dates only. Findings equip, never direct - +never "ask X" / "tell Y". Any bullet carrying several facts is a bold topic line + 2-3 short +sub-bullets; no multi-sentence prose bullets, no dense multi-clause lines. - **Main one-pager:** `hypaware-reports/-security-review.md` (create the dir if - needed). Dated so reports accumulate. Lay it out in exactly these blocks, separated by `---` rules: - 1. **Title + scope** - `# Security & Risk Review`, then a `## · ` subtitle. - 2. **Thesis** - ONE **bold** sentence: the posture - how autonomous the fleet is - (% `bypassPermissions`) and whether anything needs urgent attention. - 3. **`### Key numbers`** - a small metric→readout table, ~4-6 rows, each one glanceable fact with - no link and all redacted (% `bypassPermissions`, count of HIGH findings, the top risk class, - total tool calls + % Bash, coverage of `tool_args` / `permission_mode`). This is the ONLY - table on the one-pager. - 4. **`## What this shows`** - 3-5 numbered findings, highest-severity first; each is a - `### N. ` followed by 2-3 plain-language sentences with **bold** severities (redacted), - then a single `[
→](/.md)` link on its own line (autonomy posture, the top - risk findings, the single most important guardrail). - 5. **`## Caveat`** - the standing capture caveat in plain language + a `[capture caveats →]` link. - 6. **`## Report details`** - a one-line footer linking **every** section file written this run - (not just the cited ones), so nothing is orphaned - e.g. + needed). Dated so reports accumulate. It is a **terse bullet brief** with standard + business-report headings - **Key metrics / Key findings / Data limitations / Supporting + analysis** - never coinages like "What this shows" / "Key numbers" / "Report details". Blocks: + 1. **Title + scope** - an eyebrow line ` · `, then `# Security & Risk Review`. + 2. **Thesis** - one SHORT bold sentence: the posture - how autonomous the fleet is + (% `bypassPermissions`) and whether anything needs urgent attention. Supporting clauses go + in 2-3 plain sub-bullets under it, never packed into the sentence. + 3. **`## Key metrics`** - 4-6 **bold topic** bullets, each with 1-2 short sub-bullets carrying + the redacted numbers (% `bypassPermissions`, count of HIGH findings, the top risk class, + total tool calls + % Bash, coverage of `tool_args` / `permission_mode`). No tables on the + one-pager. + 4. **`## Key findings`** - 3-5 bullets, highest-severity first (autonomy posture, the top risk + findings, the single most important guardrail); each is a **bold headline** with **bold** + severity, then 2-3 short sub-bullets with the redacted evidence, the last ending with the + section link. Proposed guardrails ship ready to apply in their section file: settings/policy + → the exact config block; AGENTS.md rule → a real diff; redaction → the exact pattern. + 5. **`## Data limitations`** - ~2 bullets: the standing capture caveat in plain language + a + `[capture caveats]` link. + 6. **`## Supporting analysis`** - a one-line footer linking **every** section file written this + run (not just the cited ones), so nothing is orphaned - e.g. `[autonomy baseline](/autonomy-baseline.md) · [command profile](/command-profile.md) · [risk findings](/risk-findings.md) · [guardrails](/guardrails.md) · [trends](/trends.md) · [capture caveats](/capture-caveats.md)`. - **Section files are analysis, not inventory.** Each detail section is its own `/.md`, held to the same standard as the one-pager: it argues one claim, opens with that takeaway, and ties every number to what it means for the reader. A section file that is only a stat table has failed - fold it back into the one-pager rather than shipping it as a page. Cut table narration ("how to read the table") and standing bookkeeping prose; compress source/window/method to a few lines. - **No scope apologies (in any file).** Scope rules (what routes to which report) are authoring guidance, never report copy. Don't write "descriptive only", "no recommendations here", or routing disclaimers in the one-pager or any section file (redaction still applies); state findings plainly, and where a sibling report owns the action a plain cross-link is enough. diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-spend-report/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-spend-report/SKILL.md index 33599948..883ab16e 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-spend-report/SKILL.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-spend-report/SKILL.md @@ -1,6 +1,6 @@ --- name: hypaware-ai-spend-report -description: AI Spend Review for a HypAware server: where tokens go (by user/repo/model/gateway), what work they go on (graph-clustered), and how to spend less (waste scorecard + cost levers). Token volume, never dollars. Saves a dated report under hypaware-reports/; first asks which HypAware source to query (local logs or a remote server) via the hypaware-query skill. +description: "SUPERSEDED by hypaware-ai-usage-report (merged with the adoption profile, 2026-07-13) — invoke that skill instead unless the user explicitly asks for a standalone spend review. Original scope: AI Spend Review for a HypAware server: where tokens go (by user/repo/model/gateway), what work they go on (graph-clustered), and how to spend less (waste scorecard + cost levers). Token volume, never dollars. Saves a dated report under hypaware-reports/; first asks which HypAware source to query (local logs or a remote server) via the hypaware-query skill." --- # AI Spend Review @@ -19,8 +19,7 @@ list it once). Present the options, ask which one (or more) to run the review against, then proceed against the chosen source. **Tokens, never dollars.** Capture is partial, so stop at token volume and behavioral proxies. -Query mechanics (`--remote`, date-pruning the 30s timeout, the SQL dialect, sampling) live in -the **hypaware-query** skill; read it first. +Query mechanics are step 0 of the procedure — a mandatory read, not a reference. ## Token math (get this right; every breakdown reconciles to it) Usage is in `attributes.usage` (NOT `raw_frame`): `input_tokens`, `output_tokens`, @@ -46,6 +45,22 @@ WHERE date BETWEEN '' AND '' ``` ## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). Route by shape, per hypaware-query's "when the graph + answers it cheaper" boundary: entity/connection questions go to the graph's tiny `node`/`edge` + tables — sessions per repo/model/tool/file, skill and program rollups (graph-only facets), + client mix, work-type clustering by shared-file `touched` edges, and gateway→person + attribution (`min/max(session_id)` per gateway from messages — an ID-only aggregate — then + look those session_ids up in graph Session nodes' `props.cwd`). The **token spine stays on + `ai_gateway_messages`**: token sums, counts, ordering — date-sliced to server-sized ranges, + and **never GROUP BY / DISTINCT / row-fetch wide content columns (`cwd`, `content_text`) at + scale** (that query shape kills servers). Capture stderr and check it even on success — + truncation and server-cap notices land there. If a query fails, come back to this step; don't + iterate on the failing SQL. 1. **Coverage + footprint.** Window; usage coverage, `model`-column coverage (token-weighted), user/repo coverage; distinct gateways, sessions, claude/codex mix. `user_id` is ~always null → attribute by `gateway_id` (≈ one machine/user) and always show an explicit **unattributed** diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-usage-report/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-usage-report/SKILL.md new file mode 100644 index 00000000..06a938eb --- /dev/null +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-ai-usage-report/SKILL.md @@ -0,0 +1,227 @@ +--- +name: hypaware-ai-usage-report +description: Team AI Usage Review for a HypAware server, written for whoever supervises the team (eng manager, lead, CTO — no HypAware knowledge assumed): who's using AI and how much, what work it goes on and whether it pays off, which way it's trending, where tokens are wasted, and the levers to improve — the former adoption profile and spend review merged onto one token spine. Token volume, never dollars. Saves a dated report under hypaware-reports/; first asks which HypAware source to query (local logs or a remote server) via the hypaware-query skill. Supersedes hypaware-ai-adoption-report and hypaware-ai-spend-report — run this instead of either. +--- + +# Team AI Usage Review + +One report, one window, one token spine, answering the four questions a supervisor +actually has: + +1. **Who is using AI, how much, and on what?** — per-person utilization, and allocation by + person / repo / model. +2. **What does the work look like, and is the sophistication paying off?** — each person's + focus, recurring work-types sized by tokens, multi-agent fan-out and whether it earns + its cost, code that actually landed (GitHub reach, where enriched). +3. **Which way is it trending?** — weekly volume/output, deltas vs the last review. +4. **Where is it wasted, and what can I do about it this week?** — waste scorecard + + ranked levers, each with an estimated weekly token saving. + +The old adoption-profile and spend-review reports each told half of this and repeated the +other half; this report replaces both. Don't run those skills alongside it. + +## Audience contract (this is the point of the merge — enforce it everywhere) + +The reader is **the person supervising the team** — an engineering manager, team lead, +CTO, or ops owner. Assume they have never seen a HypAware table, don't know the schema +vocabulary, and are reading to make decisions about their team, not to audit queries. +Concretely: + +- **People, not infrastructure.** A `gateway_id` is ≈ one person's machine: attribute it + to a named person (via session `cwd`) and say the person. "phil's laptop", never + "gateway a3f9". Keep an explicit **unattributed** row when attribution fails. +- **Every number carries a judgment and a "so what".** Not "cache-read ratio 99.5%" but + what that means for them (context reuse is healthy / this is where the bill comes from). + Terms the report can't avoid (cache-read, subagent, output tokens) get a one-line + plain-language gloss the first time they appear — once, then use them freely. +- **Plain literal language, everywhere.** Pipeline vocabulary (ingest, sidechain, + provenance, projector, org-scoped, part) never appears in report copy — translate to + what it means for the numbers ("the logs record which tokens belong to helper + agents"). Coined names (a harness like "gastown") get a one-line gloss at first + mention on EACH page that uses them. Headings state the literal fact, never a + metaphor ("Message counts for 07-09 → 07-14 don't reflect real activity", not "The + volume meter broke"). This bans report-coined shorthand too: "marathon sessions" was + flagged by the user as vague (2026-07-14) — write the literal behavior, "sessions + kept open across days re-read their entire history each morning" / "the 60 longest + sessions". When a skill or tool is named as a fix, add one clause on what it + literally does ("`/handoff` writes a short end-of-day note of where the work stands + so the next day starts a fresh session"), never just its name. Time references are + absolute dates ("07-09 → 07-14", "wk of 07-13"), never relatives like "final week" + or "this week" that decay after the run date. +- **Equip the reader; never direct them.** The report's job is to surface what a + supervisor needs to act — who is involved, the size of the effect, what would change + it, and the expected result — stated as findings and sized options, NOT as + instructions ("ask X about Y", "talk to kenny", "set an expectation", "sanction Z"). + The reader knows how to manage their team; telling them how reads as presumptuous and + buries the information. Write "the 60 longest sessions are 32 phil / 24 kenny; a 25% + context trim returns ~290M/wk", never "ask phil and kenny to trim their sessions". + Deep tooling work (skills to build, harness changes) still routes to + **hypaware-ai-improvement-report** as a plain cross-link — surface it, don't spec it + here. +- **Comparisons over absolutes.** A supervisor can't judge "69.5M output tokens" in a + vacuum; they can judge shares, trends vs the last review, and spread across the team. + Lead with those. +- **Tokens, never dollars.** Capture is partial, so stop at token volume and behavioral + proxies; say so once in the caveat, not in every section. + +IMPORTANT: Don't assume which logs to read: **ask first.** Start by listing the data +sources and let the user choose which to query: **local logs** (this machine's own +recordings, `hyp query sql …`, no `--remote`) and **each remote HypAware server** (every +target from `hyp remote list`, plus any hypaware MCP server already available to you as +MCP tools (a `query_sql` / `graph_neighbors` tool in your toolset); the same server can +appear both ways, list it once). Present the options, ask which one (or more) to review, +then proceed against the chosen source. + +## Token math (get this right; every breakdown reconciles to it) + +Usage is in `attributes.usage` (NOT `raw_frame`): `input_tokens`, `output_tokens`, +`cache_read_tokens`, `cache_write_tokens` (+ `reasoning_tokens` for Codex). Usage rides +exactly one row per response (the last assistant part; non-carrier parts are null), so a +plain `SUM` over assistant rows is correct with no dedup (the one-carrier rule, LLP 0035); +`input_tokens` is net of cache, so it never double-counts. Report the four types +separately (cache-read is usually the bulk; output the scarce slice). + +```sql +SELECT + sum(CAST(JSON_EXTRACT(attributes,'$.usage.input_tokens') AS BIGINT)) t_in, + sum(CAST(JSON_EXTRACT(attributes,'$.usage.output_tokens') AS BIGINT)) t_out, + sum(CAST(JSON_EXTRACT(attributes,'$.usage.cache_write_tokens') AS BIGINT)) t_cw, + sum(CAST(JSON_EXTRACT(attributes,'$.usage.cache_read_tokens') AS BIGINT)) t_cr +FROM ai_gateway_messages +WHERE date BETWEEN '' AND '' + AND role='assistant' AND JSON_EXTRACT(attributes,'$.usage') IS NOT NULL; +-- One carrier row per response (LLP 0035): a plain SUM is correct, no dedup. +-- Slice by adding gateway_id / model / repo_root / date to SELECT + GROUP BY. +-- Defensive equivalent: max(...) GROUP BY session_id, message_id -- session_id is the +-- uniform key; conversation_id is null for Claude and only separates Codex threads. +``` + +## Procedure + +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user + picks a source and before any `hyp query sql`, read the **hypaware-query** skill + (invoke it or Read its SKILL.md), and the **hypaware-graph** skill if `hyp query + status` lists `node`/`edge` datasets. Memory notes from past runs do NOT substitute — + stale notes have cost real runs failed queries and server crashes (a phantom "100-row + output cap"; message-table `cwd` scans that 504'd then OOM'd the prod server). Route + by shape, per hypaware-query's "when the graph answers it cheaper" boundary: + - **Graph first (`node`/`edge` — tiny, join-safe) for every entity/connection + question:** which sessions used a repo/model/tool/file, skill and program rollups + (graph-only facets — SQL reconstructions disagree with the projection), client mix, + work-type clustering by shared-file `touched` edges, co-occurrence, and + gateway→person attribution (`min/max(session_id)` per gateway from messages — an + ID-only aggregate — then look those session_ids up in graph Session nodes' + `props.cwd`, `props.client_name`). + - **Messages (`ai_gateway_messages`) only for per-message measures:** token sums, + distinct part/session counts, timestamps and ordering, `is_sidechain`/`agent_id`, + content sampling. Slice long windows into server-sized date ranges. **Never GROUP + BY / DISTINCT / row-fetch wide content columns (`cwd`, `content_text`) on the + messages table at scale** — that query shape kills servers. Capture stderr and + check it even on success (truncation and server-cap notices land there). + If a query fails, come back to this step; don't iterate on the failing SQL. + +1. **Scope + coverage.** Window; distinct `gateway_id` (the unit; `user_id` is ~always + null, so never measure reach by it) mapped to named people; usage coverage, + `model`-column coverage (token-weighted), user/repo coverage; claude/codex mix; + subagent provenance (`agent_id` / `is_sidechain` / `parent_thread_id`, + transcript-enriched, may not survive ingest). Decide cost-capable vs volume-only and + which parallelism dimensions are real vs proxied (the `Task`-call proxy). State N; if + it's effectively one person / dogfood, say so. If usage is thin, fall back to + behavioral proxies (turns, tool calls, length), labeled as estimates. + **ALWAYS verify GitHub enrichment before deciding it's out of scope: probe, never + assume.** Run the probe every run: `hyp query sql "SELECT node_type, projector, + count(*) AS n, max(first_seen) AS newest FROM node GROUP BY node_type, projector" + --remote ` (and check `edge` exists via `hyp query status`). If a `github.t0` + projector with `PullRequest` / `Review` nodes is present, **GitHub reach is IN SCOPE + and MUST be computed in step 3**: record the node counts and max `first_seen` per type + as the graph's as-of date, and treat every reach figure as a floor. If the probe finds + nothing, state **"checked - no GitHub enrichment present"** explicitly. Never write + "not assessed" for reach — that phrasing means the probe was skipped. + +2. **Who, how much, and which way it's moving.** One row per person: volume (messages, + sessions, active days, first/last seen), tokens + cache-read ratio + `cache_read/(cache_read+input)`. Slice the token spine by person / repo / model + (→ `(unknown)` bucket) with shares; weekly trend with WoW deltas vs the last review; + top-spend outlier sessions. This one spine feeds every later section — reconcile, + don't re-derive. + +3. **What the work is, and whether it pays off.** Per person: top models, tools (Bash + dominance + top commands), repos, client, and 2–4 recurring work themes (sampled, + redacted), distilled into a one-line **focus label** a supervisor can repeat. Cluster + sessions into recurring **work-types** (shared-file overlap for code work, tool-set + signature for no-file work; context graph if projected, else SQL), sized by tokens. + Parallelism as a payoff question: % of sessions that fan out to subagents (incl. the + zero bucket), breadth/depth, true concurrency vs serial, main-loop-vs-subagent token + split, fan-out vs tokens-to-resolution — say plainly whether the sophisticated + pattern is earning its cost and who on the team has the habit worth spreading. When + step 1 found `github.t0` enrichment, add each person's real *reach*: repos and PRs + their AI-assisted work landed in (`Session -at-> Commit <-references- PullRequest`) + and whether it drew review (`… PullRequest <-on- Review <-submitted- Actor`), dated to + the graph's freshness. This is the "did the tokens become shipped code" evidence a + supervisor most wants and the messages cannot show — not optional when the graph + supports it. + +4. **Where to save, and the information to act.** Score the waste: cache-read ratio + (the biggest lever, feature it), long-session concentration (sessions kept open + across days), retry loops, + abandoned costly sessions, model over-spec, context bloat. Turn the top few into + **ranked levers, each with an estimated weekly token saving and the facts a + supervisor needs to act on it** — who is involved, what behavior or default drives + it, and what a stated change would return. Per the audience contract: findings and + sized options, never instructions to the manager. Hand expensive recurring + work-types to **hypaware-ai-improvement-report** as packaging candidates (surface + them; don't build them here). + +## Output - SAVE A SHORT MAIN FILE + ONE FILE PER SECTION + +A **short bullet brief** is the main deliverable (~40 lines of content): a reader gets +the whole story from scannable bullets, and every detail lives in a linked section file. +Headings are standard business-report vocabulary — never AI-flavored coinages like "The +numbers", "What this shows", or "Where the leverage is". + +- **Main brief:** `hypaware-reports/-usage-review.md` (create the dir if + needed). Dated so reviews accumulate. Lay it out in exactly these blocks: + 1. **Title + scope** - an eyebrow line ` · `, then + `# Team AI Usage Review`. + 2. **Headline** - ONE short **bold** sentence a supervisor could repeat in a meeting: + the trend, the biggest concentration, the top leverage point. Facts, not + instructions. + 3. **`## Key metrics`** - grouped bullets, each a **bold topic line + 2-3 short + sub-bullets** (topics ≈ Volume / Who / Trend / The work / Fan-out / Health): + glanceable facts with bold numbers, no prose sentences. Each topic line ends with + ` · [
](/.md)` linking its detail section. + 4. **`## Key findings`** - 3-5 ranked findings as the same bold-topic + sub-bullets + shape: each names the finding, who is involved, the driver, and the estimated + weekly token effect where one exists, with the topic line linking its detail + section like Key metrics. This is data reporting, not consulting — sized facts, + never instructions to the manager (audience contract) and never pitch-flavored + headings ("Opportunities", "Recommendations"). + 5. **`## Data limitations`** - 2-3 bullets: the caveats that most change how to read + the report (tokens-never-dollars + partial capture; whether subagent identity + survived ingest; any capture anomalies this window). + 6. **`## Supporting analysis`** - a one-line footer linking **every** section file + written this run (not just the cited ones), so nothing is orphaned - e.g. + `[scope & coverage](/scope-coverage.md) · [team usage](/team-usage.md) · [trends](/trends.md) · [focus & reach](/focus-and-reach.md) · [work-types](/work-types.md) · [parallelism payoff](/parallelism-payoff.md) · [waste scorecard](/waste-scorecard.md) · [levers](/levers.md) · [caveats](/caveats.md)`. +- **Chart the who-breakdowns.** Keep the one-row-per-person tables as the record, and + pair each with a breakdown chart following the HTML renderer's authoring contract + (`hypaware-report-to-html/authoring.md`; component snippets in `components.md` next to + it): share of messages and tokens by person on the team-usage page, main-vs-subagent + token split on the parallelism page, waste levers as a ranked bar set. Where a real + team grouping exists (a user-supplied mapping, or cwd naming), add a by-team rollup; + never invent teams the data doesn't show. +- **Section files are analysis, not inventory.** Each detail section is its own + `/.md`, held to the same standard as the main brief: it argues one claim, + opens with a SHORT bold thrust line (a few clauses, not a paragraph — optionally + followed by 2-4 bullets), and ties every number to what it means for the reader. + Body lists use the same bold-topic + short-sub-bullets shape as the main brief; + multi-sentence prose bullets are hard to scan and not allowed. A + section file that is only a stat table has failed - fold it back into the main brief + rather than shipping it as a page. Cut table narration and standing bookkeeping prose; + compress source/window/method to a few lines. +- **No scope apologies (in any file).** Scope rules (what routes to which report) are + authoring guidance, never report copy. Don't write "descriptive only" or routing + disclaimers; state findings plainly, and where a sibling report owns the action a plain + cross-link is enough. +- **Capture-health note:** if subagent provenance doesn't reach the server, the standing + #1 caveat is "subagent identity must survive ingest"; run fan-out adoption off the + sub-agent-invocation proxy (the tool calls that spawn subagents) and flag it. diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-graph/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-graph/SKILL.md new file mode 100644 index 00000000..e15da7b1 --- /dev/null +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-graph/SKILL.md @@ -0,0 +1,170 @@ +--- +name: hypaware-graph +description: Explore the HypAware context graph: the activity graph projected from ai_gateway_messages (Sessions, Apps, Models, Tools, Files, Skills, Programs, and git Repos/Commits, and how they connect), plus optional server-side GitHub enrichment (Repos, PullRequests, Commits, Reviewers) that bridges AI sessions to code review. Use when the user asks what connects to a file/session/tool/model, which sessions ran a skill or invoked a program, wants co-occurrence or N-hop traversal, wants to join AI sessions to GitHub repos/PRs/reviewers, or wants to build/refresh the graph. Covers `hyp graph project` and `hyp graph neighbors`. +--- + +# HypAware Graph + +`hyp graph` turns HypAware recordings into a queryable **activity graph** and walks it. The graph is a derived projection of the `ai_gateway_messages` dataset (the same data the `hypaware-query` skill reads), read as *relationships* instead of rows. + +## Availability: is the graph enabled here? + +If `hyp graph` comes back as an unknown command, or `node`/`edge` are missing from the datasets in `hyp query status`, the graph plugins are not in the active config. They ship bundled with the package but activate only when the config names them, and on fleet-joined hosts the plugin set comes from the central config layer, which may omit them. + +The fix is additive and works even on fleet-locked installs (the central layer locks only the plugin names it declares; the local layer may contribute the rest): add `{"name": "@hypaware/context-graph"}` and `{"name": "@hypaware/ai-gateway-graph"}` to the `plugins` array of the local `/hypaware-config.json`, verify with `hyp config validate`, then build the graph with `hyp graph project`. **This edits the user's config - propose the change and get their go-ahead rather than editing it silently.** If the central config later adds the same plugins, the local duplicates are dropped benignly (recorded as `collides_with_central`, visible in `hyp status`). + +A fleet server may have the graph enabled even when this machine does not (or vice versa): `hyp graph neighbors ... --remote ` and `hyp query sql "... from node ..." --remote ` use the server's graph and the server's projection schedule, not the local ones. Local and remote graphs answer at different scopes (this machine vs the fleet) and can legitimately disagree; treat the difference as coverage, not error. + +## Build or refresh the graph + +The projection runs **on demand**; it does not auto-update. Run it before querying, and again after new sessions are recorded: + +```bash +hyp graph project # project ai_gateway_messages -> node/edge tables (idempotent) +hyp graph project --dry-run # show what would be written +hyp graph compact # merge duplicate rows, rewrite partitions sorted (maintenance) +``` + +`hyp graph project` prints `N node(s), M edge(s) - wrote ...`. If `node`/`edge` come back empty, the projection has not been run yet (or there are no recordings). For recency-sensitive questions, check `max(first_seen)` in `node`, or just re-project first: it is idempotent and cheap. + +## The graph model + +Deterministic T0 projection: exact-key, no models. The core is **Session-rooted**, plus a small git-provenance sub-web (`Repo`, `Commit`) that doubles as the bridge to GitHub enrichment (see below). + +- **Node types:** `Session` (one per session), `App` (client), `Model`, `Tool`, `File`, `Skill`, `Program`, and, when the session's git context is captured, `Repo` and `Commit`. +- **Edge types:** `via` (Session->App), `used_model` (Session->Model), `used` (Session->Tool), `touched` (Session->File), `ran` (Session->Skill), `invoked` (Session->Program), `in` (Session->Repo), `at` (Session->Commit, the HEAD the session sat on), and `in` (Commit->Repo). +- **natural_key** is the human key per type: Session = `session_id` (the always-present session container; `conversation_id` is a nullable thread identity, null for Claude), App = `client_name`, Model = model id, Tool = tool name, File = full path (or `owner/repo:relpath` when the repo remote is known; its `label` is the basename), Skill = bare skill name, Program = lowercased basename, Repo = `owner/repo`, Commit = full 40-hex sha. Every row carries inline provenance (`source_dataset`, `projector`, ...). +- **Skills and programs are derived facets.** They have no column in `ai_gateway_messages`: `ran` edges come from multi-surface skill-activation detection (and carry `dispatch_*` boolean props saying how the skill was activated), `invoked` edges from argv[0] extraction with wrapper unwrapping. Ad hoc reconstruction from raw rows measurably disagrees with the canonical derivation (a 3-surface LIKE approximation returned 52 sessions where the strict rules give 44; a first-token approximation of "programs" returned 470 garbage tokens against the graph's 86 clean ones). Always answer skill/program questions from the graph. +- **Keys converge where raw spellings diverge.** Skill and Program nodes are keyed identically across claude and codex, so per-skill and per-program questions span both clients for free. Repo nodes normalize remote-URL forms that a raw `git_remote LIKE` misses (in one measurement the graph found 312 sessions in a repo where the LIKE found 240). +- **Bridge-ready keys.** `Repo`, `Commit`, and `File` use shared, content-addressed natural keys, so a node minted from a session's git context and the same node minted by the GitHub source converge on one id. This is what lets an AI session and a pull request meet at the same commit. + +## GitHub enrichment (server-side) + +The base graph above comes from `ai_gateway_messages` and is available anywhere, including a local install. A **server** can additionally run the `@hypaware/github` source, which captures repo / commit / PR / issue / review events and projects a second contract into the **same** `node` / `edge` tables. This is the graph's biggest payoff, and none of it is answerable from the message data alone. + +**Caveats first, so you don't query nodes that aren't there:** + +- **Server-only and opt-in.** GitHub nodes exist only on a host where the `@hypaware/github` source is configured and has captured events (normally the central server, reached with `--remote`). A plain local projection has none of them. An empty GitHub query usually means the source is not configured on that host, or the graph has not been re-projected since capture, not that the true answer is zero. +- **`Actor` is a GitHub login, not the AI user.** The identity that authored a commit or opened a PR is the git actor, not the `user_id` of whoever ran the agent. Cross-domain identity merge is later work (T1/T2); never equate an `Actor` with an AI operator. +- **Freshness applies here too.** The GitHub domain is only as current as the last projection on that host, and you cannot project through the read-only query token (projection is admin-side). On a stale central graph, recent PRs and reviews are simply missing. One subtlety: freshly projected rows sit in a spool until a settling read runs **on the server** (an admin-side query; the remote query surface never settles), and `node` and `edge` settle independently, so a graph can briefly show fresh nodes joined by stale edges. If a cross-domain join returns implausibly few rows against fresh-looking nodes, suspect an unsettled `edge` dataset before doubting the data. + +**What it adds:** + +- **Nodes:** `Actor` (login), `Issue`, `PullRequest`, `Review`, plus enriched `Repo` / `Commit` / `File`. +- **Edges:** `authored` (Actor->Commit), `opened` and `commented` (Actor->Issue | PullRequest), `submitted` (Actor->Review), `on` (Review->PullRequest), `references` (PullRequest->Commit), `touched` (Commit->File and PullRequest->File), `in` (Commit | File | Issue | PullRequest->Repo). + +**Why it matters, the possibility.** Because `Repo`, `Commit`, and `File` are bridge-ready, the AI-session web and the GitHub web are *one graph*. The commit a session sat on (`Session -at-> Commit`) is the same node GitHub knows through `PullRequest -references-> Commit` and `Actor -authored-> Commit`. So you can walk from an agent's activity into the code-review reality around it, questions `ai_gateway_messages` cannot express: + +- **AI work to the PR that shipped it:** `Session -at-> Commit <-references- PullRequest`. +- **AI work to who reviewed it:** continue `PullRequest <-on- Review <-submitted- Actor`. +- **Coverage, honestly:** which repos and PRs an agent's work actually reached, not just which cwd it ran in. +- **Reverse:** start from a `PullRequest` or `Repo` and walk inbound to every AI session that touched it. + +```bash +# Sessions whose HEAD commit is referenced by a PR (AI work that reached code review). +# Note: no --refresh with --remote; the server owns its freshness. +hyp query sql "select distinct s.natural_key session + from edge a join node s on a.src_id = s.node_id + join edge r on r.dst_id = a.dst_id and r.edge_type = 'references' + where a.edge_type = 'at'" --remote HYP_CENTRAL + +# From a PR, walk out to its reviews, actors, and referenced commits. +hyp graph neighbors owner/repo#123 --type PullRequest --depth 2 --direction both --remote HYP_CENTRAL +``` + +Reach for the GitHub domain whenever a question spans **both** AI activity and code collaboration (sessions to PRs, agents to reviewers, work to repos). If it is purely one side, the base graph or plain message SQL is enough. + +## Two ways to query + +### 1. SQL over `node` / `edge`: counts, filters, aggregates + +Flat SQL through the normal query surface (no recursion). Use for "how many", "top N", "group by", one- or two-hop joins: + +```bash +hyp query sql "select node_type, count(*) n from node group by node_type" --refresh always +hyp query sql "select t.natural_key tool, count(*) n from edge e join node t on e.dst_id = t.node_id + where e.edge_type = 'used' group by tool order by n desc" --refresh always +``` + +`node`/`edge` are ordinary datasets; see the `hypaware-query` skill for `hyp query` mechanics (read stderr, freshness, `--format json`). Use `--refresh always` so the read settles freshly-projected rows. + +### 2. Traversal with `hyp graph neighbors`: relationships and depth + +For "what connects to X", co-occurrence, and N-hop walks, what flat SQL can't express. + +```bash +hyp graph neighbors [--depth N] [--type T] [--edge-type T] [--direction out|in|both] [--limit N] [--json] +``` + +- **``** resolves by `node_id`, then `natural_key`, then `label`, so pass a session id, a file path (or basename), a model id, or a tool/skill/program/app name. If it's ambiguous the command lists the candidates; narrow with `--type Session|App|Model|Tool|File|Skill|Program|Repo|Commit|Actor|Issue|PullRequest|Review` (the last four require GitHub enrichment). +- **`--direction`** is load-bearing because edges are Session-rooted: + - `out` from a **Session** -> its app, model, tools, files, skills, programs. + - `in` from a **File / Tool / Skill / Program / Model / App** -> the **Sessions** that touched/used/ran/invoked it. + - `both` at `--depth 2` from a **File** -> **co-occurrence**: the other files/tools/models reached through the sessions that share it. +- **`--edge-type`** restricts which relations are walked (e.g. `--edge-type used`); repeatable or comma-separated. +- **`--limit`** caps output in BFS order and reports the true total when it truncates (`... - truncated; raise --limit`). +- **`--json`** emits `{ seed, neighbors: [{ hop, edge_type, direction, node }], reachable, truncated }` for follow-up reasoning, with **full node ids**. Note the flag is `--json`, not `--format json` (the latter is silently ignored and you get the text table). The bracketed id in the text output (`[c1446c4f2b01]`) is display-truncated - pasting it into SQL matches nothing; take full ids from `--json`, or resolve them from the `node` table by `natural_key`. + +Examples: + +```bash +hyp graph neighbors sess-abc123 --direction out # what a session used / touched / ran +hyp graph neighbors src/auth.py --direction in # which sessions touched a file +hyp graph neighbors src/auth.py --depth 2 --direction both # files/tools that co-occur with it +hyp graph neighbors Bash --direction in --edge-type used # sessions that ran a tool +hyp graph neighbors dataviz --type Skill --direction in # sessions that ran a skill +hyp graph neighbors aws --type Program --direction in # sessions that invoked a program +hyp graph neighbors claude-opus-4-8 --type Model --direction in # sessions that used a model +``` + +**File-node identity is split**: the same physical file can exist as both a repo-scoped node (`owner/repo:src/x.js`) and one or more absolute-path nodes (worktree and tmp-dir copies included). For a complete "who touched this file" answer, enumerate the keys first, then walk each: + +```bash +hyp query sql "select node_id, natural_key from node where node_type='File' and natural_key like '%src/core/runtime/bundled.js'" --format json +hyp graph neighbors --direction in +``` + +## Choosing the right tool + +Ask: does answering require *reading* rows, or only knowing they *exist and connect*? Route to the graph when the question is any of: (1) the answer is a set of identifiers, not text (membership/reachability); (2) the predicate is derived, not stored (skills, programs - see the model section); (3) it crosses two or more relationships (co-occurrence, indirect association) - the graph pre-materialized the join the raw route would express as a brittle correlated subquery; (4) it is an inventory/existence question - the node table is a pre-computed DISTINCT over all history; (5) identity needs normalizing across raw spellings (repos, cross-client skills). + +Then pick the surface: + +- Counting, ranking, grouping, "how often" -> **`hyp query sql`** over `node`/`edge`. Distinct-session counts key on the edge (`count(distinct src_id)`), far fewer rows than `count(distinct session_id)` over messages (benchmarked ~12x fewer for the repo rollup): sessions per tool = `used`, per model = `used_model`, per file = `touched`, per skill = `ran`, per program = `invoked`, per app = `via`, per repo = `in`, per commit = `at`. +- "What connects to X", paths, neighborhoods, co-occurrence, depth -> **`hyp graph neighbors`**. + +Stay on `ai_gateway_messages` when the measure lives on the message, not the relationship: + +- token sums and cache-read ratios; `count(*)` call totals (the graph keeps one edge per (session, entity) pair - an edge means "at least once", never a count); `is_error` / `is_sidechain` / stop-reason; ordering and time inside a session; and `content_text` classification. +- per-`gateway_id` or per-`user_id` rollups: there are no Gateway or User nodes yet. + +**Default strategy is two-stage**: the graph decides WHICH sessions or entities matter, then raw SQL reads WHAT happened inside them - a `session_id`-scoped messages query is as fast as the graph (~0.15s), while an unscoped one grows with history. The join is direct: a `Session` node's `natural_key` IS the `session_id` column in `ai_gateway_messages`. + +```bash +# 1. Which sessions used the tool (take full ids and session UUIDs from --json) +hyp graph neighbors --type Tool --direction in --json +# 2. The calls themselves, with args +hyp query sql "select message_index, tool_call_id, json_extract(tool_args,'\$.') a + from ai_gateway_messages where session_id='' and part_type='tool_call' and tool_name=''" --format json +# 3. Each call's result: tool_result rows point back via tool_result_for +hyp query sql "select content_text from ai_gateway_messages + where session_id='' and tool_result_for=''" --format json +# 4. Surrounding conversation: window on message_index +hyp query sql "select message_index, role, part_type, content_text from ai_gateway_messages + where session_id='' and message_index between and order by message_index, part_index" --format json +``` + +Coverage of graph and messages is near-identical when the graph is freshly projected, but they can drift (the graph only updates on `hyp graph project`; message rows can be pruned by retention), so treat an empty drill-down as "check freshness", not "no data". + +## SQL performance over node/edge + +SQL over `node`/`edge` has sharp performance tiers (measured): `graph neighbors` traversal ~0.2s; an edge self-join anchored on a **literal node_id** ~3s; the same join with a scalar subquery (`e1.dst_id = (select node_id from node where ...)`) ~33s. Resolve seed node_ids first (via `--json` or a separate lookup query) and inline them as literals. Use SQL only when you need per-edge weights (`count(distinct e.src_id)`) that `neighbors` (a deduplicating BFS) cannot report. + +The join planner has intermittently failed non-trivial edge self-joins with `Column ... not found`. If that happens, keep the edge self-join adjacent and early, or materialize it as a subquery and join `node` in the outer query; resolve a second node lookup in a separate query. + +## Guardrails + +- **Project first.** The graph is only as fresh as the last `hyp graph project`; empty results usually mean it has not run. +- **Read stderr for errors.** `graph neighbors` writes not-found / ambiguity notes (and the large-graph note) to stderr; exit `1` is a resolution error, `2` is a usage error. **Truncation is part of the result**, so it goes to **stdout** (`... - truncated; raise --limit`) and the `--json` `truncated` field, not stderr. +- The graph is **derived and rebuildable**, never the source of truth. To change what it contains, fix capture/projection and re-project; don't hand-edit `node`/`edge`. +- Basic traversal loads the graph in memory per call, fine at activity-graph scale; a very large graph prints a note pointing at the future indexed path. diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-query/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-query/SKILL.md index e2350183..f27da653 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-query/SKILL.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-query/SKILL.md @@ -58,8 +58,16 @@ Never read a smaller CLI row count as "fewer rows matched" — it is the display ## SQL dialect notes -- `json_extract_scalar()` does not exist. `JSON_EXTRACT` does, but it errors on rows where a JSON-typed column (notably `tool_args`) holds a plain string instead of a JSON object ("first argument must be JSON string or object, got string"). +The engine is SELECT-only with a deliberately small SQL surface. Every bullet below is a rejection observed in recorded sessions; when a query fails, the error message echoes the available columns, so read it before retrying. + +- SELECT-only: `SHOW`, `DESCRIBE`, DDL, and `information_schema` are parse errors. Discover a table's columns with `hyp query schema
` or `SELECT * FROM
LIMIT 1`, never introspection statements. Dataset names come from `hyp query status` (on a standard install: `ai_gateway_messages`, `node`, `edge`); never guess a table name. +- Boolean predicates: `IS NOT TRUE` / `IS TRUE` are not parsed (`NOT` must be followed by `NULL`). Compare directly: `col = true`, `col = false`, or `col IS NULL`. +- Cast types are only STRING, INT, BIGINT, FLOAT, BOOL. `TRY_CAST`, `CAST(... AS TIMESTAMP)`, and `TIMESTAMP '...'` literals do not exist. Filter time ranges on the STRING `date` column (`date >= 'YYYY-MM-DD'`); the event-time column is `message_created_at` (there is no `timestamp` column). +- `ANY_VALUE` does not exist: use `MAX`/`MIN`. `regexp_like` does not exist: use `REGEXP_MATCHES` for a boolean match, `REGEXP_SUBSTR` to extract, or plain `LIKE`. `LIKE ... ESCAPE` is not parsed. +- Regexp position arguments are 1-based: `regexp_extract(str, pattern, 1)`, never `0`. +- `json_extract_scalar()` does not exist. `JSON_EXTRACT` does, but it errors on rows where a JSON-typed column (notably `tool_args`) holds a plain string instead of a JSON object ("first argument must be JSON string or object, got string"). Dotted identifiers (`usage.output_tokens`) are not columns; extract JSON fields explicitly. - The robust pattern for extracting fields from `tool_args` is a regex over the raw text, e.g. `regexp_extract(CAST(tool_args AS VARCHAR), '"command":"([^"]+)', 1)`. +- Never invent a `--remote` target name: discover configured targets with `hyp remote list` first. ## AI gateway message model diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/SKILL.md index f5b31e40..23435b4e 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/SKILL.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/SKILL.md @@ -15,8 +15,9 @@ Two moving parts: 1. **`build.sh`** (in the repo, pandoc-based) converts each `.md` — plus any `/` sections — into a self-contained `html//` folder: `index.html` for the - one-pager, one `
.html` per section with a "← Back to the report" nav, - `assets/style.css`, and a `.nojekyll`. It rewrites inter-file `.md` links to `.html` + one-pager with a "← All reports" nav back to the top-level landing page + (`../../index.html`), one `
.html` per section with a "← Back to the report" + nav, `assets/style.css`, and a `.nojekyll`. It rewrites inter-file `.md` links to `.html` on the **emitted HTML** (`href` attributes), which catches Markdown-syntax links and links inside raw-HTML components (`rec` cards, callouts) in one pass: it flattens the one-pager's `/section.md` links to `section.html` and maps cross-report links @@ -66,17 +67,20 @@ Work relative to the repo root `~/hypaware-reports`. system; do not hand-tune per-page CSS. 3. **Enrich the report Markdown (the step that makes it a data report).** For each - top-level `.md`, check whether it already uses the component vocabulary — - **both** halves of it, because partial enrichment is the common failure: + top-level `.md`, check whether it already uses the component vocabulary: ```bash - grep -L 'class="metric-grid"' *.md # reports with no headline metric strip at all - grep -L 'class="rec"' *.md # reports whose findings are still prose-only + grep -L 'class="rec"' *.md # reports whose findings/changes are still prose-only + grep -L 'class="metric-grid"' *.md # reports with no headline metric strip (see below) ``` - A one-pager that has a `metric-grid` but no `rec` cards is **half-done, not done**: - its findings sit as `###` heading + paragraph + trailing link, so every number is - buried in a sentence and the reader has to open the section pages to see anything at - a glance. Finish it — convert the findings to cards (and rename any scaffolding - headings) rather than skipping it. + `rec` cards are required on every one-pager (findings or proposed changes). A + `metric-grid` is required **only where the source has a headline-numbers section** + (the usage and security reviews' "Key metrics"). The improvement review deliberately + has none — it opens with a numbered **Proposed changes** list (user-approved shape, + 2026-07-14); each numbered change becomes one `rec` card (bold what = card title, + why-sentence = body, evidence numbers = stat row, section link = the card) and **no + metric strip is added above or instead of the change list**. A one-pager with a + metric-grid but no `rec` cards is **half-done, not done** — finish the cards rather + than skipping it. For every report needing work, proceed in **two phases — inventory first, markup second**: @@ -94,15 +98,25 @@ Work relative to the repo root `~/hypaware-reports`. existing structure. Apply the recipe in [`authoring.md`](authoring.md) using ONLY the Phase A inventory, with [`example-enrichment.md`](example-enrichment.md) as a *shape* reference — and take a designer's liberties: - - **Restructure freely.** Reorder sections for narrative strength, merge or retitle - weak headings, delete decorative `---` rules and boilerplate scaffolding - ("Key numbers", "What this shows" → headings that say something). The document's - source order is not sacred; its facts are. + - **Restructure within the approved skeleton.** Merge or retitle weak headings inside + sections, delete decorative `---` rules — but the one-pager's top-level order is + **user-approved structure, not scaffolding** (2026-07-14): proposed changes / findings + lead, limitations and links follow. Never move a metric strip above a change list, + never split the evidence back out into a separate findings section, and keep the + standard heading vocabulary (Proposed changes / Key metrics / Key findings / Data + limitations / Supporting analysis) — retitle only headings that aren't part of that + skeleton. - **Rewrite for the surface.** Metric labels, card titles, stat labels, tag words, chart titles, and notes are *display copy* — write them fresh (2–4 word labels, one - punchy "so what" note), never paste sentence fragments from the prose. Body - paragraphs — the analysis itself — stay intact apart from trims where a visual now - carries the point. + plain "so what" note), never paste sentence fragments from the prose. Display copy + obeys the report-language rules: literal words, no metaphors or coined shorthand + (write "sessions open across days", never compress to a coinage like "marathon + sessions"), no pipeline vocabulary, absolute dates. Body paragraphs — the analysis + itself — stay intact apart from trims where a visual now carries the point. + - **Ready-to-apply artifacts are verbatim.** Proposed diffs, full skill/subagent file + drafts, tool-description text, and source→destination move tables render as the + code blocks / tables they are — never trimmed, carded, summarized, or reworded. They + are the deliverable, not display copy. - **Give every headline number the big treatment.** Any number the report leads with belongs in a `metric`, `gauge`, `rec-stat`, or chart — large, colored by judgment, with a note — not bolded inline in a sentence. After the pass, a number that matters @@ -149,6 +163,25 @@ Work relative to the repo root `~/hypaware-reports`. It prints `Built html/ : N report(s) …`. `html/` is wiped and rebuilt, so deleted or renamed reports never leave stale HTML behind. + **Every report page must offer a way back to the landing page.** `build.sh` is + responsible for this: it prepends a `topnav` to each one-pager before pandoc runs. If + the repo's `build.sh` predates this (no `All reports` string in it — + `grep -q 'All reports' build.sh`), add the injection where the one-pager is built, then + re-run it: + ```bash + { + printf '\n\n' + cat "$src" + } | pandoc -f gfm -t html5 -s \ + --css assets/style.css \ + --metadata pagetitle="$(page_title "$src" "$slug")" \ + -o "$out/index.html" + ``` + (i.e. pipe the nav + source into pandoc instead of passing `"$src"` as the input + file.) Section pages already chain back: "← Back to the report" → one-pager → + "← All reports" → landing page. The nav goes in **build.sh, not the source `.md`** — + the Markdown must stay renderer-agnostic. + 5. **Regenerate the top-level `index.html` as an at-a-glance dashboard, not a table of contents.** Write a fresh landing page (template in [`components.md`](components.md) → *Landing-page template*). One card per built report, newest first (slugs are @@ -158,11 +191,13 @@ Work relative to the repo root `~/hypaware-reports`. - **Kicker** - the report's scope line (the italic `*Source: … · Window: … *` line for adoption profiles, or the `## · ` subtitle for the others), trimmed to a short phrase, as the card's `rec-kind` eyebrow. - - **Stats** - the report's top 3-4 `metric-grid` tiles (step 3 guarantees they exist), - re-rendered as `rec-stat`s on the card: same values, same crit/warn/good judgments, - labels compressed to 2-4 words, notes dropped. Rules in components.md. This hoists - each report's key results and progress onto the landing page, so a reader gets the - fleet's state without opening a report. + - **Stats** - the report's top 3-4 headline numbers as `rec-stat`s on the card: from + its `metric-grid` tiles where it has one, otherwise (change-list reports like the + improvement review) from the `rec` cards' stat rows — same values, same + crit/warn/good judgments, labels compressed to 2-4 plain words (no coined + shorthand), notes dropped. Rules in components.md. This hoists each report's key + results and progress onto the landing page, so a reader gets the fleet's state + without opening a report. Link each report by its explicit `html//index.html`, **not** a bare `html//` directory URL. A trailing-slash directory link relies on server-side @@ -177,16 +212,17 @@ Work relative to the repo root `~/hypaware-reports`. ls html/ # one dir per report grep -o '[^<]*' html/*/index.html grep -rlo 'href="[^"]*\.md"' html/ || echo "no leftover .md links ✓" - grep -L 'metric-grid' html/*/index.html # should print nothing - grep -L 'class="rec"' html/*/index.html # should print nothing: findings are carded + grep -L 'class="rec"' html/*/index.html # should print nothing: findings/changes are carded grep -c 'rec-stat' index.html # ≥ number of reports: landing cards carry stats + grep -L 'All reports' html/*/index.html # should print nothing: every report links back ``` `href="….md"` in any built page means a link wasn't rewritten — remember links live both in Markdown syntax **and** inside raw-HTML components (`rec` card and callout `href`s), and may point across reports; investigate before publishing. A page missing - `metric-grid` means step 3 was skipped for that report; one missing `rec` cards means - step 3 stopped halfway (headline strip done, findings left as prose); a landing page - without `rec-stat`s means step 5 produced a bare link list. + `rec` cards means step 3 was skipped or stopped halfway; a `metric-grid` is only + expected where the source report has a headline-numbers section (do NOT add one to a + change-list report to satisfy a check); a landing page without `rec-stat`s means + step 5 produced a bare link list. Optionally open `index.html` (or `html//index.html`) in a browser to eyeball it (check both light and dark — the stylesheet supports both). diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/authoring.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/authoring.md index 80b8ec4c..f9a5230d 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/authoring.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/authoring.md @@ -44,12 +44,16 @@ failures … and let the whole team run a flow only phil has.** The scope/date line becomes an `eyebrow` above the title. The thesis stays one bold paragraph — the stylesheet turns it into the hero box. -## 2. "Key numbers" — metric grid, not a table +## 2. Headline numbers — metric grid, not a table (only where the report has them) -Every report's headline numbers section (the 2-column Metric/Readout table) becomes a -`metric-grid` of 3–6 cards. Color carries judgment: `is-crit` = problem, `is-warn` = -exposure, `is-good` = healthy/solved, no class = neutral. Keep any *detail* tables that -follow; only the headline strip converts. +A report's headline-numbers section (the usage and security reviews' **Key metrics**) +becomes a `metric-grid` of 3–6 cards. Color carries judgment: `is-crit` = problem, +`is-warn` = exposure, `is-good` = healthy/solved, no class = neutral. Keep any *detail* +tables that follow; only the headline strip converts. + +**Not every report has one.** The improvement review opens with a numbered **Proposed +changes** list and has no metrics section by design (user-approved, 2026-07-14) — do NOT +add a metric strip to it; its changes become `rec` cards (§3) and lead the page. ```markdown

The numbers that set the agenda

@@ -71,16 +75,25 @@ follow; only the headline strip converts. Every card needs a `note` that says why the number matters — a bare number is not a finding. -## 3. Findings index — rec cards, not `###` + link +## 3. Findings / proposed changes — rec cards, not `###` + link -On the one-pager, each finding/recommendation that links to a section page becomes an +On the one-pager, each finding or proposed change that links to a section page becomes an `` card (number badge, kind eyebrow, title, 1–2 sentence body, 2–3 stat row, go-link). See the full snippet in `components.md`. The `###` heading + trailing `section →` pattern is replaced by the card — don't emit both. +For the improvement review's numbered change list, the mapping is fixed: bold imperative += card title, the why-sentence = body, the evidence numbers = stat row, the section link += the card itself; card order = list order (highest leverage first — never resequenced). + Stat-row discipline: 2–3 stats per card, each `valuelabel`, color class only when it carries judgment. +**Ready-to-apply artifacts stay verbatim.** Proposed AGENTS.md diffs, full skill/subagent +file drafts, tool-description text, and source→destination move tables are deliverables, +not display copy: render them as the code blocks / tables they are, never trimmed, +carded, or reworded. + ## 4. Section pages: every claim gets a visual, breakdowns get charts Each section page opens with its own bold thesis directly under its `# ` title (hero @@ -125,9 +138,14 @@ literal): - **Tag words** — a single judgment noun: Exposure, Risk, Solved, Caveat, Basis. - **Chart titles** — name the axis and scope ("Edit-tool errors by message · 30 days"); **chart-foot** — the takeaway, one line. -- **Section headings** — replace scaffolding ("Key numbers", "What this shows") with - headings that carry the message ("The numbers that set the agenda", "The four - recommendations"). +- **Language rules bind display copy too** (user feedback 2026-07-14): literal words + only — no metaphors, pipeline vocabulary, or coined shorthand (write "sessions open + across days", never a coinage like "marathon sessions"); when a card names a skill or + tool as a fix, the body says in one clause what it literally does; dates absolute. +- **Section headings** — the one-pager's skeleton headings (Proposed changes / Key + metrics / Key findings / Data limitations / Supporting analysis) are user-approved + standard vocabulary: keep them. Inside section pages, retitle weak headings to state + the literal fact ("Worker lanes default to Opus"), never a punchy coinage or metaphor. Conversion is the floor, not the bar: a page that preserves the Markdown's structure and phrasing with components sprinkled in is a failed pass. The Markdown supplies facts, @@ -138,8 +156,12 @@ are designed. - [ ] Eyebrow + `# title` + bold thesis, nothing between them — on the one-pager AND every section page. -- [ ] Headline numbers are a `metric-grid` with judgment colors and notes. -- [ ] Findings on the one-pager are `rec` cards with stat rows. +- [ ] Headline numbers are a `metric-grid` with judgment colors and notes — only on + reports that have a Key metrics section; none added to change-list reports. +- [ ] Findings / proposed changes on the one-pager are `rec` cards with stat rows, in + source order. +- [ ] Diffs, proposed files, and move tables are verbatim code blocks/tables — nothing + trimmed or reworded. - [ ] Each section page's visuals each carry a distinct claim (typically 2-3 per page); source tables kept where numbers matter. - [ ] Every per-entity rollup (by user/gateway, team, repo, model) has a companion diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/example-enrichment.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/example-enrichment.md index 48590819..9f10fdb7 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/example-enrichment.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-report-to-html/example-enrichment.md @@ -13,6 +13,14 @@ the reference for SKILL.md step 3: same moves, same class names, numbers taken v from the plain version. Component reference: [`components.md`](components.md); rules: [`authoring.md`](authoring.md). +> ⚠ **The BEFORE below is the improvement review's OLD source shape.** Since 2026-07-14 +> that report emits a numbered **Proposed changes** list with no "Key numbers" table and +> no findings section (authoring.md §2–3). For today's improvement review: no +> `metric-grid` anywhere on its one-pager — map each numbered change to one `rec` card +> (bold what = title, why = body, evidence = stat row) in source order. The +> metric-grid moves below still apply to reports that HAVE a headline-numbers section +> (usage, security). The class names and width/judgment mechanics are unchanged. + ## BEFORE — plain Markdown as the report skills emit it ```markdown diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-sensitive-scan/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-sensitive-scan/SKILL.md new file mode 100644 index 00000000..3a757f05 --- /dev/null +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-sensitive-scan/SKILL.md @@ -0,0 +1,107 @@ +--- +name: hypaware-sensitive-scan +description: Scan HypAware recordings for sensitive content (secrets, credentials, secret-file reads, PII, private-repo code) and recommend what to opt out of recording - a folder-scoped `.hypignore` / `hyp ignore ` for directory-bound exposure, or a per-session opt-out for one-off sensitive sessions. Use when the user says "scan my logs for secrets", "what should I hypignore", "did I record anything sensitive", or wants ignore recommendations grounded in recorded data. REDACTS every secret (never echoes a value). Recommends PROSPECTIVE opt-outs only - it does NOT purge already-recorded rows. Asks which source to scan (local logs or a remote server) first. +--- + +# Scan recordings for sensitive data, recommend ignores + +Turn a window of recorded AI-gateway exchanges into a short, ranked answer to one +question: **what sensitive content did HypAware capture, and where should the user +turn recording off so it stops happening.** The deliverable is copy-paste ignore +commands grouped by scope, not a saved report. + +IMPORTANT: Both ignore mechanisms are **prospective only** - a `.hypignore` and the +session opt-out gate *future* recording; neither purges rows already in the cache +(LLP 0049 §prospective-only). So the sensitive rows you find stay captured. Say this +plainly and never imply the scan removes anything. + +IMPORTANT: **REDACT everything.** Never echo a secret, token, key, credential, or raw +PII value - not in a finding, not from `content_text`, not from `tool_args`. A secret +appearing in the recording *is* the finding; name the scope and class, quote nothing. + +IMPORTANT: **Ask which source first.** List the data sources and let the user choose: +**local logs** (this machine, `hyp query sql …`, no `--remote`) and each remote +HypAware server (every `hyp remote list` target, plus any hypaware MCP `query_sql` +tool already in your toolset; the same server can appear both ways - list it once). +Query mechanics are step 0 of the procedure — a mandatory read, not a reference. + +## What counts as sensitive (match over `content_text` AND `CAST(tool_args AS VARCHAR)`) +| Class | Match signals | Why it matters | +| --- | --- | --- | +| Secret in payload | `AKIA…`, `sk-…`, `ghp_…`/`gho_…`, `xox[baprs]-…`, `-----BEGIN … PRIVATE KEY-----`, `Bearer `, `password=`/`api_key=`, `postgres://user:pass@` | a live credential now sits in the cache | +| Secret-file read | reads of `.env`, `id_rsa`, `.ssh/`, `.aws/credentials`, `.netrc`, `*.pem`, `*.key`, `credentials.json` | key/config contents captured verbatim | +| PII | email, phone, national-id and card-like number runs | personal data captured | +| Private code / infra | private `git_remote`, internal hostnames, connection strings, non-public source paths | proprietary IP in the cache | + +## Procedure +0. **Load query mechanics BEFORE the first query — skills, not memory.** After the user picks a + source and before any `hyp query sql`, read the **hypaware-query** skill (invoke it or Read + its SKILL.md), and the **hypaware-graph** skill if `hyp query status` lists `node`/`edge` + datasets. Memory notes from past runs do NOT substitute — stale notes have cost real runs + failed queries and server crashes (a phantom "100-row output cap"; message-table `cwd` scans + that 504'd then OOM'd the prod server). This scan's core job IS reading wide columns + (`content_text`, `tool_args`) — keep those queries aggregate-shaped regex matches over + **date-sliced, server-sized ranges** (especially against a remote server), never bulk + row-fetches of raw content; the step-2 rollup GROUP BY is on scope keys, not content. Use the + graph's tiny `node`/`edge` tables for the scope inventory where projected (which + sessions/cwds/repos exist, session→directory attribution via Session nodes' `props.cwd`) + instead of DISTINCT-scanning message columns. Capture stderr and check it even on success — + truncation and server-cap notices land there, and a clipped scan reads as "no hits". If a + query fails, come back to this step; don't iterate on the failing SQL. +1. **Pick the source** (above), then a **baseline**: window; total rows; distinct + `repo_root` / `cwd` / `session_id`; coverage of `content_text` and `tool_args` (a + sparse column bounds what the scan can see - say so). State N. +2. **Scan.** Match each class over `content_text` and `CAST(tool_args AS VARCHAR)` + (`tool_args` is JSON that may hold a plain string - never `JSON_EXTRACT` it; regex + the cast text per the hypaware-query dialect notes). For every hit, capture the + **scope keys** `repo_root` (fallback `cwd`), `session_id`, and `class` - **not** the + matched value. Roll up hit counts per `(scope, class)` and per `(session_id, class)`. + + ```sql + SELECT repo_root, cwd, session_id, + count(*) AS hits + FROM ai_gateway_messages + WHERE regexp_matches(coalesce(content_text,'') || ' ' || CAST(tool_args AS VARCHAR), + '(AKIA[0-9A-Z]{16}|sk-[A-Za-z0-9]{20,}|ghp_[A-Za-z0-9]{20,}|-----BEGIN [A-Z ]*PRIVATE KEY-----|xox[baprs]-|postgres://[^:]+:[^@]+@)') + GROUP BY repo_root, cwd, session_id + ORDER BY hits DESC + ``` +3. **Route each scope to a recommendation.** Two shapes, matched to how the exposure is + bound: + - **Directory-bound** (a `repo_root`/`cwd` that keeps producing hits across sessions, + or is inherently sensitive - a secrets repo, an infra tree) -> a **folder + `.hypignore`**, durable and committable/team-wide. + - **One-off** (hits isolated to a single `session_id` in a directory the user would + not want permanently ignored) -> a **per-session opt-out** at session start. + +## Recommendations (the output) +Print findings **most sensitive first** (weight: live-credential > secret-file > PII > +private-code; break ties by hit count and by how many distinct sessions a scope spans). +All examples redacted. Two groups, then the caveat: + +- **Folder ignores** - one line per scope: a redacted headline (` in , + N hits across M sessions`) followed by the copy-paste command on its own line: + ```bash + hyp ignore # writes a committable .hypignore; covers all future sessions in this tree + ``` + Note that an empty `.hypignore` at the repo root does the same for the whole repo, and + that folder matching keys on `cwd`/`repo_root` (Claude + Codex only; a no-op for + raw-proxy / OTEL sources, which carry no `cwd` - LLP 0049 §non-goals). +- **Session opt-outs** - for one-off sensitive work in a directory the user won't + permanently ignore. The in-session opt-out skill (`/hypaware-ignore`, reversed with + `/hypaware-unignore`) ships for Claude sessions only; for one-off work in Codex, + recommend a narrow folder ignore instead (`hyp ignore `, removable with + `hyp unignore` once the sensitive work is done). +- **Caveat (always).** The sensitive rows above are **already recorded**; ignoring is + prospective and does not delete them. Show the residual with + `hyp ignore --check ` (reports the governor and how many cached rows from the + scope remain). Retroactive purge is out of scope. + +## Notes +- Redaction is absolute; a raw secret in `tool_args` is itself worth flagging (the + gateway captured it) - name it, don't reproduce it. +- Read stderr on every query (staleness / row-budget notices land there); an empty + stdout is not the same as zero hits. +- Keep the pattern set honest: a high-recall regex over free text produces false + positives (an example string, a doc). Spot-check counts before recommending an ignore, + and prefer recommending the *scope* over asserting a specific leak you cannot verify. diff --git a/hypaware-core/plugins-workspace/codex/src/index.js b/hypaware-core/plugins-workspace/codex/src/index.js index 30f79656..1bdbdee5 100644 --- a/hypaware-core/plugins-workspace/codex/src/index.js +++ b/hypaware-core/plugins-workspace/codex/src/index.js @@ -43,9 +43,10 @@ export const configSection = { section: CODEX_CONFIG_SECTION, validate: validate * * Resolves the `hypaware.ai-gateway` capability, registers the * OpenAI-compatible upstream preset, wires Codex's config.toml - * `attach()`, and contributes the `hypaware-query` skill plus - * the AI report skills (adoption/improvement/security/spend) and - * the report-to-html renderer for Codex installs. + * `attach()`, and contributes the `hypaware-query`, `hypaware-graph`, + * and `hypaware-sensitive-scan` skills plus the AI report skills + * (usage, plus the superseded adoption/spend, improvement, security) + * and the report-to-html renderer for Codex installs. * * `attach()` emits a `client.attach` span tagged with `hyp_plugin`, * `client_name`, `status`, and `restored=true|false`. The reversing @@ -215,6 +216,9 @@ export async function activate(ctx) { 'hypaware-query', 'hypaware-reference', 'hypaware-privacy', + 'hypaware-graph', + 'hypaware-sensitive-scan', + 'hypaware-ai-usage-report', 'hypaware-ai-adoption-report', 'hypaware-ai-improvement-report', 'hypaware-ai-security-report',