diff --git a/.gitignore b/.gitignore index 43ca9bd9..a09c39a3 100644 --- a/.gitignore +++ b/.gitignore @@ -43,4 +43,7 @@ openspec/ # Sandcat .devcontainer -.sandcat/settings.local.json \ No newline at end of file +.sandcat/settings.local.json +# agent scratch (SDD ledger/reports, transient worktrees) +.superpowers/ +.claude/worktrees/ diff --git a/AGENTS.md b/AGENTS.md index 63e06eb2..a7860967 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,36 +16,76 @@ listed in the README. ``` orca/ ├── build.sbt / project/ -├── tools/ # tool interfaces + os-backed impls + structured I/O + event bus -├── flow/ # FlowContext, stage/fail; orca.review (ReviewTypes, ReviewLoop, Reviewers); orca.bug; orca.plan -├── claude/ # Claude Code backend + DefaultClaudeTool + DefaultLlmCall -├── codex/ # Codex backend (codex exec --json over stdio) -└── runner/ # flow() entry + DefaultFlowContext + terminal layer +├── tools/ # tool traits + os-backed impls (git/gh/fs), LLM SPI + session registry, InStage, events, subprocess +├── flow/ # stage/display/fail + FlowContext/FlowControl; orca.{plan,review,pr,progress} +├── claude/ codex/ gemini/ opencode/ pi/ # one module per coding-agent backend +└── runner/ # flow() entry, DefaultFlowContext, FlowLifecycle, terminal UI ``` Dependency graph: ``` tools (standalone) - ├── flow → tools - ├── claude → tools - ├── codex → tools - └── runner → tools + flow + claude + codex + ├── flow → tools + ├── claude / codex / gemini / + │ opencode / pi → tools + └── runner → tools + flow + all five backends ``` The runner module owns the `flow` entry point (`package orca`) and wires -defaults via `DefaultFlowContext` (`package orca.runner`). Its terminal UI -lives in its own sub-package, `orca.runner.terminal`, so swapping it for a -Slack or HTTP equivalent is a matter of substituting one `Interaction` at -the call site rather than rewiring modules. - -Only the user-facing surface lives in `package orca` (the `flow` entry -point, the tool traits, the accessors, `JsonData`, `OrcaArgs`). -Implementations live in focused subpackages: `orca.tools.fs` / -`orca.tools.git` / `orca.tools.github` (os-backed tool impls), -`orca.tools.claude` / `orca.tools.codex` (LLM backends), `orca.subprocess` -(subprocess shim), `orca.io` (structured-I/O plumbing), `orca.runner` / -`orca.runner.terminal` (wiring and terminal UI). +defaults via `DefaultFlowContext` (`package orca.runner`); the stage +setup/teardown/recovery state machine is `orca.runner.FlowLifecycle`. The +terminal UI lives in `orca.runner.terminal`, behind an `Interaction`, so +swapping it for a Slack or HTTP equivalent is one substitution at the call +site rather than rewiring modules. + +The user-facing surface lives in `package orca` (the `flow` entry, the tool +accessors — including `agent`, the backend-agnostic leading-agent accessor (an +ordinary `FlowContext` member typed `Agent[ctx.LeadB]`, where `flow[B]` captures +the selector's backend tag into the `FlowContext { type LeadB }` member) — +`stage`/`display`/`fail`, `JsonData`, `OrcaArgs`). Implementations +live in focused subpackages: `orca.tools` (os-backed git/gh/fs impls + their +traits), `orca.agents` + `orca.backend` (LLM SPI, `SessionRegistry`, conversation +driver), `orca.subprocess` (subprocess shim), `orca.events` (event bus), one +`orca.tools.` per coding agent, and `orca.runner` / `orca.runner.terminal` +(wiring + terminal UI). The flow module adds `orca.{plan,review,pr,progress}`. + +## The stage-bound runtime + +The flow runtime is specified in [ADR 0018](adr/0018-stage-bound-flow-runtime.md) — +read it before touching `stage`, the progress log, or sessions. The invariants +most easily broken: + +- **Capability gating.** Three compile-time capabilities gate side effects: + `FlowContext` (reads + emit; thread-safe), `FlowControl <: FlowContext` + (authority to start a stage; thread-affine), and the opaque `InStage` token + (in `tools`, `package orca`). Every mutating tool method — git writes, + `fs.write`, `gh` writes, every `agent.*.run` — takes `(using InStage)`, which + only a `stage` body mints. Don't relax this: don't mint `InStage.unsafe` + outside the runtime (`Flow` / `FlowLifecycle` / `Session`), and don't drop a + `(using InStage)` to "make it compile" — thread it up to the nearest stage. + `orcacaps.InStageNegativeTest` pins that a mutation outside a stage fails to + compile. + +- **Progress log + recovery.** A run commits `.orca/progress-.json` (hash = + prompt, so the path is branch-independent) with one entry per completed stage; + a re-run replays recorded entries and skips them. The header is untrusted on + load — `orca.progress.RecoveryCheck` validates it (safe ref, prompt-hash match, + protected-branch refusal) before any destructive git op. + +- **Sessions.** `SessionRegistry` has two shapes: `ClaimedOnce` (claude/pi — the + client id IS the wire id) and `ClientToServer` (codex/gemini/opencode — a + server-minted id learned from the protocol). The id persisted for resume is the **resume wire + id** — uniformly "the id to put on the wire when resuming" (`Dispatch.wireId`): + for codex/gemini/opencode a server-thread id, for claude the client id itself, + for pi `None` (ephemeral). A backend whose sessions survive a process restart + MUST wire **both** `resumeWireId` (so the runtime records it in the log) and + `registerSession` (so `rehydrateSessions` re-claims it on resume) — claude and + codex each shipped a resume bug from getting this wrong. Note `ClaimedOnce` is + not enough on its own: claude overrides `resumeWireId` to persist (durable + on-disk sessions), pi does **not** (its temp-dir sessions can't survive). + `sessionExists` is a best-effort, non-destructive probe; when it can't confirm a + live session the flow re-seeds, the uniform fallback that holds on every backend. ## Build and test @@ -73,20 +113,21 @@ Some tests shell out to real external tools and skip by default: ```bash ORCA_INTEGRATION=1 sbt test -ORCA_INTEGRATION=1 sbt "claude/testOnly orca.claude.ClaudeIntegrationTest" +ORCA_INTEGRATION=1 sbt "claude/testOnly orca.tools.claude.ClaudeIntegrationTest" ORCA_INTEGRATION=1 sbt "tools/testOnly orca.tools.OsGitHubIntegrationTest" ORCA_INTEGRATION=1 sbt "runner/testOnly orca.runner.terminal.ScalaCliSmokeTest" ``` | Suite | Needs | |---|---| -| `ClaudeIntegrationTest` | `claude` authenticated | +| `{Claude,Codex,Gemini,Opencode,Pi}IntegrationTest` (one per `orca.tools.`) | that backend's CLI authenticated | | `OsGitHubIntegrationTest` | `gh` authenticated | | `ScalaCliSmokeTest` | `scala-cli`; runs `sbt publishLocal` internally | -Unit tests use in-memory fakes (`StubCliRunner`, `FakeLlmTool`, -`FakeCliProcess`, `TestFlowContext`) — no network, no real filesystem -outside of `os.temp.dir()`. +Unit tests use in-memory fakes (`StubCliRunner` / `SpawnStubCliRunner`, +`FakeAgent`, `FakePipedCliProcess`, `TestFlowContext` / `TestFlowControl`) and +the shared `orca.testkit.GitRepo` temp-repo fixture (published via `tools % +test->test`) — no network, no real filesystem outside `os.temp.dir()`. ### Iterating quickly @@ -140,9 +181,9 @@ sbt publishLocal ``` Installs `org.virtuslab::orca:0.0.14` plus its transitive modules -(`orca-tools`, `orca-flow`, `orca-claude`, `orca-codex`) into -`~/.ivy2/local` so a flow script with `//> using repository ivy2Local` can -resolve them. +(`orca-tools`, `orca-flow`, and the five backends +`orca-{claude,codex,gemini,opencode,pi}`) into `~/.ivy2/local` so a flow script +with `//> using repository ivy2Local` can resolve them. For an iteration loop while hacking on Orca itself, run sbt in one terminal with a `~` watch-and-publish: diff --git a/README.md b/README.md index 40c8aa2a..a19e2a06 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,8 @@ Deterministic, AI-driven development flows. Orca allows you to programmatically define software development workflows where AI agents perform the coding. If you want AI-generated code to always be -reviewed by another agent, don’t try to coerce the agents; just express that -requirement in code. Don’t waste tokens on formatting, committing, or creating +reviewed by another agent, don't try to coerce the agents; just express that +requirement in code. Don't waste tokens on formatting, committing, or creating PRs - all of this can be handled by an ordinary script. Orca flow scripts are written in Scala, and can be run with a single command @@ -29,44 +29,41 @@ Save this as `implement.sc` and run it with your task: import orca.{*, given} -flow(OrcaArgs(args)): - // Plan persists to `.orca/plan-.md` so a re-run with the same prompt - // resumes from the first incomplete task. Plan.autonomous.from runs the - // planner as one agentic turn (Plan.interactive.from lets it ask clarifying - // questions); `.value` keeps just the plan, dropping the planner's session. - // `recoverOrCreate` checks out the branch and writes the file before we start. - val planFile = Plan.defaultPath(userPrompt) - val plan = stage("Acquire plan"): - Plan.recoverOrCreate(planFile, "orca: starting work"): - Plan.autonomous.from(userPrompt, claude).value - - // Stable session reused across tasks so the implementer retains context. - // The planner's isn't carried forward — it's read-only and would stay so - // on resume. - val session = claude.newSession - - // Per task: implement, then review & fix. `implementTaskLoop` ticks the - // checkbox, commits per task, and removes the plan file at the end. The one - // commit captures the implementation, formatting, and any reviewer fixes. - Plan.implementTaskLoop(planFile, plan): task => - stage(s"Implement task: ${task.title}"): - stage("Implementation"): - val _ = claude.autonomous.run(task.description, session) - reviewAndFixLoop( - coder = claude, - sessionId = session, - reviewers = allReviewers(claude), - // Cheap model picks the per-task reviewers from their descriptions and - // the changed files. Swap for ReviewerSelector.allEveryRound to run all. - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), +// `_.claude` selects the leading agent (the coding harness — claude, codex, …). +// Inside the body, reference it as `agent`, not `claude`, so the flow is +// backend-agnostic: switch the selector to `_.codex` and the whole flow follows. +flow(OrcaArgs(args), _.claude): + // `stage` is the committing, resumable unit of work. The plan is produced in + // one agentic turn and recorded in the stage log; a re-run with the same + // prompt skips this stage and reads the stored Plan back. + // plan.brief is always present — feed it to `agent.session(seed = plan.brief)`. + val plan = stage("Plan"): + Plan.autonomous.from(userPrompt, agent).value // .value takes the Plan, discarding the planner's session + + // Get-or-create the implementer session (pure: id reserved, backend created + // on first use). The seed (plan.brief) primes it on first use and is + // replayed if the backend session is lost on resume. + val session = agent.session(seed = plan.brief) + + // One stage per task: each stage commits its work + a progress-log entry as + // one commit. Completed stages are skipped on resume — re-running the same + // prompt picks up from the first incomplete task. + for task <- plan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + agent.runSeeded(task.description, session) + reviewAndFixLoop( // runs under this stage + coder = agent, sessionId = session, + reviewers = allReviewers(agent), + // agent.cheap picks the per-task reviewer subset; swap for + // `ReviewerSelector.allEveryRound` to run every reviewer. + reviewerSelection = ReviewerSelector.agentDriven(agent.cheap), task = task.title.value, - // Runs after every edit so commits stay formatted and reviewers skip - // style nits. - formatCommand = Some("sbt scalafmtAll"), - // Cheap sanity gate; correctness is the reviewers' and CI's job, so - // skip the heavier test suite. - lintCommand = Some("sbt Test/compile"), - lintLlm = Some(claude.haiku) + // Format after every edit so commits stay formatted and reviewers + // skip style nits. + formatCommand = Some("cargo fmt"), + // Cheap sanity gate; correctness is the reviewers' and CI's job. + lintCommand = Some("cargo check --tests"), + lintAgent = Some(agent.cheap) ) ``` @@ -74,6 +71,15 @@ flow(OrcaArgs(args)): scala-cli run implement.sc -- "Add a rate-limiter to the /login endpoint" ``` +The feature branch is auto-created from the prompt. On success you're left **on +the feature branch**, with the work — this flow opens no PR, so you stay where +the code is (ready to test or open a PR by hand). A throwaway branch (nothing +substantive landed, e.g. an early-exit flow) is deleted and HEAD returns to the +starting branch instead. Flows that open a PR pass `returnToStartBranch = true` +to switch back to the starting branch afterward. On failure the flow stays on +the feature branch so a re-run resumes in place — HEAD is already on the right +branch and the committed progress log is in the working tree. + There are two runnable examples under [`examples/runnable/`](examples/runnable/): * [01-simple](examples/runnable/01-simple/) (in-memory plan + review, autonomous planner), @@ -93,13 +99,13 @@ The following are available inside a `flow(...) { ... }`: | Tool | Methods | Purpose | |---|---|---| -| `claude` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `newSession`, `haiku`/`sonnet`/`opus`/`fable`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withNetworkTools`, `withSelfManagedGit` | Claude Code coding/reviewing agent. Bare `claude` is **Opus with the 1M-token context window** (the long-lived implementer needs it; reviewers share it); use `claude.sonnet` / `claude.haiku` for cheap one-shot calls (reviewer picker, lint, PR summariser), or `claude.fable` for the most capable tier on the hardest one-shots. `interactive` mode lives only on `resultAs[O]`. | -| `codex` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `newSession`, `mini`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | OpenAI Codex coding/reviewing agent. | -| `opencode` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `newSession`, `anthropicOpus`/`anthropicSonnet`/`anthropicHaiku`, `openaiGpt5`/`openaiGpt5Codex`/`openaiGpt5Mini`, `withModel(providerModel)` / `withModel(provider, modelId)`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | [OpenCode](https://opencode.ai) coding/reviewing agent, driven over HTTP+SSE against a headless `opencode serve` (started lazily, shared for the run). Spans providers, so models are provider-qualified: use an accessor (`opencode.openaiGpt5Mini`) or `opencode.withModel("openai/gpt-4o-mini")` / `opencode.withModel("ollama", "llama3.1")`. Inherits the user's configured `opencode` providers/auth. | -| `pi` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `newSession`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | [Pi](https://pi.dev/) coding agent backend, driven through `pi --mode rpc`. Pi handles provider/model selection through its own CLI configuration; pin a model with `pi.withConfig(LlmConfig(model = Some(Model("provider/model"))))`. Interactive calls can ask clarifying questions via Orca's `ask_user` bridge. | -| `gemini` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `newSession`, `flash`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | Google Gemini CLI coding/reviewing agent, driven via `gemini --output-format stream-json`. Bare `gemini` pins **Gemini 2.5 Pro**; use `gemini.flash` for cheaper one-shot calls. Structured output is prompt-enforced (Gemini has no schema flag); `withReadOnly` maps to `--approval-mode plan`. See [ADR 0015](adr/0015-gemini-stream-json-driver.md). | -| `git` | `createBranch`, `checkout`, `checkoutOrCreate`, `ensureClean`, `commit`, `push`, `currentBranch`, `diff`, `log`, `addWorktree`, `removeWorktree`, `listWorktrees` | Git operations against the working tree. Recoverable failures (`BranchAlreadyExists`, `BranchNotFound`, `NothingToCommit`, `PushRejected`, `WorktreeAddFailed`, `WorktreeNotFound`) surface as `Either`; `.orThrow` converts a `Left` back to an exception when the case is unexpected. | -| `gh` | `createPr`, `updatePr`, `readIssue`, `readIssueComments`, `readPrComments`, `writeComment(pr, body)` / `writeComment(issue, body)`, `buildStatus`, `waitForBuild` | GitHub PR + CI integration via the `gh` CLI. `createPr` returns `Either[PrCreateFailed, …]` (covers `PrAlreadyExists` / `NoCommitsToPr`); `updatePr` replaces a PR's title + body (refresh a tentative description once the fix lands); `waitForBuild` returns `Either[BuildWaitFailed, …]`. | +| `claude` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `session(seed)`, `runSeeded(prompt, session)`, `haiku`/`sonnet`/`opus`/`fable`, `cheap` (→ haiku), `withModel(Model)`, `withCheapModel`, `sessionExists(session)`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withNetworkTools`, `withSelfManagedGit` | Claude Code coding/reviewing agent. Bare `claude` is **Opus with the 1M-token context window** (the long-lived implementer; reviewers share it); use `claude.sonnet`/`claude.haiku` for cheap one-shot calls, or `claude.fable` for the hardest ones. `interactive` mode lives only on `resultAs[O]`. `session`/`runSeeded` are flow extensions (see [Sessions](#sessions)). | +| `codex` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `session(seed)`, `runSeeded(prompt, session)`, `mini`, `cheap` (→ mini), `withModel(Model)`, `withCheapModel`, `sessionExists(session)`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | OpenAI Codex coding/reviewing agent. | +| `opencode` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `session(seed)`, `runSeeded(prompt, session)`, `anthropicOpus`/`anthropicSonnet`/`anthropicHaiku`, `openaiGpt5`/`openaiGpt5Codex`/`openaiGpt5Mini`, `cheap` (provider-matched: openai→mini, else anthropicHaiku), `withCheapModel`, `withModel(providerModel)` / `withModel(provider, modelId)`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | [OpenCode](https://opencode.ai) coding/reviewing agent, driven over HTTP+SSE against a headless `opencode serve` (started lazily, shared for the run). Spans providers, so models are provider-qualified: use an accessor (`opencode.openaiGpt5Mini`) or `opencode.withModel("openai/gpt-4o-mini")` / `opencode.withModel("ollama", "llama3.1")`. Inherits the user's configured `opencode` providers/auth. | +| `pi` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `session(seed)`, `runSeeded(prompt, session)`, `withModel(Model)`, `withCheapModel`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | [Pi](https://pi.dev/) coding agent backend, driven through `pi --mode rpc`. Pi handles provider/model selection through its own CLI configuration; pin a model with `pi.withModel(Model("provider/model"))`. Interactive calls can ask clarifying questions via Orca's `ask_user` bridge. | +| `gemini` | `autonomous.run(prompt, session?)`, `resultAs[O].{autonomous,interactive}.run(input, session?)`, `session(seed)`, `runSeeded(prompt, session)`, `flash`, `cheap` (→ flash), `withModel(Model)`, `withCheapModel`, `withConfig`, `withSystemPrompt`, `withName`, `withReadOnly`, `withNetworkOnly`, `withSelfManagedGit` | Google Gemini CLI coding/reviewing agent, driven via `gemini --output-format stream-json`. Bare `gemini` pins **Gemini 2.5 Pro**; use `gemini.flash` for cheaper one-shot calls. Structured output is prompt-enforced (Gemini has no schema flag); `withReadOnly` maps to `--approval-mode plan`. See [ADR 0015](adr/0015-gemini-stream-json-driver.md). | +| `git` | `createBranch`, `checkout`, `checkoutOrCreate`, `ensureClean`, `commit`, `forceAdd`, `push`, `currentBranch`, `diff`, `diffVsBase`, `defaultBase`, `log`, `resetHard`, `deleteBranch`, `addWorktree`, `removeWorktree`, `listWorktrees`, `diffBranchExcludingOrca` | Git operations against the working tree. Recoverable failures (`BranchAlreadyExists`, `BranchNotFound`, `NothingToCommit`, `PushRejected`, `WorktreeAddFailed`, `WorktreeNotFound`) surface as `Either`; `.orThrow` converts a `Left` back to an exception when the case is unexpected. `forceAdd`, `resetHard`, `deleteBranch` are used by the flow runtime for bookkeeping and teardown. | +| `gh` | `createPr`, `updatePr`, `readIssue`, `readIssueComments`, `readPrComments`, `writeComment(pr, body)` / `writeComment(issue, body)`, `upsertComment(pr, marker, body)` / `upsertComment(issue, marker, body)`, `buildStatus`, `waitForBuild` | GitHub PR + CI integration via the `gh` CLI. `createPr` is idempotent by branch (returns the existing PR if one is open); `upsertComment` finds a prior comment carrying `marker` and edits it in place (safe on re-run — use `orcaCommentMarker(userPrompt, purpose)` to embed the prompt hash as the marker). `updatePr` replaces a PR's title + body. `waitForBuild` returns `Either[BuildWaitFailed, …]`. | | `fs` | `read`, `write`, `list` | Working-tree file I/O. `read` returns `Option[String]` so a missing file is a branch point, not an exception. | The runtime owns git: every write-capable agent turn is told not to commit, @@ -115,38 +121,40 @@ case class) for schema generation and deserialization. Additionally, you might define an `Announce[O]` so that a friendly summary is printed in the event log, instead of a raw json. -For multi-task loops, pre-allocate one session and pass it on every call: - -```scala -val session = claude.newSession -for task <- tasks do - val _ = claude.autonomous.run(task.description, session) -``` - -The first call opens the session; subsequent calls resume it. The library -tracks fresh-vs-resume internally per backend (`--session-id` then `--resume` -on claude; a client→server mapping on codex, opencode, and gemini — codex and -gemini mint their own id and resume via `codex exec resume` / `gemini --resume`; -a per-session `--session-dir` resumed with `--continue` on pi). - A minimal Pi-backed flow looks the same; Pi reads your normal Pi configuration and is driven through RPC mode under the hood: ```scala flow(OrcaArgs(args)): - val session = pi.newSession - val _ = pi.autonomous.run(userPrompt, session) + val session = pi.session(seed = userPrompt) + stage("Run"): + pi.runSeeded(userPrompt, session) ``` ## Coding agent tools +There are two ways to drive a model in a flow: + +- **The leading agent — `agent`.** Backend-agnostic: it's whatever the `flow` + selector picked (`_.claude`, `_.codex`, …). Use it for the flow's planning, + implementation, reviewing, and its session (`agent.session(seed)` → + `agent.runSeeded(...)`). Switch the selector and the whole flow follows; you + never name a backend in the body. +- **A specific agent + model — `claude.opus`, `codex.mini`, `opencode.openaiGpt5Mini`.** + Use a concrete accessor when you want a particular backend or tier, or for + interactive planning (`Plan.interactive` needs a concrete backend). The tier + accessors (`.opus`/`.sonnet`/…) live on the concrete agents, not on `agent` — + so `agent.opus` won't compile; that's the cue to name the backend. Pin any + other model with `withModel(Model("…"))`. Don't mix the two for one session + (a `SessionId` is backend-typed). + > [!WARNING] > **Coding agent tool usage is auto-approved by default** (`tools = > ToolSet.Full`, `autoApprove = AutoApprove.All`): write-capable turns let the > agent edit files and run shell commands without prompting. Constrain this in > code, or isolate the whole run in a sandbox. -Two axes constrain an agent. **Capability** (`LlmConfig.tools: ToolSet`) is +Two axes constrain an agent. **Capability** (`AgentConfig.tools: ToolSet`) is which tools exist at all: ```scala @@ -170,7 +178,7 @@ only on `Full`: ```scala // Restrict auto-approval to a named tool set (honoured by claude/codex). val limited = claude.withConfig( - LlmConfig(autoApprove = AutoApprove.Only(Set("Read", "Edit", "Grep"))) + AgentConfig(autoApprove = AutoApprove.Only(Set("Read", "Edit", "Grep"))) ) ``` @@ -186,72 +194,159 @@ solution. Top-level, available via `import orca.*`: -| Method | Use | -|---|---| -| `flow(args, ...)(body)` | Entry point. Sets up the `FlowContext` for the body. | -| `stage(name)(body)` | Wrap an operation in a named stage. Emits `StageStarted`/`StageCompleted` and shows in the status-bar breadcrumb. | -| `fail(message)` | Abort the current stage with an error. | +| Method | Signature | Use | +|---|---|---| +| `flow(args, agent, ...)(body)` | `flow(args: OrcaArgs, agent, branchNaming?, returnToStartBranch = false, progressStore?)(body)` | Entry point. Creates one feature branch + one progress log for the run. `agent` selects the leading coding agent — e.g. `_.claude` or `_.codex`. Inside the body, reference the lead via the backend-agnostic `agent` accessor instead of a concrete `claude`/`codex` (autonomous, tier-agnostic flows). Branching defaults to a slug of the prompt; pass `branchNaming = Some(BranchNamingStrategy.issue(handle))` for issue flows. On success HEAD stays on the feature branch by default; pass `returnToStartBranch = true` (PR flows) to return to the starting branch. | +| `agent` (in-body accessor) | `agent: Agent[?]` | The leading agent resolved from the `flow` selector. Use it for autonomous, tier-agnostic work (`agent.session`, `agent.runSeeded`, `agent.cheap`, `Plan.autonomous.from(_, agent)`) so the body is backend-agnostic. Backend-specific tiers (`claude.opus`) and interactive planning (`Plan.interactive`, needs `CanAskUser`) still use a concrete accessor (`claude`/`codex`). | +| `stage[T: JsonData](name, commitMessage?)(body)` | `(name: String, commitMessage: Option[T => String] = None)(body): T` | The committing, resumable unit of work. On success, records the result, force-adds the progress log, and commits (code changes + log delta = one commit). On re-run, a stage whose result is still recorded is skipped and the stored value is returned. `T` must have `JsonData` — `case class Foo(...) derives JsonData` is enough. Commit message defaults to an `agent.cheap` summary of the diff; override via `commitMessage`. | +| `display(message)` | `(message: String): Unit` | Progress-only output: no stage, no commit, no log entry. Callable anywhere — outside a stage or inside a fork. | +| `fail(message)` | `(message: String): Nothing` | Abort with a message. Triggers failure teardown: stays on the feature branch so a re-run resumes. | + +### Side effects happen inside stages + +Every side-effecting call must happen inside a `stage` body, and **the compiler +enforces it** — a mutation written outside a stage doesn't compile, so a flow +that side-effects without a checkpoint is a compile error, not a runtime +surprise. That covers git mutations (`commit`/`push`/`resetHard`/…), `fs.write`, +`gh` writes (`createPr`/`updatePr`/`writeComment`/`upsertComment`), and every +`agent.*.run`. + +Reads (`git.diff`, `git.log`, `git.currentBranch`, `gh.readIssue`, +`gh.buildStatus`/`waitForBuild`, `fs.read`), `display`, and `fail` run anywhere. +`agent.session(seed)` also runs outside a stage — it records a session, not a side +effect (see [Sessions](#sessions)). + +### The flow lifecycle + +Each `flow(...)` run is bound to exactly one feature branch and one progress +log (`.orca/progress-.json`, where `` is derived from the prompt): + +- **Start:** stash a dirty working tree with a warning (recover with `git stash + pop`); create + checkout the feature branch; write and commit the progress log + header. +- **Resume:** the progress log lives at a branch-independent, prompt-derived path, + so recovery finds it before any checkout. Its header is validated as untrusted + input (branch must match orca naming rules, prompt hash must match), then the run + resumes from the first incomplete stage. +- **Success teardown:** remove the progress-log file in a final commit. A + throwaway feature branch (no substantive changes vs the starting branch) is + deleted and HEAD returns to the starting branch. Otherwise the feature branch + is kept and HEAD **stays on it by default** (so you end on the work); pass + `returnToStartBranch = true` — for flows that open a PR — to return HEAD to the + starting branch instead. +- **Failure teardown:** discard the failed stage's uncommitted partial edits with + `git reset --hard`; stay on the feature branch so a re-run resumes in place. + +### Sessions + +`agent.session(seed)` is a get-or-create keyed by call-site position — the same +call site resumes the same session across re-runs. It reserves a `SessionId` and +records it in the progress log (no LLM call), and is callable outside a stage — +recording a session isn't a side effect. -Planning utilities, available via `import orca.plan.*`: +```scala +val session = agent.session(seed = plan.brief) +agent.runSeeded(task.description, session) +``` + +The `seed` is the essential context to rebuild the agent — typically the **plan +brief**, or the issue body when there is no brief. `runSeeded` primes a fresh +session with the seed on first use; if the backend session is lost on resume it +re-seeds, prepending a progress preamble naming the completed stages; if the +session is still alive it continues it directly. For a one-off call that doesn't +need to survive restarts, just omit the session argument (`agent.autonomous.run(prompt)` +mints a fresh one per call). + +`agent.cheap` returns the backend's cheap/fast variant (claude → haiku, codex → +mini, gemini → flash, opencode → anthropicHaiku, others → self) — used by the +runtime for branch naming and default commit messages. + +## Authoring rules + +Mutations outside a stage body are compile errors (see [Side effects happen +inside stages](#side-effects-happen-inside-stages)). The rules below are the +structural conventions you choose to follow as a flow author. + +1. **Reads outside, mutations inside.** Only side-effecting work goes in a + stage. Pure reads (`git.diff`, `gh.readIssue`, `fs.read`, `gh.waitForBuild`) + run outside stages — staging them wastes commits and checkpoints. + `agent.session(seed)` also runs outside stages, but it isn't a pure read — it + records a session in the progress log. + +2. **Push lives in a later stage than the edit that produced it.** A stage + commits only on completion: a `git.push()` in the same stage as the edit would + push nothing (the edit isn't committed yet). The push must be in a *separate, + later* stage: + + ```scala + stage("Write failing test"): + agent.runSeeded("Write the failing test …", session) // commits on completion + + val pr = stage("Push + open PR"): // LATER stage — the test commit exists now + git.push().orThrow + gh.createPr(title = …, body = …).orThrow + ``` + +3. **One commit per stage.** Each stage produces exactly one commit (code + changes + the progress-log entry). Don't call `git.commit` inside a stage + body — the runtime commits for you when the stage completes. + +4. **Idempotent external effects, each in its own stage.** Put each PR-open, + comment-post, or push in a dedicated stage so it's checkpointed. `gh.createPr` + is idempotent by branch (an open PR is reused, not duplicated) and + `gh.upsertComment(target, marker, body)` edits a prior comment carrying + `marker` in place — so if a crash re-opens the stage on resume, the re-run + reuses the PR/comment instead of duplicating it. Use + `orcaCommentMarker(userPrompt, purpose)` so the marker is unique to this run. + +5. **Name stages descriptively.** The stage name appears in the event log, + the commit message (when no override is provided), and the progress preamble + on resume. A name like `"Push + open PR"` lets a reader (and the resuming + agent) understand the checkpoint without reading code. + +## Planning utilities + +Available via `import orca.plan.*`: The planning entry points form a **mode × operation grid**. The two axes are orthogonal — every combination is valid. Mode is picked at the call site -(`Plan.autonomous.*` vs `Plan.interactive.*`), mirroring how `LlmTool` itself +(`Plan.autonomous.*` vs `Plan.interactive.*`), mirroring how `Agent` itself splits `autonomous` / `interactive`: | Operation | Result | `autonomous` (read-only + network, no human) | `interactive` (agent can `ask_user`) | |---|---|---|---| -| `from(userPrompt, llm, instructions?)` | `Plan` | plan in one agentic turn | drive the planner conversationally | -| `assessThenPlan(userPrompt, llm, instructions?)` | `Verdict[Plan]` | assess, then `Proceed(plan)` or `Rejection(kind, body)` | same, but can ask the reporter to clarify instead of rejecting | -| `triage(report, llm, instructions?)` | `Triage` | classify a bug report (not-a-bug / untestable / testable) | same, with clarifying questions | +| `from(userPrompt, agent, instructions?)` | `Plan` | plan in one agentic turn | drive the planner conversationally | +| `assessThenPlan(userPrompt, agent, instructions?)` | `Verdict[Plan]` | assess, then `Proceed(plan)` or `Rejection(kind, body)` | same, but can ask the reporter to clarify instead of rejecting | +| `triage(report, agent, instructions?)` | `Triage` | classify a bug report (not-a-bug / untestable / testable) | same, with clarifying questions | Every cell returns `Sessioned[B, ]` — the result paired with the agent session that produced it. Continue that session into implementation -(`llm.autonomous.run(task, sessioned.sessionId)` — the planning turn's session -is still resumable with write access), or `.value` it and mint a fresh -session via `llm.newSession`. Destructure positionally when you want both: +(`agent.runSeeded(task, session)` — the planning turn's session is still resumable +with write access), or `.value` it and get a fresh implementer session via +`agent.session(seed = plan.brief)`. Destructure positionally when you want both: `val Sessioned(session, plan) = Plan.autonomous.from(...)`. -From a `Sessioned[B, Plan]`, two optional steps refine the plan before -implementing — both resume the planner session read-only: `.reviewed(llm)` (the -planner critiques its own draft → improved `Plan`) and `.briefed(llm)` (the -planner writes a codebase brief for the implementers → `PlanWithBrief`, prepended -to each task by `taskPrompt`). Chain either order, e.g. -`Plan.autonomous.from(...).reviewed(claude).briefed(claude)`. +From a `Sessioned[B, Plan]`, an optional `.reviewed(agent)` step refines the plan +before implementing — the planner critiques its own draft, producing an improved +`Plan`. Chain it: `Plan.autonomous.from(...).reviewed(claude).value`. `assessThenPlan` returns a `Verdict`: `Verdict.Proceed(plan)` to implement, or `Verdict.Rejection(kind, body)` — a follow-up question, critique, or rebuff the caller surfaces back to the reporter. `triage` returns a `Triage` sum type the caller pattern-matches (`NotABug` / `Untestable` / `Testable`). -Persistence + iteration helpers: - -| Method | Use | -|---|---| -| `Plan.{autonomous,interactive}.loadOrGenerate(file, userPrompt, llm, instructions?)` | Idempotent plan acquisition: parse `file` if it exists (resume), otherwise generate (in the chosen mode) and persist as markdown. | -| `Plan.defaultPath(userPrompt, workDir?)` | Returns `/.orca/plan-.md` — the conventional persistent-plan path. `` is the first 6 bytes of SHA-256(userPrompt) rendered as 12 hex chars, so unrelated prompts in the same repo don't collide. | -| `Plan.recoverOrCreate(file, stashMessage?)(generate)` | Resume from `file` if it exists, else `ensureClean` + evaluate `generate`, check out the plan's branch, and persist. The acquisition entry point for resumable flows. | -| `Plan.recover(file)` | Lower-level resume-from-crash: if `file` exists, stash pending edits (`git stash pop` recovers them), switch to the plan's branch, parse, return `Some(plan)`; else `None`. | -| `Plan.implementTaskLoop(file, plan)(body)` / `Plan.implementTaskLoop(plan)(body)` | Iterate `plan` running `body(task)` per task, committing each. The `file` overload also ticks the on-disk checkbox and removes the file at the end (resumable); the file-less overload tracks completion in memory (for flows with their own non-restartable state machine). | -| `Plan.persistComplete(file, title)` | Mark one task complete on disk. Lower-level primitive that the `file` loop is built on. | - -Persistent plans are the default mode for multi-task flows — `implement.sc`, -`implement-interactive.sc`, `epic.sc`, and `issue-pr.sc` all use -`Plan.defaultPath` + `Plan.recoverOrCreate` + `Plan.implementTaskLoop`. See ADR -[0013](adr/0013-persistent-plans.md) for the convention and migration notes. - Review utilities, available via `import orca.review.*`: | Method | Use | |---|---| -| `lint(command, llm, instructions?)` | Run a shell lint, write its combined output to a temp file, and have `llm` read and summarise it as a `ReviewResult` (file, not prompt, so unbounded output can't overflow the context). | +| `lint(command, agent, instructions?)` | Run a shell lint, write its combined output to a temp file, and have `agent` read and summarise it as a `ReviewResult` (file, not prompt, so unbounded output can't overflow the context). | | `reviewAndFixLoop(coder, sessionId, reviewers, task, ..., fixInstructions?)` | Run reviewers against `task`, collect findings above the confidence threshold, hand them to `coder` to fix, re-evaluate. Halts when reviewers come back clean, the fixer marks every remaining issue as won't-fix, or the iteration cap is reached. | | `allReviewers(base)` | All eight canonical reviewer agents (code-functionality, test, readability, code-structure, simplicity, performance, security, scala-fp) layered on top of `base`. | | `minimalReviewers(base)` | Universally-applicable subset (code-functionality, readability, test). Pair with the default LLM-driven selector when the full set is overkill. | | `fixLoop(evaluate, fix, ...)` | Lower-level primitive `reviewAndFixLoop` is built on. | `reviewAndFixLoop` requires a `reviewerSelection: ReviewerSelector` argument. -Typically `ReviewerSelector.llmDriven(claude.haiku)` — the picker LLM (use a +Typically `ReviewerSelector.agentDriven(claude.cheap)` — the picker LLM (use a cheap model) sees each reviewer's description plus the changed file paths and narrows the supplied list per task. Pass `ReviewerSelector.allEveryRound` to run every reviewer every iteration, or @@ -262,14 +357,13 @@ PR utilities, available via `import orca.pr.*`: | Method | Use | |---|---| -| `summarisePr(llm, diff, context?, instructions?)` | Fold a branch diff into a `PrSummary(title, body)` for `gh.createPr`. `context` is an optional preamble (originating issue link, user prompt, etc.) the model anchors the description to. Use a cheap model (`claude.haiku`, `codex.mini`). | +| `summarisePr(agent, diff, context?, instructions?)` | Fold a branch diff into a `PrSummary(title, body)` for `gh.createPr`. `context` is an optional preamble (originating issue link, user prompt, etc.) the model anchors the description to. Use a cheap model (`claude.cheap`, `.cheap`). | ### Customising prompts -Every domain helper that bundles an LLM brief takes the prompt as a -default-valued `instructions: String` parameter; the default value lives on a -sibling `XxxPrompts` object in the same package. Override by passing a -different string, or compose with the default to extend it: +Every domain helper that bundles an LLM brief takes its prompt as a +default-valued `instructions: String`; the default lives on a sibling +`XxxPrompts` object. Override it, or compose with the default to extend it: ```scala import orca.plan.{Plan, PlanPrompts} @@ -281,35 +375,40 @@ Plan.interactive.from( ) ``` -Where the defaults live: -- `orca.plan.PlanPrompts` — `Planning`, `AssessThenPlan`, `Triage`, `Review`, - `Brief` +
+Where the defaults live + +- `orca.plan.PlanPrompts` — `Planning`, `AssessThenPlan`, `Triage`, `Review` - `orca.pr.PrPrompts` — `Summarise` - `orca.review.ReviewLoopPrompts` — `Fix`, `SelectReviewers`, `SummariseLint`, `ReReview` - `orca.review.ReviewerPrompts` — per-reviewer system prompts (compose your own list to swap or extend `allReviewers`/`minimalReviewers`) -The lower-level per-call wrappers (autonomous/interactive/retry) are a -separate layer — replace the whole set via `flow(prompts = ...)`. See ADR -[0010](adr/0010-prompts-and-helpers-convention.md) for the full convention. +The lower-level per-call wrappers (autonomous/interactive/retry) are a separate +layer — replace the whole set via `flow(prompts = ...)`. See [ADR +0010](adr/0010-prompts-and-helpers-convention.md) for the full convention. + +
## Data structures -Common types you'll see in flow scripts. All `derives JsonData`, so the agent -can generate them as structured output via `claude.resultAs[T]`: +Common types you'll see in flow scripts. Most `derives JsonData`, making them +valid stage results (the stage log can record and replay them) and usable as +structured LLM output via `claude.resultAs[T]`. Exceptions: `Sessioned` and +`Verdict` do not derive `JsonData` — they are intermediate values, not stage +results. + +
+The types, in detail (click to expand) -- **`orca.plan.Plan(epicId, description, tasks)`** — the task list the agent - generates in one round-trip; backs both in-memory use (`Plan.*.from`) and the - markdown-persisted resume path (`Plan.*.loadOrGenerate`). `epicId` is a - kebab-case id used as the plan's git branch; `description` is the planner's - epic summary. +- **`orca.plan.Plan(epicId, description, tasks, brief)`** — the task list the + agent generates in one round-trip. `epicId` is a kebab-case id used as the + plan's git branch; `description` is the planner's epic summary; `brief` is a + concise codebase briefing always included (feed it to `agent.session(seed = + plan.brief)`). `taskPrompt(task)` prepends the brief to a task's description. - **`orca.plan.Task(title, description, completed?)`** — `title` is the - human-readable label shown in the event log and used as the - `## Task: ` markdown header when persisted. -- **`orca.plan.PlanWithBrief(plan, brief)`** — a `Plan` plus a codebase brief - for the implementers, produced by `Sessioned.briefed`. Persisted as a trailing - `## Brief` section and prepended to each task by `taskPrompt`. + human-readable label shown in the event log. - **`orca.plan.Sessioned(sessionId, value)`** — every `Plan.{autonomous, interactive}.*` operation returns one: the result paired with the agent session that produced it, so the caller can continue that session into @@ -322,10 +421,15 @@ can generate them as structured output via `claude.resultAs[T]`: needs. - **`orca.plan.BugReportMatch`** — the agent's decision on whether a CI failure matches the original report. -- **`orca.Title`** — opaque alias of `String` shared by `Task.title` and - `ReviewIssue.title`. Construct via `Title("…")`; recover the string with - `.value`. Keeps short labels from being silently swapped with descriptions - or raw user input. +- **`orca.agents.SessionId[B]`** — typed session id, parameterised by backend. + Returned by `agent.session(seed)` and passed to `agent.runSeeded`. Carries the + backend identity at the type level, so you cannot accidentally pass a Claude + session to Codex. +- **`orca.Title`** — opaque `String` alias for short labels (`Task.title`, + `ReviewIssue.title`); `Title("…")` to construct, `.value` to read. +- **`orca.tools.PrHandle(owner, repo, number)`** — handle to an open pull + request, returned by `gh.createPr`. `derives JsonData` so a stage can record + it: a push-and-open-PR stage is the checkpoint before a CI wait. - **`orca.pr.PrSummary(title, body)`** — what `summarisePr` returns. The two fields feed `gh.createPr(title = …, body = …)` directly. - **`orca.review.ReviewIssue` / `ReviewResult`** — what reviewer agents return. @@ -338,6 +442,8 @@ can generate them as structured output via `claude.resultAs[T]`: - **`orca.review.IgnoredIssues`** — accumulated `IgnoredIssue(title, reason)` entries surfaced by `reviewAndFixLoop` once it halts. +</details> + ## Output While Orca runs the terminal output is split into two zones: an **event log** @@ -345,6 +451,9 @@ that grows top-to-bottom as stages and tools fire, and a **status line** pinned to the bottom, showing the active stage breadcrumb with a spinner. Nested stages are indented. +<details> +<summary>Glyph legend</summary> + | Glyph | Meaning | | ----- | ------- | | `▶` | Stage start, or a `Step` (single-line note like a branch switch) | @@ -355,6 +464,8 @@ are indented. | `✖` | Error | | `?` | Approval request | +</details> + Colours and animation auto-disable when stderr isn't a terminal. Set `NO_COLOR=1` or `ORCA_NO_ANIMATION=1` (suppresses the spinner) to force them off. @@ -365,7 +476,8 @@ Each CLI manages its own auth; Orca stores no secrets. Before running a flow, log in to the backend you use — `claude`, `codex`, `opencode`, or `pi` — and to `gh` (for the GitHub helpers), each per its own instructions. -For OpenCode with a local **Ollama** model, two options: +<details> +<summary>OpenCode with a local Ollama model</summary> - **Launcher (zero config):** `flow(OrcaArgs(args), opencodeLauncher = OpencodeLauncher.ollama("qwen3-coder"))`. Orca starts the server via `ollama @@ -377,6 +489,8 @@ For OpenCode with a local **Ollama** model, two options: models, `num_ctx` raised for tool use), then `opencode.withModel("ollama", "qwen3-coder")`. Supports several models and per-turn switching. +</details> + ## Getting set up Orca is published to Maven Central — `scala-cli` fetches the artifacts on first @@ -388,7 +502,9 @@ scala-cli run implement.sc -- "your task here" ## Documentation -- [`design.md`](design.md) — architecture and design rationale. +- [`design.md`](design.md) — architecture and design rationale (describes the + pre-0018 flow model; see [ADR 0018](adr/0018-stage-bound-flow-runtime.md) for + the current stage/session runtime design). - [`adr/`](adr/) — architecture decision records. - [`AGENTS.md`](AGENTS.md) — internals, conventions, build/test recipes; the same file AI assistants pick up. diff --git a/adr/0002-context-function-flow-dsl.md b/adr/0002-context-function-flow-dsl.md index 4260cd9e..38f54eb9 100644 --- a/adr/0002-context-function-flow-dsl.md +++ b/adr/0002-context-function-flow-dsl.md @@ -18,7 +18,7 @@ resolve against the command line rather than silently defaulting to empty), any number of named overrides (`git = ...`, `interaction = ...`, etc.), and the body as a second parameter list typed `FlowContext ?=> Unit`. Top-level `def claude(using FlowContext): -ClaudeTool`, `def git(using FlowContext): GitTool`, `def +ClaudeAgent`, `def git(using FlowContext): GitTool`, `def userPrompt(using FlowContext): String`, etc. resolve the ambient context implicitly. diff --git a/adr/0003-pluggable-llm-backends.md b/adr/0003-pluggable-llm-backends.md index b73f8c8e..25a18932 100644 --- a/adr/0003-pluggable-llm-backends.md +++ b/adr/0003-pluggable-llm-backends.md @@ -1,13 +1,13 @@ -# 0003. LLM backends are pluggable behind an `LlmBackend[B <: Backend]` trait +# 0003. LLM backends are pluggable behind an `AgentBackend[B <: Backend]` trait Status: Accepted · Date: 2026-04-22 (updated 2026-04-23) ## Decision -`LlmBackend[B]` exposes `runHeadless`, `continueHeadless`, +`AgentBackend[B]` exposes `runHeadless`, `continueHeadless`, `runInteractive`, and `continueInteractive`. The type parameter `B <: Backend` (e.g. `Backend.ClaudeCode.type`) makes `SessionId[B]`, -`LlmResult[B]`, and `Conversation[B]` phantom-typed so a Claude session +`AgentResult[B]`, and `Conversation[B]` phantom-typed so a Claude session id can't accidentally resume a Codex session. The Claude backend shells out to the `claude` CLI via a `CliRunner` @@ -42,5 +42,5 @@ Codex will run via WebSocket (sttp) on the same trait. `structured_output`. No filesystem sentinels; no platform assumptions. - Adding a non-CLI backend (e.g. a raw API client) stays possible - because `LlmBackend` isn't tied to `CliRunner` — only the Claude + because `AgentBackend` isn't tied to `CliRunner` — only the Claude backend is. diff --git a/adr/0004-module-layout.md b/adr/0004-module-layout.md index ba05bf08..252b52ed 100644 --- a/adr/0004-module-layout.md +++ b/adr/0004-module-layout.md @@ -7,7 +7,7 @@ Status: Accepted · Date: 2026-04-22 ``` tools → standalone — tool interfaces + os-backed impls + structured I/O + events flow → tools — FlowContext, stage/fail/fixLoop/reviewAndFix/lint, review types -claude → tools — Claude backend, DefaultClaudeTool, DefaultLlmCall +claude → tools — Claude backend, DefaultClaudeAgent, DefaultAgentCall codex → tools — Codex backend (future) runner → tools+flow+claude+codex — flow() entry, DefaultFlowContext, terminal layer ``` @@ -15,7 +15,7 @@ runner → tools+flow+claude+codex — flow() entry, DefaultFlowContext, termi Implementations live under `orca.tools.<capability>` sub-packages: `orca.tools.fs` (OsFsTool), `orca.tools.git` (OsGitTool), `orca.tools.github` (OsGitHubTool), `orca.tools.claude` (ClaudeBackend + -DefaultClaudeTool), `orca.tools.codex` (future). The top-level `orca` +DefaultClaudeAgent), `orca.tools.codex` (future). The top-level `orca` namespace stays reserved for the user-facing surface (traits, accessors, `flow`/`flowWith`, `JsonData`, `OrcaArgs`). diff --git a/adr/0005-flow-dsl-reshape.md b/adr/0005-flow-dsl-reshape.md index 24360962..99d56a23 100644 --- a/adr/0005-flow-dsl-reshape.md +++ b/adr/0005-flow-dsl-reshape.md @@ -17,7 +17,7 @@ bare `flow:` block: `orca.tools.git`, `orca.tools.github`. Top-level `orca` is reserved for the user-facing surface (traits, accessors, `flow`/`flowWith`, `JsonData`, `OrcaArgs`). -3. **Rename `LlmTool.result[O]` to `resultAs[O]` and retype it to +3. **Rename `Agent.result[O]` to `resultAs[O]` and retype it to `[O: JsonData]`** — the new name reads as a verb at the call site (`claude.resultAs[Plan].autonomous(...)`), the new bound keeps `JsonData` as the single typeclass users ever need to know about. @@ -25,7 +25,7 @@ bare `flow:` block: Schema/codec forwarders stay top-level in `package orca` but are invisible from the user's side unless nested `derives JsonData` macro expansion needs them. -4. **Add free-form text companions on `LlmTool`**: `ask` for one-shot +4. **Add free-form text companions on `Agent`**: `ask` for one-shot prompts, `startSession(prompt) → (id, reply)` and `continueSession(id, prompt) → reply` for multi-turn text flows. The structured `resultAs[O]` path remains for when responses should parse diff --git a/adr/0006-stream-json-conversation-driver.md b/adr/0006-stream-json-conversation-driver.md index f6160337..8370ce1b 100644 --- a/adr/0006-stream-json-conversation-driver.md +++ b/adr/0006-stream-json-conversation-driver.md @@ -1,7 +1,7 @@ # 0006. Drive Claude Code via stream-json instead of TTY handoff Status: Accepted · Date: 2026-04-23 -Amends: [ADR 0003](0003-pluggable-llm-backends.md) (backend surface) +Amends: [ADR 0003](0003-pluggable-agent-backends.md) (backend surface) Related: [ADR 0002](0002-context-function-flow-dsl.md) (flow DSL) ## Context @@ -36,7 +36,7 @@ text deltas instead of waiting for whole turns. ## Shape ``` -DefaultLlmCall.interactive +DefaultAgentCall.interactive └─ backend.runInteractive(prompt, config, workDir, schema) └─ spawnPiped → ClaudeConversation ├─ reader thread @@ -47,11 +47,11 @@ DefaultLlmCall.interactive ├─ events: Iterator[ConversationEvent] ├─ sendUserMessage(text) ├─ cancel() - └─ awaitResult(): LlmResult[B] + └─ awaitResult(): AgentResult[B] └─ interaction.drive(conversation) └─ TerminalConversationRenderer renders events; prompts for tool approvals - via JLine; returns LlmResult on success or + via JLine; returns AgentResult on success or throws OrcaInteractiveCancelled on cancel ``` @@ -61,17 +61,17 @@ Key shifts vs. the previous TTY path: claude carries the session id, usage, and (when `--json-schema` is passed) the validated `structured_output`. No transcript scraping. - **No stop hook files in `.claude/`.** The workspace is left - untouched; `prepareWorkspace` retired off the `LlmBackend` trait. + untouched; `prepareWorkspace` retired off the `AgentBackend` trait. - **The terminal is Orca's, not claude's.** Orca renders turns, streams text, displays tool calls, prompts for approvals, and decides what to show. The backend never inherits stdio. - **Approvals go through `ApproveTool` events**, each carrying a `respond: ApprovalDecision => Unit` closure the channel invokes exactly once. The driver auto-approves tools that match - `LlmConfig.autoApprove` before the event would fire; only + `AgentConfig.autoApprove` before the event would fire; only channel-level decisions surface as events. - **Cancellation surfaces as `Either`**: `Conversation.awaitResult` - returns `Either[OrcaInteractiveCancelled, LlmResult[B]]`. Genuine + returns `Either[OrcaInteractiveCancelled, AgentResult[B]]`. Genuine subprocess failures still throw, since they aren't recoverable, but user cancels are now in the type — direct callers can't accidentally ignore them. The `Interaction.drive` boundary is where the Either @@ -111,7 +111,7 @@ Key shifts vs. the previous TTY path: - The default prompt for interactive calls changed — no marker, no "emit `<<<ORCA_DONE>>>`" sentinel. Custom `Prompts` impls that relied on that convention need updating. -- `LlmBackend.runInteractive` / `continueInteractive` require an +- `AgentBackend.runInteractive` / `continueInteractive` require an explicit `outputSchema: Option[String]` argument. Callers pass the JSON Schema they expect the final turn to match; the backend forwards to `--json-schema`. `None` is legal for free-form text. @@ -131,7 +131,7 @@ Key shifts vs. the previous TTY path: Adopting now means writing the stream-json layer anyway and adding a JSON-RPC protocol layer on top. Deferred to a later migration: once stream-json is proven, add an ACP client and - switch backend dispatch at the `LlmBackend` seam. + switch backend dispatch at the `AgentBackend` seam. - **Keep TTY handoff.** The fragile part. Blocks channel abstraction — Slack/HTTP can't inherit a terminal. Rejected. diff --git a/adr/0007-codex-exec-jsonl-driver.md b/adr/0007-codex-exec-jsonl-driver.md index 3930ed75..bb5530e0 100644 --- a/adr/0007-codex-exec-jsonl-driver.md +++ b/adr/0007-codex-exec-jsonl-driver.md @@ -1,7 +1,7 @@ # 0007. Drive Codex via `codex exec --json` stdio JSONL Status: Accepted · Date: 2026-04-24 -Amends: [ADR 0003](0003-pluggable-llm-backends.md) (backend surface) +Amends: [ADR 0003](0003-pluggable-agent-backends.md) (backend surface) Related: [ADR 0006](0006-stream-json-conversation-driver.md) (Claude stream-json driver) ## Context @@ -85,12 +85,12 @@ Concretely: - `CodexBackend.runHeadless` / `continueHeadless` spawn `codex exec` (or `codex exec resume <id>`) with `--json`, consume the JSONL stream, and extract `thread_id` + final `agent_message` text + - `usage` from `turn.completed` into an `LlmResult`. Note the + `usage` from `turn.completed` into an `AgentResult`. Note the divergence from ADR 0006's claude shape: claude emits an explicit terminal `result` message that carries `output` / `structured_output` fields the driver reads directly. Codex has no such terminal message — its `turn.completed` only carries usage. The driver - therefore *synthesises* `LlmResult.output` by snapshotting the + therefore *synthesises* `AgentResult.output` by snapshotting the text of the last `item.completed` of type `agent_message` seen before `turn.completed`. The prompt template instructs the agent to make that final message JSON-only, so the snapshot is the @@ -114,7 +114,7 @@ Concretely: `ToolResult(toolName="bash", ok=(exit_code==0), content=aggregated_output)`. - `item.{started,completed}` with `type: "file_change"` → `AssistantToolCall(name="file_change", …)` / `ToolResult(…)`. - - `turn.completed` → record usage; finalize `LlmResult`. + - `turn.completed` → record usage; finalize `AgentResult`. - Unknown item types → dropped (forward-compat). Consequences of the JSONL limitations: @@ -123,7 +123,7 @@ Consequences of the JSONL limitations: single-shot stream-json path). Multi-turn interactive requires `continueInteractive(sessionId, …)` which spawns a fresh `codex exec resume`. -- **`ApproveTool` events are not emitted.** `LlmConfig.autoApprove` +- **`ApproveTool` events are not emitted.** `AgentConfig.autoApprove` is pre-baked into the spawn args: - `AutoApprove.All` → `--dangerously-bypass-approvals-and-sandbox` (or `--full-auto` if the caller prefers a sandbox). diff --git a/adr/0009-announce-typeclass.md b/adr/0009-announce-typeclass.md index 7b98ab7d..0ef16f7d 100644 --- a/adr/0009-announce-typeclass.md +++ b/adr/0009-announce-typeclass.md @@ -59,14 +59,14 @@ enum OrcaEvent: - **Catch-all default given returns `""`.** Every type has an `Announce[O]` resolvable — the empty string is the contract for - "no announcement to make". `DefaultLlmCall` normalises the + "no announcement to make". `DefaultAgentCall` normalises the empty-string sentinel to `None` so listeners can pattern-match on `summary` cleanly. - **Specific instances win via Scala 3's specificity rules.** `given Announce[Plan] = Announce.from(...)` in `Plan`'s companion is more specific than the catch-all and resolves preferentially. -- **`LlmCall.resultAs[O]` adds the bound:** - `def resultAs[O: JsonData : Announce]`. `DefaultLlmCall` consumes +- **`AgentCall.resultAs[O]` adds the bound:** + `def resultAs[O: JsonData : Announce]`. `DefaultAgentCall` consumes the instance via a `using` parameter and emits a single `StructuredResult(raw, summary)` event after parsing — on every invocation variant (`autonomous`, `startSession`, `continueSession`, @@ -124,13 +124,13 @@ The typeclass won on three counts: anything extra. - **Renderer is dumber.** ~30 lines of suppression heuristic removed; the renderer now flushes whatever the agent emitted. -- **Test stubs propagate the bound.** Every hand-rolled `LlmTool` - or `LlmCall` in tests grew `: Announce` on its `resultAs[O]` +- **Test stubs propagate the bound.** Every hand-rolled `Agent` + or `AgentCall` in tests grew `: Announce` on its `resultAs[O]` override. Mechanical churn, but a real consequence of widening the structural surface. - **Module split.** The typeclass + default + `from` helper sit in `tools/`; the `value.announce` extension sits in `flow/`. A - callsite that only needs the bound (e.g. `DefaultLlmCall`) + callsite that only needs the bound (e.g. `DefaultAgentCall`) imports from `tools/` and doesn't pull in `FlowContext`. - **The empty-string contract is load-bearing.** `Announce.from(_ => "")` and an absent specific given are observationally identical. A diff --git a/adr/0010-prompts-and-helpers-convention.md b/adr/0010-prompts-and-helpers-convention.md index 1b9ff399..c02c6f1a 100644 --- a/adr/0010-prompts-and-helpers-convention.md +++ b/adr/0010-prompts-and-helpers-convention.md @@ -11,7 +11,7 @@ Orca ships two kinds of LLM-using surface: 2. **Domain helpers** — flow-script-friendly functions that bundle a domain-specific multi-step pattern with a default LLM brief: planning (`Plan.interactive.*`, `Plan.autonomous.*`), review (`reviewAndFixLoop`, - `lint`, `ReviewerSelector.llmDriven`), the canonical reviewer set + `lint`, `ReviewerSelector.agentDriven`), the canonical reviewer set (`allReviewers`, `minimalReviewers`). Before this ADR, prompts in domain helpers lived in three different shapes: @@ -50,7 +50,7 @@ For every domain helper that bundles an LLM brief, follow this pattern: ```scala def lint( command: String, - llm: LlmTool[?], + agent: Agent[?], instructions: String = ReviewLoopPrompts.SummariseLint )(using FlowContext): ReviewResult ``` diff --git a/adr/0012-mcp-host-bridge.md b/adr/0012-mcp-host-bridge.md index 7151917d..b772b71a 100644 --- a/adr/0012-mcp-host-bridge.md +++ b/adr/0012-mcp-host-bridge.md @@ -120,8 +120,8 @@ with many sequential interactive calls doesn't accumulate bindings. ### Auto-approval The MCP tool name (`mcp__orca__ask_user`) is unioned into -`LlmConfig.autoApprove` for the conversation only — via -`LlmConfig.withAlsoAllowedTool`. The user is already typing an answer; +`AgentConfig.autoApprove` for the conversation only — via +`AgentConfig.withAlsoAllowedTool`. The user is already typing an answer; a y/n approval prompt before that would be pure noise. ### Render suppression diff --git a/adr/0013-persistent-plans.md b/adr/0013-persistent-plans.md index afb11a37..e159ef8e 100644 --- a/adr/0013-persistent-plans.md +++ b/adr/0013-persistent-plans.md @@ -1,5 +1,9 @@ ## ADR 0013: Persistent plans as the default for multi-task flows +Status: Superseded by [ADR 0018](0018-stage-bound-flow-runtime.md) · 2026-06-23 — +the stage-bound runtime makes the progress log the universal resume mechanism, +retiring `Plan.recover` / `.orca/plan-<hash>.md` / checkbox-ticking described below. + ## Context The library originally split planning into two shapes: diff --git a/adr/0014-opencode-server-driver.md b/adr/0014-opencode-server-driver.md index 672c1c4b..ae942c11 100644 --- a/adr/0014-opencode-server-driver.md +++ b/adr/0014-opencode-server-driver.md @@ -8,7 +8,7 @@ Proposed — implementation plan. ## Context -We want OpenCode (`sst/opencode`, binary `opencode`, v1.15.x) as a third `LlmTool` +We want OpenCode (`sst/opencode`, binary `opencode`, v1.15.x) as a third `Agent` backend alongside Claude and Codex. The existing two backends are **CLI drivers**: each turn spawns a one-shot subprocess (`claude --print` stream-json, ADR 0006; `codex exec --json` JSONL, ADR 0007) and parses its stdout. @@ -57,7 +57,7 @@ Session-scoped events carry `properties.sessionID`; server-level ones 1. **The result lives in the event stream.** The final `message.updated` carries the validated `structured` payload + `tokens`, so a turn is started with - `prompt_async` and its `LlmResult` is read from the SSE stream — the same + `prompt_async` and its `AgentResult` is read from the SSE stream — the same single-stream shape Codex gets from `turn.completed`, so the existing `StreamConversation` machinery applies. 2. **`ask_user` is native** (`question.asked` event + `/question/{id}/reply`). The @@ -76,7 +76,7 @@ down on Ox-scope close. use our own ids: `POST /session` has no `id` field (verified — server always mints `ses_…`). And even though the autonomous path could carry the server id back, the framework's **interactive** path pins the caller's `SessionId` and reconciles via -`backend.registerSession(client, server)` (`DefaultLlmCall` returns the caller's +`backend.registerSession(client, server)` (`DefaultAgentCall` returns the caller's `session`, not `result.sessionId`) — so the mapping lives at the framework level regardless. Use `SessionRegistry.ClientToServer` + the `registerSession` override, exactly like Codex; return `result.copy(sessionId = session)` to keep the caller's @@ -87,7 +87,7 @@ handle stable. Add an `opencode` module with an `OpencodeBackend` that drives a shared `opencode serve` over HTTP+SSE. Each turn starts with `POST …/prompt_async` and reads its **own** `GET /event` SSE stream as the single source — translating bus -events to `ConversationEvent`s and deriving the `LlmResult` from `message.updated` +events to `ConversationEvent`s and deriving the `AgentResult` from `message.updated` (native `format` gives structured output on `info.structured`) — by **reusing the existing `StreamConversation` machinery**. `ask_user` is answered natively (no MCP bridge). @@ -106,27 +106,27 @@ opencode/src/main/scala/orca/tools/opencode/ OpencodeServer.scala // shared serve process: start, base URL, HTTP wiring, teardown OpencodeHttp.scala // transport trait: postJson, events (SSE), close JavaNetOpencodeHttp.scala // OpencodeHttp over java.net.http (HTTP/1.1, basic auth) - OpencodeBackend.scala // LlmBackend[BackendTag.Opencode.type] + OpencodeBackend.scala // AgentBackend[BackendTag.Opencode.type] OpencodeConversation.scala // one turn: SSE line source → StreamConversation; prompt_async; ask_user/permission replies OpencodeApi.scala // request/response DTOs + jsoniter codecs (message body, SSE event envelope, AssistantMessage) - OpencodeArgs.scala // serve argv + message-body assembly from LlmConfig + OpencodeArgs.scala // serve argv + message-body assembly from AgentConfig OpencodeModel.scala // provider/model id construction + split - DefaultOpencodeTool.scala // OpencodeTool: provider-prefixed accessors, withModel, copyTool + DefaultOpencodeAgent.scala // OpencodeAgent: provider-prefixed accessors, withModel, copyTool adr/0014-opencode-server-driver.md ``` ## Shared-type changes (in `tools`) -- `orca.llm.BackendTag`: add `case Opencode`. -- `orca.llm.LlmTool`: add `trait OpencodeTool extends LlmTool[BackendTag.Opencode.type]` +- `orca.agents.BackendTag`: add `case Opencode`. +- `orca.agents.Agent`: add `trait OpencodeAgent extends Agent[BackendTag.Opencode.type]` with provider-prefixed model accessors — Anthropic (`anthropicOpus`/`anthropicSonnet`/`anthropicHaiku`) and OpenAI (`openaiGpt5`/`openaiGpt5Codex`/`openaiGpt5Mini`), plus a public `withModel(providerModel: String)` for any other `provider/model` (names below). -- `orca.llm.CanAskUser`: add `given CanAskUser[BackendTag.Opencode.type]` — the +- `orca.agents.CanAskUser`: add `given CanAskUser[BackendTag.Opencode.type]` — the server supports native questions on interactive turns. -- No change to `LlmBackend`, `Conversation`, `LlmResult`, `Usage`, `DefaultLlmCall`, - `BaseLlmTool` — OpenCode fits the existing SPI. +- No change to `AgentBackend`, `Conversation`, `AgentResult`, `Usage`, `DefaultAgentCall`, + `BaseAgent` — OpenCode fits the existing SPI. ## Backend design @@ -158,7 +158,7 @@ concurrent first calls). Responsibilities: Each `OpencodeConversation` owns one `GET /event` SSE connection and starts its turn with `POST …/prompt_async` (`204`). The SSE stream is the single source: the reader fork translates events to `ConversationEvent`s (mapping under *Progress events*) and -derives the `LlmResult` from the final `message.updated` (`info.structured` + +derives the `AgentResult` from the final `message.updated` (`info.structured` + `info.tokens`) at `session.idle` — the single-stream shape Codex gets from `turn.completed`. @@ -202,14 +202,14 @@ ships accessors for more than one family: are a detail — any id from `opencode models` is valid; update as the catalog moves). -Accessors are **provider-prefixed** (unlike `ClaudeTool`'s bare `opus`/`sonnet`, +Accessors are **provider-prefixed** (unlike `ClaudeAgent`'s bare `opus`/`sonnet`, which can assume a single vendor) because OpenCode spans providers — the prefix keeps the model's provider explicit at the call site and leaves room for more families. Plus a public `withModel(providerModel: String)` for any other `provider/model` (e.g. `ollama/llama3.1`, `myhost/qwen-coder`). `Model` is an opaque `String` with no allowlist, so arbitrary ids pass straight through to the message body. -`LlmConfig.model = None` lets the server use its configured default. `variant` +`AgentConfig.model = None` lets the server use its configured default. `variant` (reasoning effort) maps from a future config field; out of scope for v1. **Model-identifier utilities.** For self-hosted / custom providers, provide a tiny @@ -231,7 +231,7 @@ object OpencodeModel: case _ => throw OrcaFlowException(s"not a provider/model id: ${Model.name(m)}") ``` -`OpencodeTool.withModel(providerModel)` and the prefixed accessors both go through +`OpencodeAgent.withModel(providerModel)` and the prefixed accessors both go through `OpencodeModel`; `OpencodeArgs` uses `split` to fill `model:{providerID, modelID}`. The split-on-first-`/` matters: self-hosted model ids often contain slashes (`lmstudio/google/gemma-3n-e4b`). @@ -302,7 +302,7 @@ round-trip works end-to-end (agent unblocks with the chosen label), via `POST Both `runAutonomous` and `runInteractive` build an `OpencodeConversation` (per *Event stream*: own SSE connection + `prompt_async`, result derived from the stream); `runAutonomous` then drains it with `Conversations.drainAutonomous` -(→ `OrcaListener` progress + the `LlmResult`), exactly as the Codex backend does. +(→ `OrcaListener` progress + the `AgentResult`), exactly as the Codex backend does. ### runAutonomous @@ -314,7 +314,7 @@ stream); `runAutonomous` then drains it with `Conversations.drainAutonomous` prompt, `format` from `outputSchema`, `tools` per `AutoApprove`/`readOnly` **with `question:false`** (autonomous never asks — see *Asking the user*). 3. `Conversations.drainAutonomous(conv, events)` → emits `OrcaEvent`s and returns - the `LlmResult` the reader built from `message.updated`/`session.idle` + the `AgentResult` the reader built from `message.updated`/`session.idle` (`output` = `info.structured` serialised when `outputSchema` set, else accrued text; `usage` = `info.tokens`; `model` = `info.modelID`; `info.error` → `AgentTurnFailed`). @@ -326,7 +326,7 @@ stream); `runAutonomous` then drains it with `Conversations.drainAutonomous` Return the live `OpencodeConversation`: - `events`: the `EventQueue` iterator (single-consumer), terminating on `session.idle`/`session.error`. -- `awaitResult()`: joins the reader → the `LlmResult` it built from +- `awaitResult()`: joins the reader → the `AgentResult` it built from `message.updated`/`session.idle` (as *runAutonomous* step 3); inherited verbatim from `StreamConversation`. `Left(OrcaInteractiveCancelled)` after `cancel()`. - `sendUserMessage(text)`: starts a **new** turn — `POST /session/{id}/message` on @@ -344,7 +344,7 @@ Return the live `OpencodeConversation`: permission config (below) pre-answers so no prompt fires. `registerSession` override records client→server id (called post-drive by -`DefaultLlmCall`, as for Codex). +`DefaultAgentCall`, as for Codex). ### Progress events (display) — reusing `drainAutonomous` @@ -377,10 +377,10 @@ model/agent switches in the spike). Drive off the `message.part` lifecycle + | `message.part.delta` `{field:"reasoning", delta}` | `AssistantThinkingDelta(delta)` (dropped by autonomous drain) | | `message.part.updated` `{part:{type:"tool", tool, state:{status:"running", input}}}` | `AssistantToolCall(tool, input)` → `ToolUse` | | `message.part.updated` `{part:{type:"tool", state:{status:"completed"/"error", output}}}` | `ToolResult(tool, ok, output)` (dropped: volume) | -| `message.updated` (assistant) | capture `info` (`structured`, `tokens`, `modelID`, `finish`, `error`) — source of the `LlmResult` | +| `message.updated` (assistant) | capture `info` (`structured`, `tokens`, `modelID`, `finish`, `error`) — source of the `AgentResult` | | `question.asked` `{questions[], …}` | `UserQuestion(q, respond)` — `respond` POSTs `/question/{id}/reply` (interactive; autonomous gates the tool off) | | `permission.asked` `{id:"per_…", permission, patterns, tool}` | `ApproveTool(permission, patterns, respond)` — `respond` POSTs `/permission/{id}/reply` `{reply:"once"\|"always"\|"reject"}` (autonomous drain auto-denies) | -| `session.idle` `{sessionID}` | build `LlmResult` from captured `info`, CAS `outcomeRef`, `AssistantTurnEnd`, close queue | +| `session.idle` `{sessionID}` | build `AgentResult` from captured `info`, CAS `outcomeRef`, `AssistantTurnEnd`, close queue | | `session.error` / `info.error` | `Outcome.Failed(AgentTurnFailed)`, close queue | Ignore `session.status`/`session.updated`/`session.diff`/`session.next.*`/ @@ -403,7 +403,7 @@ its reused `EventQueue`/`Outcome`). Beyond that: ### Config mapping (`OpencodeArgs`) -| `LlmConfig` | OpenCode | +| `AgentConfig` | OpenCode | |---|---| | `model` | message `model: {providerID, modelID}` (split on first `/`) | | `systemPrompt` | message `system` (native field — no prompt-folding needed, unlike Codex) | @@ -441,15 +441,15 @@ object at `state.input` with `state.metadata.valid == true`. So the backend read `info.structured` for the payload (not text parts). Free-form turns instead carry a `text` part and `finish == "stop"`. -The existing `DefaultLlmCall` retry-on-bad-JSON loop remains as a backstop (and +The existing `DefaultAgentCall` retry-on-bad-JSON loop remains as a backstop (and covers `retryCount` exhaustion → `StructuredOutputError`). This is strictly better than Codex's resume path, which falls back to prompt-only enforcement. ## Wiring -`DefaultFlowContext.withDefaults` gains an `opencode: Option[OpencodeTool] = None` -param, defaulting to `new DefaultOpencodeTool(new OpencodeBackend(OsProcCliRunner), -LlmConfig.default, prompts, workDir, dispatcher, interaction)`. Expose it on +`DefaultFlowContext.withDefaults` gains an `opencode: Option[OpencodeAgent] = None` +param, defaulting to `new DefaultOpencodeAgent(new OpencodeBackend(OsProcCliRunner), +AgentConfig.default, prompts, workDir, dispatcher, interaction)`. Expose it on `FlowContext` next to `claude`/`codex`. ## Implementation steps @@ -513,18 +513,18 @@ LlmConfig.default, prompts, workDir, dispatcher, interaction)`. Expose it on 2. (Optional but recommended) generalise `StreamConversation` from `PipedCliProcess` to a small line-source abstraction (`lines`, `cancel`, terminal signal) so both subprocess and SSE backends share it; else reuse `EventQueue`/`Outcome` directly. -3. `opencode` sbt module + deps; add `BackendTag.Opencode`, `OpencodeTool`, +3. `opencode` sbt module + deps; add `BackendTag.Opencode`, `OpencodeAgent`, `CanAskUser` given, `OpencodeModel`. 4. `OpencodeApi` DTOs + jsoniter codecs (message body, SSE envelope, AssistantMessage) from the captured payloads. 5. `OpencodeServer`: spawn/health/teardown/auth + HTTP client (incl. streaming `GET /event`). 6. `OpencodeArgs`: serve argv + message-body assembly. -7. `OpencodeConversation`: SSE line source → events + `LlmResult` from +7. `OpencodeConversation`: SSE line source → events + `AgentResult` from `message.updated`/`session.idle`; ask_user/permission replies; abort. 8. `OpencodeBackend.runAutonomous` (`prompt_async` + `Conversations.drainAutonomous`). 9. `OpencodeBackend.runInteractive` + `registerSession`. -10. `DefaultOpencodeTool` + `DefaultFlowContext`/`FlowContext` wiring. +10. `DefaultOpencodeAgent` + `DefaultFlowContext`/`FlowContext` wiring. 11. Tests (below). ## Testing @@ -536,14 +536,14 @@ Mirror the existing three-layer pattern (`*BackendTest` fakes, flow-level stubs, is HTTP not subprocess, so instead of `FakePipedCliProcess`/`SpawnStubCliRunner`, feed a canned list of SSE `data:` lines (captured in *Spike results*) into the conversation's line source and assert the `ConversationEvent` sequence + the - `LlmResult` (structured object, `tokens`, model, `finish`, `info.error` → + `AgentResult` (structured object, `tokens`, model, `finish`, `info.error` → `AgentTurnFailed`). Separately unit-test `OpencodeArgs.message` body assembly (model split incl. multi-slash ids, `format`, `system`, `readOnly` tools, `question:false`) and `OpencodeModel.split`. Inject a stub sttp backend for the `prompt_async`/reply POSTs and assert the request bodies. - **Flow-level e2e (no server) — a simple `implement.sc`-like plan via OpenCode.** Run the real `Plan` DSL (`Plan.autonomous.from` + `implementTaskLoop`) against a - `TestFlowContext` whose `opencode` is a scripted stub (the `CannedResultLlm` / + `TestFlowContext` whose `opencode` is a scripted stub (the `CannedResultAgent` / `OrcaOverridesTest` pattern): assert a 1-task plan is produced and the task loop drives `opencode.autonomous.run`/`resultAs`. Exercises the wiring end-to-end without a live server. diff --git a/adr/0015-gemini-stream-json-driver.md b/adr/0015-gemini-stream-json-driver.md index db4722bc..283f7e0d 100644 --- a/adr/0015-gemini-stream-json-driver.md +++ b/adr/0015-gemini-stream-json-driver.md @@ -1,14 +1,14 @@ # 0015. Drive Gemini CLI via `gemini --output-format stream-json` stdio JSONL Status: Accepted · Date: 2026-06-01 -Amends: [ADR 0003](0003-pluggable-llm-backends.md) (backend surface) +Amends: [ADR 0003](0003-pluggable-agent-backends.md) (backend surface) Related: [ADR 0006](0006-stream-json-conversation-driver.md) (Claude stream-json driver), [ADR 0007](0007-codex-exec-jsonl-driver.md) (Codex exec JSONL driver), [ADR 0012](0012-mcp-host-bridge.md) (ask_user MCP bridge), [ADR 0014](0014-opencode-server-driver.md) (OpenCode server driver) ## Context Orca already ships two LLM backends — Claude (bidirectional stream-json, ADR 0006) and Codex (one-shot `exec --json`, ADR 0007). Adding Google's -`gemini` CLI as a third backend follows the established `LlmBackend[B]` +`gemini` CLI as a third backend follows the established `AgentBackend[B]` SPI. The open question was which of gemini's headless output shapes to drive and how closely it maps to the existing infrastructure. @@ -38,7 +38,7 @@ Key differences from the existing backends: 1. **No single terminal payload.** Unlike Claude's terminal `result` message (which carries `output`/`structured_output`), gemini's `result` event carries only status + stats. Like Codex, the driver - **synthesises** `LlmResult.output` — but where Codex snapshots the + **synthesises** `AgentResult.output` — but where Codex snapshots the *last* `agent_message`, gemini streams assistant prose as `message` chunks, so the driver *accumulates* the content of every non-`user` message and reads the buffer at the `result` event. @@ -52,7 +52,7 @@ Key differences from the existing backends: the only workable non-read-only mapping is `yolo`. 4. **No `--output-schema` flag.** Gemini has no native structured-output gate. `resultAs[O]` enforcement is prompt-template + the post-hoc - `DefaultLlmCall` corrective-retry loop only; `outputSchema` is still + `DefaultAgentCall` corrective-retry loop only; `outputSchema` is still threaded to `GeminiConversation` so the autonomous drain suppresses the raw JSON payload from the user log. 5. **Session resume** works via `gemini --resume <session-id> -p …`, @@ -107,7 +107,7 @@ Concretely: only the id) be keyed by name. - `tool_result` → `ToolResult(name, ok = status == "success", output)`. - `error` → `ConversationEvent.Error`. - - `result` → emit `AssistantTurnEnd`, then finalise `LlmResult` with the + - `result` → emit `AssistantTurnEnd`, then finalise `AgentResult` with the accumulated answer + usage. - Unknown top-level types → dropped (forward-compat). - **System prompt fold.** Gemini has no `--append-system-prompt`, so the @@ -117,7 +117,7 @@ Concretely: - `readOnly = true` → `--approval-mode plan` - `AutoApprove.All` → `--approval-mode yolo` - `AutoApprove.Only(_)` → `--approval-mode yolo` (widened — see below) -- **Models.** `GeminiTool.flash` pins `gemini-2.5-flash`; bare `gemini` +- **Models.** `GeminiAgent.flash` pins `gemini-2.5-flash`; bare `gemini` pins `gemini-2.5-pro` in the runtime wiring (mirroring claude's Opus default for the long-lived implementer). Model literals may rename in future CLI versions; override via `withConfig`. @@ -183,6 +183,6 @@ Two sharp edges, accepted: restore, file removal when none existed. - `GeminiBackendTest` — session id / usage extraction, resume dispatch, systemPrompt fold, MCP registration on interactive (autonomous skips it). -- `DefaultGeminiToolTest` — `flash`/pro model pins reach `--model`. +- `DefaultGeminiAgentTest` — `flash`/pro model pins reach `--model`. - `GeminiIntegrationTest` (gated on `ORCA_INTEGRATION`) — real `gemini`: headless round-trip and a resumed turn that recalls prior context. diff --git a/adr/0016-toolset-capability-axis-and-planner-network.md b/adr/0016-toolset-capability-axis-and-planner-network.md index c3702672..5fd070bc 100644 --- a/adr/0016-toolset-capability-axis-and-planner-network.md +++ b/adr/0016-toolset-capability-axis-and-planner-network.md @@ -1,7 +1,7 @@ # 0016. `ToolSet` capability axis and planner read-only network access Status: Accepted · Date: 2026-06-11 -Related: [ADR 0003](0003-pluggable-llm-backends.md) (backend surface), [ADR 0011](0011-reviewer-roster.md) (reviewers run read-only) +Related: [ADR 0003](0003-pluggable-agent-backends.md) (backend surface), [ADR 0011](0011-reviewer-roster.md) (reviewers run read-only) ## Context @@ -12,7 +12,7 @@ planner needs to read an issue/PR it was pointed at: claude's can't answer), codex's `--sandbox read-only` blocks all network, pi's read-only `--tools` has no web tool, gemini's `--approval-mode plan` gates web/shell. -Capability was previously encoded as a boolean `LlmConfig.readOnly` layered over +Capability was previously encoded as a boolean `AgentConfig.readOnly` layered over the `AutoApprove` enum, munged together in each backend's args mapping. Two problems: (1) the boolean couldn't express "read-only **plus** network"; (2) `withReadOnly` is the shared hard no-edit gate for *seven* turn kinds (two @@ -22,7 +22,7 @@ mid-review — so network must not be tied to "read-only" in general. ## Decision -Replace `readOnly: Boolean` with a capability enum on `LlmConfig`: +Replace `readOnly: Boolean` with a capability enum on `AgentConfig`: ```scala enum ToolSet: case ReadOnly, NetworkOnly, Full @@ -59,7 +59,7 @@ disabled set, so web may work (server-dependent, unverified). The claude network allowlist (`--allowedTools` strings like `Bash(gh api:*)`) is claude-specific, so it lives on `ClaudeBackend` (default -`DefaultNetworkTools`), not the shared `LlmConfig`. It is configurable per flow +`DefaultNetworkTools`), not the shared `AgentConfig`. It is configurable per flow via `claude.withNetworkTools(...)`. The default includes `Bash(gh api:*)` for broad GitHub reads — note `gh api -X POST` can mutate GitHub (not local files); flows wanting a tighter set override it. diff --git a/adr/0018-stage-bound-flow-runtime.md b/adr/0018-stage-bound-flow-runtime.md new file mode 100644 index 00000000..8d95859f --- /dev/null +++ b/adr/0018-stage-bound-flow-runtime.md @@ -0,0 +1,770 @@ +# 0018. Stage-bound flow runtime: resumable stages, capabilities, progress log + +Status: Accepted · Date: 2026-06-23 +Supersedes: [ADR 0013](0013-persistent-plans.md) (persistent plans) +Related: [ADR 0002](0002-context-function-flow-dsl.md) (flow DSL / `FlowContext`), +[ADR 0003](0003-pluggable-agent-backends.md) (backend SPI), +[ADR 0016](0016-toolset-capability-axis-and-planner-network.md) (capability axis) + +A `flow(...)` run is bound to a branch and a progress log, and is composed of +**stages**: named, resumable, committing units of work. A stage records its result, +commits its changes, and is skipped on a re-run if already complete — so a flow that +fails partway through resumes from the first incomplete stage rather than repeating +finished work. This generalises the per-task persistence the example flows hand-roll +today (`Plan.recover` + `implementTaskLoop`) up to the level of every side-effecting +step. This ADR is the complete design record; the only code below is illustrative +*flow scripts* and signatures (no implementation yet). + +## 1. Context + +Flows are long-running: a single run churns over a task list for many minutes to +hours, with agent turns, reviews, and CI waits throughout. At that duration, +interruption is the norm, not the exception — a dropped network connection, a killed +process, a hit rate limit, a machine sleeping. The goal is that **interrupting a flow +mid-run costs at most the in-progress stage**: everything already finished is recorded +and committed, and re-running the same command picks up from the first incomplete +stage rather than re-planning, re-implementing, and re-paying for work that already +landed. Restartability is the feature; stages are the unit at which it is guaranteed. + +Today only *multi-task* flows resume, via a bespoke mechanism (ADR 0013): +`Plan.recover` + `implementTaskLoop` + a committed `.orca/plan-<hash>.md` whose +checkboxes track progress. Non-task phases (triage, failing-test, CI-wait, +repro-verification) are not resumable — `issue-pr-bugfix.sc` uses the in-memory task +loop precisely because those phases "aren't restartable from a plan file alone." The +resume logic is hand-rolled per flow. + +Backward compatibility is explicitly **not** a goal. Orca is early and has very few +users, so this design is free to break existing tool signatures and flow scripts +where that yields a cleaner result (the `InStage` change in particular touches every +flow). + +--- + +## 2. Requirements & design (the decision) + +Each subsection lists the requirements it must meet — the **R**n labels are stable +identifiers referenced elsewhere in this document, grouped by topic rather than +ordered numerically — followed by the design that meets them. (Where a requirement and +its mechanism say the same thing the requirement is kept terse and the detail lives in +the design.) + +**Cross-cutting requirements** (hold throughout): +- **R27** — The API is type-safe: the capabilities (`FlowContext`, `FlowControl`, + `InStage`), stage results (`JsonData`), and session ids (`SessionId[B]`, + parameterised by backend) are checked at compile time; the runtime never relies on + stringly-typed or untyped state where a type would do. +- **R28** — The on-disk serialisation format is JSON throughout (the progress log, + stage results, the header), via jsoniter codecs. + +### 2.1 Stages + +**Requirements.** +- **R7** — The stage is the unit of work: a named region that, on success, records + its return value in the progress log. +- **R8** — Every stage commits on completion (progress-log delta + code changes) as + one commit. orca force-adds its progress file (`git add -f` of the single + progress-log path — never a glob or directory, so nothing else gitignored is swept + in), so it is committed even when the project gitignores `.orca/`. Projects are in + fact encouraged to gitignore `.orca/` — that keeps orca's scratch state out of `git + status`, while the force-add still tracks the one file that must travel with the + branch. The commit is therefore never empty and the log stays on the branch. +- **R10** — A stage's id is its name plus an occurrence index (disambiguating + duplicates), never a global execution counter. +- **R11** — On resume, a stage whose recorded result decodes to its call-site type + is skipped (stored value returned); a result that fails to decode (the stage's + type changed) re-runs the stage — fail-safe over silent misattribution. +- **R12** — Stages may nest on one thread (each nested stage commits) but must not + run concurrently — two stages committing at once would corrupt the shared git + index. This is structural: starting a stage needs `FlowControl` (R29), and + concurrency combinators hand forks only the thread-safe `FlowContext`, never + `FlowControl`, so a fork (e.g. a parallel reviewer) cannot start a stage. It is a + convention today (a fork could still lexically capture an outer `FlowControl`) and + becomes compiler-enforced under capture checking. +- **R13** — The commit message defaults to an `agent.cheap` summary of the diff; a + stage may pass `commitMessage: Option[T => String]` to derive it from its result. +- **R14** — `display(...)` gives progress-only output: no stage, id, commit, or log + entry. + +**Design.** + +```scala +def stage[T: JsonData](name: String, commitMessage: Option[T => String] = None) + (body: InStage ?=> T)(using FlowControl): T +def display(message: String)(using FlowContext): Unit +def fail(message: String)(using FlowContext): Nothing +``` + +A `stage` runs `body` with `InStage` evidence in scope, then records and commits: + +1. **Resume check.** Compute the id. If the log holds a completed entry for it, + decode the stored JSON with `JsonData[T].codec` and return it without running + `body`. If decoding fails (the script changed the stage's result type under this + id), run `body` instead — fail-safe over feeding back a wrong value. +2. **Run.** `body` executes; its mutating operations consume the supplied + `InStage`. +3. **Record & commit.** Append a `StageEntry` (id, name, result JSON), force-add the + log, then make one commit covering the log delta plus any code changes. The + message is `commitMessage(result)` if the stage supplied one, otherwise an + `agent.cheap` summary of the changed files. The log always changes, so the commit + is never empty. + +`display` shows a progress line without checkpointing; `fail` emits an error and +throws, unwinding to the failure teardown (§2.5). Both need only `FlowContext`, so +they're callable anywhere — outside a stage, or inside a fork. + +Resume is uniform across stage kinds: an interactive stage (a planning +conversation the user drove) is skipped on resume just like any other, returning +its stored result — the user is not re-prompted for work already done. + +**Id.** `id = name + "#" + occurrenceIndex`, where the index counts prior stages +with the same name in this run. Because the key is the name rather than an +execution-order counter, inserting, removing, or reordering *other* stages between +runs does not shift a stage's id: a removed stage leaves a harmless orphan entry; a +newly inserted stage simply has no recorded result and runs. A stage whose dynamic +name changes between runs (e.g. a task title from a regenerated plan) re-runs — +fail-safe. + +**Nesting.** Nested stages each commit, so nest only to introduce an extra +checkpoint; wrapping a stage solely around other stages yields a commit carrying just +that stage's progress entry. Why two stages can't run concurrently — the +`FlowControl`-vs-`FlowContext` capability split — is in §2.2 (R12). + +### 2.2 Capability gating + +**Requirements.** +- **R15** — Every side-effecting operation — file writes, git mutations, GitHub + mutations, and **all** LLM calls — requires `InStage` evidence, which only the + runtime can construct and which a stage body receives as a context parameter. +- **R16** — Pure reads (`fs.read`, `git.diff` / `log` / `currentBranch`, + `gh.readIssue` / `buildStatus` / `waitForBuild`) do not require `InStage` and are + callable outside stages. +- **R17** — Library helpers that perform side effects take `(using InStage)` and run + under the caller's stage rather than opening their own committing stages, so a + task still yields a single commit. A helper that itself *starts* stages instead + declares `using FlowControl` (R29), making that visible in its signature. +- **R29** — Starting a stage requires a `FlowControl` capability, where + `FlowControl <: FlowContext`: everything a `FlowContext` is (reads, `agent`, `emit`) + plus the authority to open a stage — but **thread-affine**, never handed to a fork. + `flow` provides it; `stage` requires it. At a direct `stage(...)` in a flow body it + resolves implicitly (zero ceremony); a stage-starting *helper* spells out + `using FlowControl`, so the fact is visible in its type. + +**Design.** + +```scala +trait FlowContext // thread-safe, shareable: reads + agent + emit +trait FlowControl extends FlowContext // + authority to start stages; thread-affine +opaque type InStage // in-stage mutation token, from `stage(...)` +``` + +Three capabilities, all constructible only inside `orca`: + +- **`FlowContext`** — the narrow, thread-safe context (tool reads, `agent`, + `userPrompt`, `emit`/`display`). Safe to share into parallel forks, so concurrent + reviewers each use it. +- **`FlowControl`** — a *subtype* of `FlowContext` adding the authority to start a + stage. Subtyping (not a derived given) is the point: a `FlowControl` satisfies any + `using FlowContext`, and the **downgrade is a one-way upcast** — concurrency + combinators run each fork with only the `FlowContext` (`val ctx: FlowContext = + control`), so `stage` (which needs `FlowControl`) cannot be called in a fork. + Pre-capture-checking this is convention (a fork could lexically capture an outer + `FlowControl`); capture checking later makes it a hard error (see below). +- **`InStage`** — the in-stage mutation token (R15), handed to a stage body by + `stage`, in the spirit of Ox's `using Ox`. + +Every side-effecting tool method gains a `(using InStage)` clause. The methods gated: + +- `GitTool`: `createBranch`, `checkout*`, `commit`, `push`, `addWorktree`, + `removeWorktree`, `ensureClean`. +- `FsTool`: `write`. +- `GitHubTool`: `createPr`, `updatePr`, `writeComment`, `upsertComment` *(new)*. +- Every `Agent` / `AgentCall` / `AutonomousTextCall` entry point (LLM calls are + side-effecting: cost and non-determinism). + +Side-effecting library helpers (`reviewAndFixLoop`, `fixLoop`, `lint`, +`summarisePr`, `Plan` generation, the reviewer fan-out) take `(using InStage)` and +run under the caller's stage, so the compiler enforces "no mutation outside a +stage" while a whole task still produces one commit. + +Both `InStage` and `FlowControl` are guard-rails today — a token can be captured and +used out of scope (an `InStage` smuggled past a stage; a `FlowControl` closed over in +a fork). The move to a real guarantee is additive: mark each a `caps.Capability` and +type the escaping positions (stage bodies, fork thunks) as pure, so escape becomes a +compile error; call sites don't change. (This is the §5 "leaky pre-capture-checking" +limitation and the §6 future-work item.) + +### 2.3 Stage results + +**Requirements.** +- **R9** — A stage's return value must be JSON-serialisable, witnessed by `JsonData` + (the existing tapir-`Schema` + jsoniter-codec bundle). Its codec must round-trip + losslessly (a resumed value equals the original). A return type without a + `JsonData` instance is a compile error. + +**Design.** + +Stage results reuse the existing `JsonData[A]` (tapir `Schema` + jsoniter codec) +rather than a dedicated typeclass — `stage[T: JsonData]`. Every type that already +travels through LLM calls (`Plan`/`PlanLike`, `Issue`, `PrHandle`, `IgnoredIssues`, +…) is therefore a valid stage result with no extra work, and `derives JsonData` on a +new case class is enough. The persistence path uses only the codec; the `Schema` +rides along unused — a fair price for not maintaining a second typeclass. + +The library adds `JsonData` givens for the handful of non-case-class results flows +return — primitives, `Unit`, `Option`, `List`, small tuples, and the opaque +`SessionId[B]` (`JsonData.derived` only covers `Mirror` types). A +return type with no `JsonData` instance — a live handle, a closure — fails to +compile, the intended boundary. Codecs must round-trip losslessly: a resumed run +reads the value back from JSON, so a lossy codec would diverge from a fresh run. A +human-readable summary for the log and `display` reuses the existing `Announce` +typeclass where one is in scope. + +### 2.4 Progress log, store, and recovery + +**Requirements.** +- **R18** — The progress log lives at a prompt-deterministic, branch-independent + path (`.orca/progress-<hash>.json`) so recovery can locate and read it before + checking out the feature branch. +- **R19** — The header records the starting branch, the feature-branch name, and the + prompt hash, and is committed as the branch's first commit. +- **R20** — Recovery reads the header from the working tree using a + snapshot-before-stash sequence (so an uncommitted/untracked log survives the + start-of-flow stash), then checks out the recorded branch and replays. +- **R21** — The branch-naming strategy and the progress store (its path and format, + via the `ProgressStore` below) are pluggable via flow configuration, with the + defaults below. (Commit messages plug in per stage — R13; the log format is JSON — + R28.) + +**Design.** + +```scala +case class ProgressHeader(startingBranch: String, branch: String, promptHash: String) +case class StageEntry(id: String, name: String, resultJson: String) +case class ProgressLog(header: ProgressHeader, entries: List[StageEntry]) + +trait ProgressStore: + def load(): Option[ProgressLog] + def writeHeader(header: ProgressHeader)(using InStage): Unit + def appendEntry(entry: StageEntry)(using InStage): Unit // upsert by id, last write wins + +object ProgressStore: + def default: ProgressStore // JSON at .orca/progress-<hash>.json (the seam for R21) +``` + +The log is heterogeneous — successive entries hold different result types — so a +`StageEntry` is type-erased at rest (`resultJson`); a typed `StageEntry[T]` can't +live in one `List`. Type-safety is recovered at the *decode* boundary: `stage[T]` +decodes with `JsonData[T].codec`, and a decode failure re-runs the stage (R11), so a +wrong-typed entry never reaches the call site. + +The default store serialises to `.orca/progress-<hash>.json`, where `hash` is the +first 12 hex chars of `SHA-256(userPrompt)` (the same scheme `Plan.defaultPath` uses +today). The path is independent of the branch, so recovery finds it without first +knowing which branch the work is on. `appendEntry` upserts by id: a fail-safe re-run +overwrites the stale entry rather than leaving two under one id. A wholly malformed +or truncated log (e.g. a crash mid-write) is treated as *no log* — the run starts +fresh — rather than crashing recovery. + +Recovery mirrors the proven `Plan.recover` sequence: snapshot the log file, +`git.ensureClean` (stash any pending edits — recoverable via `git stash pop`), +restore the log from the snapshot if the stash removed it, read the header, +**validate it as untrusted input (R32)**, `checkout` the recorded branch, and +replay. Because the branch name lives in the header, a non-deterministic branch name +is read back rather than recomputed. + +The runtime also cross-checks the header against git state (R30): if the log +surfaces on a branch it does *not* name (e.g. an in-progress branch was merged, +carrying the log along), it aborts with a clear message rather than resuming against +the wrong branch. + +### 2.5 Flow lifecycle & config + +**Requirements.** +- **R1** — Each `flow(...)` run is bound to exactly one feature branch and one + progress log. +- **R2** — The feature branch is created up-front, before the body runs, from the + current HEAD, via a pluggable `BranchNamingStrategy`. Built-ins cover the common + cases **safely**: **deterministic** ones derive a git-ref-safe name from structured + input (e.g. `issue(handle)` → `fix/issue-42`) or slug arbitrary text (`fromText`), + and a **prompt-shortening** one condenses a free-form prompt via the leading + model's cheap variant (`agent.cheap`, R31) then slugs it. **All** free text — author + titles and untrusted LLM output alike — routes through `slug`, which strips + leading/trailing hyphens, forces a leading alphanumeric, caps length, and falls + back to `flow-<shorthash>` on empty: so a branch name can never begin with `-` + (CLI-flag/argument injection into `git`/`gh`) nor be empty. A non-deterministic name is + computed once and recorded, never recomputed on resume; a deterministic name needs + no read-back. +- **R3** — The starting branch is recorded. On a **successful** exit HEAD stays on + the feature branch by default (so the author ends on the work — the common case + is a flow that opens no PR); a flow that opens a PR passes `returnToStartBranch = + true` to switch back to the starting branch afterward. (A throwaway branch is the + exception — see R5 — where HEAD always returns to the starting branch.) On + **failure** the flow stays on the feature branch, so a re-run resumes in place — + HEAD is already on the right branch and the committed log is in the working tree. +- **R4** — On flow start a dirty working tree is stashed, with a user-visible + warning, so the flow begins clean. +- **R5** — On **successful** exit the progress-log file is removed in a final + commit, and a feature branch left with no changes other than the progress log is + deleted (throwaway-branch cleanup). On a **failure** exit the feature branch and + its committed log are kept intact so the next run can resume; only the failed + stage's uncommitted partial edits are discarded (it re-runs on resume). +- **R6** — Push and PR creation are flow-controlled and usable at any point; the + runtime imposes no single terminal push. +- **R30** — On startup the runtime cross-checks the header's recorded branch against + the working state. If a progress log is found on a branch *other* than the one it + records — e.g. an in-progress branch was merged, carrying the log into another + branch — the run aborts with a clear message rather than resuming against the + wrong branch. +- **R31** — The leading **agent** (the coding harness — `Agent` is an agent, not + a model: each drives several models) is named by a required `agent` **selector** + argument to `flow[B](...)` — `FlowContext => Agent[B]`. `B` is the leading agent's + backend tag, **inferred** from the selector (`_.claude` ⇒ `ClaudeCode`) and never + written by callers; `flow(OrcaArgs(args), _.claude)` is unchanged. The only way to + name an agent is an accessor on the flow context (`_.claude`, `_.codex`, …), which + isn't in scope at the `flow(...)` argument position, so the selector defers + resolution until the context is built. Inside the body the resolved lead is reached + via the backend-agnostic top-level `agent` accessor (`def agent(using ctx: + FlowContext): Agent[ctx.LeadB]`) — an ordinary accessor over `FlowContext`, exactly + like `claude`/`git` — so the body reads the same regardless of backend; switch the + selector and the whole flow follows. The threading works because `B` is captured at + construction into a **type member** `FlowContext { type LeadB }` (a *member*, not a + parameter, so the whole `using FlowContext` surface stays unparametrised), making + `agent: Agent[ctx.LeadB]` concretely typed: a session from `agent.session` threads + into `agent.runSeeded` and the reviewers (R27). The runtime reads the same lead off + `ctx.agent` (erased to `Agent[?]`) for branch naming and session rehydration, and + the in-stage default commit message uses `ctx.agent.cheapOneShot`. `Agent` gains a + `cheap` method returning the backend's cheap variant (claude → haiku, gemini → + flash, codex → mini). There is no implicit default agent. (Boundary: the agnostic + `agent` covers the autonomous, tier-agnostic path; backend-specific tiers — + `claude.opus` — and interactive planning — `Plan.interactive`, which needs + `CanAskUser[B]`, defined only per concrete backend — use a concrete accessor.) +- **R32** — The progress header is **untrusted input** on load (the log is + human-visible and pushable — R26 — so it may be edited). Before any destructive + action the runtime validates it: `branch`/`startingBranch` must be safe refs + (`slug` rules), `promptHash` must equal the recomputed prompt hash, and it refuses + to `checkout`, `reset --hard`, or delete a protected branch (the default branch / + `main` / `master`) or any branch outside the orca naming scheme. + +**Design.** + +```scala +def flow[B <: BackendTag]( // B inferred from the selector, never written + args: OrcaArgs, + agent: FlowContext => Agent[B], // required leading-agent selector (R31) + // ^ resolved against the built context: `_.claude`, `_.codex`, … + // … existing tool overrides … + branchNaming: Option[BranchNamingStrategy] = None, + // ^ None ⇒ slug the prompt; or Some(BranchNamingStrategy.issue(handle)) for issue flows + returnToStartBranch: Boolean = false, // R3; false ⇒ stay on the feature branch + progressStore: Option[ProgressStore] = None // §2.4; pluggable path + format +)(body: FlowControl ?=> Unit): Unit // body is non-generic; `B` is pinned into FlowContext.LeadB +``` + +Per R31, `agent` is a selector resolved against the built `FlowContext` — the +only way to name an agent is an accessor on the context, which isn't in scope at the +`flow(...)` argument position, so resolution is deferred. The resolved lead is +reached in the body via the `agent` accessor (`Agent[ctx.LeadB]`); the runtime reads +the same lead off `ctx.agent` (erased) for branch naming, and the in-stage +default-commit-message path uses `ctx.agent.cheapOneShot`, overridable per stage via +`commitMessage` (§2.1). The lifecycle therefore builds the context (and +the progress store) **before** running branch setup, since branch naming needs the +resolved model. The body is `FlowControl ?=> Unit` +(R29): a direct `stage(...)` resolves its authority while forks see only +`FlowContext`. + +`ProgressStore` (§2.4) is the seam behind R21: the default writes JSON to +`.orca/progress-<hash>.json`; a custom store changes the path or format. + +`BranchNamingStrategy` ships the common cases plus a safe slugger, so a branch name +is always a valid git ref regardless of what the prompt or issue title contained: + +```scala +object BranchNamingStrategy: + /** git-ref-safe slug. Lower-cased; `[a-z0-9-]` only; runs collapsed; leading/ + * trailing `-` stripped so the ref never begins with `-` (CLI-flag/argument + * injection into `git`/`gh`); forced to start alphanumeric; capped to `maxLen`; + * empty input falls back to `flow-<shorthash>` so a ref is never empty. */ + def slug(text: String, maxLen: Int = 50): String + /** `<prefix>/issue-<number>` — number is an Int, prefix slugged; safe by construction. */ + def issue(handle: IssueHandle, prefix: String = "fix"): BranchNamingStrategy + /** Deterministic strategy from arbitrary text, routed through `slug`. */ + def fromText(text: => String): BranchNamingStrategy + /** Cheap-model shortening of a free-form prompt, then `slug`. */ + val shortenPrompt: BranchNamingStrategy +``` + +There is no `convention(rawString)` taking a pre-built name: that would invite +`s"fix/$title"`-style interpolation of unsanitised values. Structured inputs +(`issue`) and the slugger (`fromText`) cover the cases without that footgun. + +Setup (before the body): + +1. Record the starting branch (R3). +2. `ensureClean` — stash a dirty tree with a warning (R4). +3. Resolve the feature branch: if a header already exists (resume), take its branch + name; otherwise compute the name via `branchNaming` (R2) — the deterministic + strategies are pure, the prompt-shortening one calls `agent.cheap`. Branch naming is + a *setup step*, not a committing stage — it runs before the branch exists and its + result is captured in the header. +4. Fresh run: create + checkout the branch, then write and commit the header (R19). + Resume: checkout the existing branch — the header is already committed. + +Teardown on **success**: + +5. Remove the progress-log file in a final commit (R5). +6. If the branch now holds no changes vs. the starting branch (no code was + committed, only progress entries that step 5 just removed), delete it (R5). +7. Checkout the starting branch (R3); any stash from step 2 can be popped. + +Teardown on **failure**: + +8. Discard the failed stage's uncommitted partial edits with `git reset --hard` + (which restores the last committed log) — **not** `git clean -x .orca/`, or a log + force-added but not yet committed in the crash window would be wiped. They re-run + on resume. **Stay on the feature branch** (R3); do not return to the starting + branch — HEAD is then already where the next run resumes and the committed log is + in the working tree, so resume needs no branch lookup. The user switches back to + the starting branch themselves once they've dealt with the failure. + +Setup's own git and LLM operations (branch naming, header commit) run with +runtime-supplied `InStage` — the runtime is the privileged constructor of the +token, so it can mutate during setup before any user stage exists. Push and PR +creation remain ordinary stage actions (R6). The leading model (`agent`) and the +resolved branch are exposed through `FlowContext` accessors. The branch-naming +strategy and the progress store are overridable (R21). + +### 2.6 Sessions + +**Requirements.** +- **R22** — A stage result may carry an LLM `SessionId`, persisted in the log (with + the client→server map for server-id backends). On resume, whether the *live* + backend conversation can be continued is decided by a **non-destructive existence + probe** (`AgentBackend.sessionExists(id)`) rather than guessed: most backends expose + one (an on-disk session file, a list command, or a `GET`), pi does not. If the + probe is absent or says gone, resume falls back to re-seed (R23). +- **R23** — A session is obtained via a get-or-create (`agent.session`) whose id is + recorded in the log, so a retry reuses it rather than minting a second. A flow + attaches a `seed` — the essential context to rebuild the agent if its backend + conversation is lost (typically the plan brief, or the issue/plan when there is no + brief); the runtime additionally prepends a progress preamble derived from the log + (which stages have completed). **Re-seed is the reliable, uniform path**: on resume + orca re-mints and primes the session with preamble + seed unless a backend existence + probe (R22) confirms the recorded session is still live, in which case it continues + it. + +**Design.** + +A flow obtains a session via `agent.session(...)` — a *get-or-create* keyed by the +log, not a plain `new`. It is **pure**: it reserves a session id (a UUID) and records +the id + seed in the log; the backend conversation is created lazily on the first +gated `run`. So `agent.session(...)` is callable outside a stage while the actual LLM +effect stays inside one (R15). On resume it returns the recorded id. Naming it +`session` rather than `newSession` reflects the upsert: a retry does not create a +second session. + +```scala +val session = agent.session(seed = plan.brief) // author's essential context (a String) +``` + +`seed` is a string the author composes from whatever the agent needs to rebuild +itself — typically the **plan brief** (or the **issue / plan** when there's no +brief). It is applied on first use, priming the fresh session; if the session is +later lost on resume it is replayed into the re-minted one, with a **progress +preamble** the runtime prepends from the log (e.g. "Completed: planning, task 1, +task 2; resuming at task 3") — so the author needn't track progress in the seed. A +retry that finds the session still alive continues it directly and does not re-send +the seed. The seed is a single composed blob, not a transcript replay: cheap, +predictable, free of re-applying past effects. + +On resume orca decides continue-vs-re-seed by a **non-destructive existence probe** +per backend — `AgentBackend.sessionExists(id)` — not by attempting a resume and +catching an error (which some CLIs don't reliably raise). The recorded id (and, for +server-id backends, the client→server map) is persisted in the log and rehydrated, +then probed: + +| Backend | Probe | +|----------|-------------------------------------------------------------------------| +| claude | on-disk `~/.claude/projects/<cwd-slug>/<id>.jsonl` (cwd-scoped; 30-day prune) | +| codex | on-disk `find ~/.codex/sessions -name "rollout-*-<id>.jsonl"` (global) | +| gemini | `gemini --list-sessions` (project-scoped) | +| opencode | `GET /session/<id>` → 200 vs 404 (durable across server restarts) | +| pi | none — sessions live in a `deleteOnExit` temp dir, gone across runs | + +If the probe confirms the session, orca continues it; if it returns absent (or the +backend has no probe — pi), it re-mints and re-seeds. **Re-seed stays the guaranteed +fallback**; the probe only makes "continue the live session" a deterministic decision +rather than a hope. *Implementation notes:* this needs (a) the persisted id/map — +today's `SessionRegistry` is in-memory and not rehydrated from the log; and (b) a +`sessionExists` method on the backend SPI. The probe mechanisms are verified against +current CLIs (claude/codex on-machine; the rest from docs/source); gemini's +list output and opencode's directory-scoping should be pinned when the probes land. + +### 2.7 External-effect idempotency + +**Requirements.** +- **R24** — Externally-observable GitHub effects are idempotent by default, to + survive the non-atomic window between an effect and its progress commit: + `createPr` reuses an existing open PR matched on the **head branch** (orca creates + exactly one PR per feature branch, so head alone is unambiguous; a head-*and*-base + match would only matter for stacked-PR setups, which orca never produces); + `upsertComment(marker, + …)` accompanies append-only `writeComment`, where the **marker embeds the prompt + hash** so it's unique per flow and can't collide with another run's (or a + third-party) comment. The progress entry is committed immediately after the effect. +- **R26** — A *merged* PR must not contain the progress log: the final cleanup + commit (R5) removes it, so a clean branch merges clean. While a PR is open + mid-flow the file is present in the pushed history; this is accepted as a known + limitation (§5) rather than rewriting history on every push. + +**Design.** + +`gh.createPr`, on hitting GitHub's "a PR for this branch already exists", looks up the +open PR for this head branch and returns it instead of failing — so a resume after "PR +created but progress commit lost" reuses rather than duplicates. (Matching on the head +branch is sufficient because orca owns one feature branch per flow and opens a single +PR from it; head+base matching would only be needed to disambiguate stacked PRs, which +this runtime never creates.) `gh.upsertComment(marker, body)` finds a prior comment carrying `marker` and +edits it in place; the marker embeds the prompt hash so it's unique to this flow +(not forgeable into another run's comment). Plain `writeComment` stays append-only +for genuinely additive comments. The stage commits the progress entry immediately +after the effect, narrowing the non-atomic window to the smallest possible. + +### 2.8 Replacing `Plan` persistence + +**Requirements.** +- **R25** — The stage log is the sole resume mechanism. `Plan.recover`, + `.orca/plan-<hash>.md`, and checkbox-ticking *as resume state* are retired; + `Plan`'s generation API is kept and its task loop collapses to per-task stages. A + human-readable progress checklist may still be rendered as a convenience, but it + is cosmetic — never read back for resume. + +**Design.** + +`Plan.recover`, `recoverOrCreate`, the `.orca/plan-<hash>.md` file, and +checkbox-ticking are removed. The plan becomes the result of a `stage("Plan")`, and +completion is stage completion in the log. `implementTaskLoop` collapses to: + +```scala +for task <- plan.tasks do + stage(s"task: ${task.title}")(/* implement + review, under this stage */) +``` + +`Plan`'s generation grid (`autonomous` / `interactive` × `from` / +`assessThenPlan` / `triage`) is unchanged except that **every generated plan now +carries its brief**: the `Plan` / `PlanWithBrief` split and the explicit `.briefed` +step are removed, so `plan.brief` is always available (it feeds the session seed, +§2.6). This is a small `Plan`-API simplification adjacent to this work. + +A flow may still render a human-readable checklist for users — a ticked task list in +the PR body, or a committed `PLAN.md` updated as part of each task stage's commit. +This is purely cosmetic: resume reads the stage log, never the checklist, so the two +cannot desync into a wrong resume. + +--- + +## 3. Example flows (after the refactor) + +Illustrative flow scripts only. Reads run outside stages; side-effecting work is +staged and resumable; the branch is auto-created from the prompt and auto-deleted +if nothing of substance happens; pushes are mid-flow. + +### 3.1 `implement.sc` + +```scala +//> using dep "org.virtuslab::orca:0.2" +//> using jvm 21 +import orca.{*, given} + +flow(OrcaArgs(args), _.claude): // required agent selector + val plan = stage("Plan"): + Plan.autonomous.from(userPrompt, agent).value // Plan (always briefed; has JsonData) + + // Get-or-create the implementer session (pure: id reserved, backend created on + // first use). The seed (plan brief) primes it on first use, and is replayed if the + // backend session is lost on resume. + val session = agent.session(seed = plan.brief) + + for task <- plan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + agent.runSeeded(task.description, session) + reviewAndFixLoop( // runs under this stage (using InStage) + coder = agent, sessionId = session, + reviewers = allReviewers(agent), + reviewerSelection = ReviewerSelector.agentDriven(agent.cheap), + task = task.title.value, + formatCommand = Some("cargo fmt") + ) + // one commit per task: code + progress entry +``` + +`git.commit`, `agent.runSeeded`, and `reviewAndFixLoop` require `InStage`, +supplied by the enclosing `stage`. There is no plan markdown file — the progress +log subsumes it, which also removes any checkbox state to keep in sync (a +human-readable checklist, if wanted, is cosmetic — §2.8). + +`issue-pr.sc` adds only two patterns over 3.1: it picks the deterministic +`BranchNamingStrategy.issue(issueHandle)` so the branch is named from the issue +up-front, and a read-only assess stage that may reject (post a comment, return +`None`, and let the throwaway branch be deleted on exit) before a trailing +`stage("Open PR")` that pushes then `gh.createPr` (idempotent by branch). It is +otherwise 3.1's task loop. + +### 3.2 `issue-pr-bugfix.sc` + +The hard case: it may early-exit (post a comment, no code), and it pushes +mid-flow — first the failing test, then the fix. + +```scala +// Parse the handle up-front so it can seed the deterministic branch naming. +val orcaArgs = OrcaArgs(args) +val issueHandle = IssueHandle.parseOrThrow(orcaArgs.userPrompt) + +flow(orcaArgs, branchNaming = Some(BranchNamingStrategy.issue(issueHandle))): // claude default + val issue = gh.readIssue(issueHandle) // read + val session = claude.session(seed = issue.body) // get-or-create + val triage = stage("Triage"): // LLM → staged + Plan.autonomous.triage(report(issue), claude).value + + triage match + case Triage.NotABug(why) => + stage("Comment: not a bug")(gh.writeComment(issueHandle, why)) // throwaway branch, deleted on exit + + case Triage.Untestable(_, steps) => + stage("Comment: repro steps")(gh.writeComment(issueHandle, steps)) + + case Triage.Testable(summary, _, failingTestPath) => + stage("Write failing test"): // commits the test + claude.runSeeded(s"Write the failing test at $failingTestPath …", session) + + val pr = stage("Push + open tentative PR"): // later stage: the test is committed + git.push().orThrow // push #1 + gh.createPr(title = summary, body = "Failing test only.").orThrow // idempotent by branch + + if gh.waitForBuild(pr, 30.minutes).orThrow.outcome == BuildOutcome.Success then // read + fail("CI passed on the failing-test commit — the reproduction is wrong.") + display(s"CI red on ${pr.shortRef} — reproduction confirmed") // progress-only + + stage("Confirm repro")(/* sonnet inspects the run via gh, posts a comment */) + + val fixPlan = stage("Plan the fix"): + Plan.autonomous.from(s"Fix ${issueHandle.shortRef}; a failing test exists.", claude).value + for task <- fixPlan.tasks do + stage(s"task: ${task.title}")(/* implement + reviewAndFixLoop, under this stage */) + + stage("Push fix + finalise PR"): + git.push().orThrow // push #2 + gh.updatePr(pr, title = summary, body = "Failing test + fix.") +``` + +> **Authoring rule (R8).** A stage commits only on completion, so any operation +> needing an existing commit — `push` above all — must live in a *later* stage than +> the code that produced it. A push in the same stage as the edit would push +> nothing. (This and the other authoring rules go in the README — Epic G.) + +--- + +## 4. Implementation + +Sequenced as Epics A–G in the task-level plan `plan-stage-runtime-impl.md` (file map, +per-task TDD steps, dependency order). Summary: + +- **A — `JsonData` for stage results.** Givens for primitives / `Unit` / `Option` / + `List` / small tuples / `SessionId[B]`; a sum codec for the sealed `PlanLike`. No + new typeclass. Self-contained; lands first. +- **B — Capabilities + tool gating.** `FlowControl`, `InStage`; `(using InStage)` on + the §2.2 methods (plus `gh.upsertComment` and `createPr` reuse); thread `InStage` + through side-effecting helpers, `FlowControl` through stage-starting ones. The + widest, compiler-guided change. Negative compile tests: `git.commit` outside a + stage, and `stage` inside a fork, must fail to compile. +- **C — Progress log model + store.** `ProgressHeader`/`StageEntry`/`ProgressLog`; + `ProgressStore` default at `.orca/progress-<hash>.json`; recovery + untrusted-header + validation (R32). +- **D — Stage runtime + resumption.** `stage`/`display`/`fail`; decode-or-rerun; + per-stage commit; the `sessionExists` probe on the backend SPI and a persistable + `SessionRegistry`; `agent.session` / `agent.cheap`; re-seed on resume. +- **E — Flow lifecycle + config.** `flow(...)` setup/teardown, `FlowControl` + provision, `BranchNamingStrategy`, mandatory leading `agent`. +- **F — Replace `Plan` persistence; migrate examples.** Retire `Plan.recover` etc.; + always-briefed plans; convert the example flows. +- **G — User documentation.** README coverage of the authoring rules, the capability + model, resume semantics, and the accepted limitations. + +Dependency order: A, B, C independent; D needs A+B+C; E needs B+C+D; F needs D+E; G +alongside. + +--- + +## 5. Consequences — risks & accepted limitations + +- **Resumability becomes uniform** across every side-effecting step, not just tasks — + the bugfix flow's triage/CI phases now resume. +- **Progress log in the open PR (R26) — unsolved.** Committed-on-branch gives + atomic code+progress, but every mid-flow push carries `.orca/progress-<hash>.json` + into the PR until the final cleanup commit removes it. Stripping it per-push would + rewrite history and break resume; this is accepted as a known wart of the "keep it + committed" choice. Merged PRs are clean (R26); open ones are not. Beyond tidiness + it is a **confidentiality** surface: stage results (LLM outputs, issue bodies) and + commit messages are serialised verbatim into the log and pushed history (and + survive in reflog/forks after the cleanup commit), so a flow must not return + secrets as stage results. A redaction hook is possible but out of scope for v1 + (§6). +- **Capabilities are leaky pre-capture-checking.** Both `InStage` (no mutation + outside a stage) and `FlowControl` (no stage inside a fork) prove *lexical* + enclosure, not the real invariant: a token can be captured and used where it + shouldn't (an `InStage` smuggled out of a stage; a `FlowControl` closed over inside + a `parallel` fork). They are kept for the compile-time signal — guard-rails, not + yet guarantees — and both are future-proofable via Scala 3 capture checking (§6). + Gating is also the widest change in the project (four tool traits, ~8 helpers, + every flow script); acceptable here because breaking changes are fine at this stage. +- **Resume is decode-safe, not meaning-safe.** A stored result that no longer decodes + to the call-site type re-runs the stage; but a stage whose *meaning* changes under + a stable name and a still-decodable type silently reuses the stale value — the more + likely edit between a crash and a resume. Relatedly, inserting a *same-named* stage + before existing ones shifts later occurrence-indices (R10), so their stored entries + no longer match and that work re-runs — harmless but worth knowing. +- **Resume preserves files, not agent context.** File state is committed and + restored; the LLM conversation is not. After a crash the re-seed blob restores + only the seed (e.g. the plan), so a long implementer session resumes materially + "dumber" — the seed must carry enough for the remaining tasks to stand alone. +- **Cross-backend live-session resume is non-uniform.** A non-destructive existence + probe exists for claude/codex/gemini/opencode (on-disk file, `--list-sessions`, + `GET /session/<id>`) but **not pi** (`deleteOnExit` temp dir), so pi always + re-seeds. Even where a probe exists it needs the persisted id/map (today's + `SessionRegistry` is in-memory) and carries caveats — claude is cwd-scoped + 30-day + pruned, gemini's id↔file is fuzzy (use the list, not the file), opencode is + project-scoped. So live continuation is a per-backend optimization gated on the + probe, and **re-seed remains the path that holds on every backend** (R22/R23/§2.6). +- **Branch naming: only the prompt-shortening strategy is non-deterministic.** The + deterministic strategies (`issue(handle)` → `fix/issue-42`, `fromText`) are + recoverable by recomputation; the prompt-shortening one (for free-form `implement` + prompts) is why the header persists the once-computed name for read-back on resume. +- **`JsonData` coverage.** Sealed `PlanLike` needs jsoniter sum-type config; + primitives/tuples/`SessionId` need hand-written `JsonData` givens; codecs must be + lossless (R9). +- **Custom external effects.** Built-in idempotency covers `createPr` / + `upsertComment` (R24); a flow's own irreversible side effect must be made + idempotent by its author or it repeats on resume. +- **Stage ordering discipline.** Push-after-commit (§3.2) and no-concurrent-stages + (R12) are authoring rules the compiler cannot enforce. + +--- + +## 6. Out of scope & future work + +Deliberately deferred; recorded here so nothing is lost: + +- **Capture-checking migration.** Turn the convention guard-rails into compile-time + guarantees: mark `InStage` and `FlowControl` as `caps.Capability` and type the + escaping positions (stage bodies, fork thunks) as pure, so a smuggled `InStage` or + a `FlowControl` captured into a fork is a compile error. Additive — no call-site + changes (§2.2, §5). +- **Progress-log redaction hook.** A seam to redact/secret-scrub stage results before + they are committed, addressing the open-PR confidentiality surface (§5, R26). +- **Nested, repeated, or concurrent `flow(...)` calls.** A flow binds one branch and + one log; a flow nested inside another, or two runs sharing one working tree / git + index / `.orca/` path (e.g. the same prompt re-run while the first is still alive), + is undefined here. The lifecycle must at minimum not corrupt an outer/other flow's + state, but multi-flow semantics (and any locking) are deferred. +- **Cross-machine resume.** Backend LLM sessions are local; resuming on a different + machine always falls back to the re-seed path (R23). Persisting/transferring backend + sessions across machines is out of scope. diff --git a/build.sbt b/build.sbt index 214357e2..948b0a12 100644 --- a/build.sbt +++ b/build.sbt @@ -113,7 +113,7 @@ lazy val gemini = (project in file("gemini")) ) lazy val flow = (project in file("flow")) - .dependsOn(tools) + .dependsOn(tools, tools % "test->test") .settings(commonSettings) .settings( name := "orca-flow", @@ -121,7 +121,7 @@ lazy val flow = (project in file("flow")) ) lazy val runner = (project in file("runner")) - .dependsOn(tools, flow, claude, codex, opencode, pi, gemini) + .dependsOn(tools, tools % "test->test", flow, claude, codex, opencode, pi, gemini) .settings(commonSettings) .settings( // Published as just "orca" so flow-script coordinates stay short. diff --git a/claude/src/main/scala/orca/tools/claude/ClaudeArgs.scala b/claude/src/main/scala/orca/tools/claude/ClaudeArgs.scala index cc3b6c1d..3875b5e2 100644 --- a/claude/src/main/scala/orca/tools/claude/ClaudeArgs.scala +++ b/claude/src/main/scala/orca/tools/claude/ClaudeArgs.scala @@ -1,9 +1,9 @@ package orca.tools.claude import orca.backend.{CliArgs, Dispatch} -import orca.llm.{AutoApprove, BackendTag, LlmConfig, SessionId, ToolSet} +import orca.agents.{AutoApprove, BackendTag, AgentConfig, SessionId, ToolSet} -/** Maps LlmConfig fields to Claude Code CLI flags. `systemPrompt` is consumed +/** Maps AgentConfig fields to Claude Code CLI flags. `systemPrompt` is consumed * by the backend (written to a file whose path is passed in via * `systemPromptFile`); `onUnapproved` and `retrySchedule` have no CLI * equivalent and are handled by the orchestrator at runtime. @@ -23,7 +23,7 @@ private[claude] object ClaudeArgs: * stay open. */ def streamJson( - config: LlmConfig, + config: AgentConfig, systemPromptFile: Option[os.Path], dispatch: Dispatch[BackendTag.ClaudeCode.type], jsonSchema: Option[String] = None, @@ -72,11 +72,11 @@ private[claude] object ClaudeArgs: private def mcpConfigArgs(file: Option[os.Path]): Seq[String] = file.toSeq.flatMap(f => Seq("--mcp-config", f.toString)) - /** Maps [[LlmConfig.tools]] to claude's permission flags. Both read-only + /** Maps [[AgentConfig.tools]] to claude's permission flags. Both read-only * tiers use `--permission-mode plan`, which makes Edit/Write/Bash * unavailable (not just non-auto-approved) — turning the planner's advisory * "don't edit" prompt into a hard guarantee. `Full` follows - * [[LlmConfig.autoApprove]]. + * [[AgentConfig.autoApprove]]. * * `NetworkOnly` additionally pre-approves `networkTools` via * `--allowedTools`, layering read-only network access (web + scoped `gh`) @@ -86,7 +86,7 @@ private[claude] object ClaudeArgs: * plain plan mode. */ private def autoApproveArgs( - config: LlmConfig, + config: AgentConfig, networkTools: Seq[String] ): Seq[String] = config.tools match diff --git a/claude/src/main/scala/orca/tools/claude/ClaudeBackend.scala b/claude/src/main/scala/orca/tools/claude/ClaudeBackend.scala index c952340d..66972a18 100644 --- a/claude/src/main/scala/orca/tools/claude/ClaudeBackend.scala +++ b/claude/src/main/scala/orca/tools/claude/ClaudeBackend.scala @@ -1,21 +1,21 @@ package orca.tools.claude import orca.events.OrcaListener -import orca.llm.{BackendTag, LlmConfig, SessionId} -import orca.{AgentTurnFailed, OrcaFlowException} +import orca.agents.{BackendTag, AgentConfig, SessionId} import orca.backend.{ Conversation, Conversations, - LlmBackend, - LlmResult, + AgentBackend, + AgentResult, SessionMode, SessionRegistry, + SubprocessSpawn, SystemPromptComposer } import orca.subprocess.CliRunner import orca.backend.mcp.{AskUserMcpServer, AskUserSession} import orca.tools.claude.streamjson.OutboundMessage -import ox.Ox +import ox.{Ox, supervised} import ox.channels.BufferCapacity /** Claude Code backend. All calls — autonomous and interactive — drive a @@ -27,7 +27,7 @@ import ox.channels.BufferCapacity * * The autonomous path drains those events via * [[orca.backend.Conversations.drainAutonomous]] and returns the awaited - * `LlmResult`. The interactive path hands the `Conversation` back to the + * `AgentResult`. The interactive path hands the `Conversation` back to the * caller who runs `Interaction.drive`. * * Interactive calls also stand up an MCP host bridge: a tiny HTTP server (via @@ -39,17 +39,34 @@ import ox.channels.BufferCapacity */ private[orca] class ClaudeBackend( cli: CliRunner, - networkTools: Seq[String] = ClaudeBackend.DefaultNetworkTools -)(using Ox, BufferCapacity) - extends LlmBackend[BackendTag.ClaudeCode.type]: + networkTools: Seq[String] = ClaudeBackend.DefaultNetworkTools, + private[claude] val projectsDir: os.Path = os.home / ".claude" / "projects", + private[claude] val cwdForProbe: os.Path = os.pwd +)(using BufferCapacity) + extends AgentBackend[BackendTag.ClaudeCode.type]: /** Return a sibling backend that, on [[ToolSet.NetworkOnly]] turns, * pre-approves `tools` (claude `--allowedTools` syntax). The configuration - * seam behind `ClaudeTool.withNetworkTools`; lives on the backend, not - * `LlmConfig`, since the strings are claude-specific. + * seam behind `ClaudeAgent.withNetworkTools`; lives on the backend, not + * `AgentConfig`, since the strings are claude-specific. */ def withNetworkTools(tools: Seq[String]): ClaudeBackend = - new ClaudeBackend(cli, tools) + new ClaudeBackend(cli, tools, projectsDir, cwdForProbe) + + /** Best-effort probe: checks for the on-disk transcript file at + * `<projectsDir>/<cwdSlug>/<id>.jsonl`. The project-dir slug is derived from + * the working directory by replacing every `/` with `-` (e.g. + * `/home/foo/orca` → `-home-foo-orca`). Returns `false` — safe re-seed — + * when the file is absent, the projects dir doesn't exist yet, or the id + * fails the [[orca.agents.isSafeSessionId]] guard (blocks path traversal + * such as `../../etc/passwd`). + */ + override def sessionExists( + session: SessionId[BackendTag.ClaudeCode.type] + ): Boolean = + probeGuarded(SessionId.value(session)): id => + val slug = ClaudeBackend.cwdSlug(cwdForProbe) + os.exists(projectsDir / slug / s"$id.jsonl") /** Tracks which session ids we've already claimed via `--session-id` so * subsequent calls use `--resume` (the CLI refuses to reuse `--session-id` @@ -58,45 +75,63 @@ private[orca] class ClaudeBackend( private val sessions = new SessionRegistry.ClaimedOnce[BackendTag.ClaudeCode.type] + // Claude's sessions live on disk (`~/.claude/projects/.../<id>.jsonl`) and + // outlive the process, so the `--session-id` (claim) → `--resume` (continue) + // decision must survive a resume. Wire the claim into the persist/rehydrate + // hooks: `resumeWireId` lets the runtime record the claimed id (claude's + // client id IS the wire id) into the progress log, and `registerSession` + // re-claims it on rehydrate so a resumed task uses `--resume` rather than + // re-creating an already-existing session id. Unlike pi, whose sessions live + // in a deleteOnExit temp dir and are gone across runs, claude's are durable, + // so it must participate in persist/rehydrate rather than always re-seeding. + override def resumeWireId( + client: SessionId[BackendTag.ClaudeCode.type] + ): Option[SessionId[BackendTag.ClaudeCode.type]] = + sessions.resumeWireId(client) + + override def registerSession( + client: SessionId[BackendTag.ClaudeCode.type], + server: SessionId[BackendTag.ClaudeCode.type] + ): Unit = + sessions.commitSuccess(client, server) + def runAutonomous( prompt: String, session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener = OrcaListener.noop, outputSchema: Option[String] = None - ): LlmResult[BackendTag.ClaudeCode.type] = - val conv = openConversation( - prompt = prompt, - mode = SessionMode.Autonomous, - session = session, - config = config, - workDir = workDir, - outputSchema = outputSchema - ) - val result = - try Conversations.drainAutonomous(conv, events) - catch - // Preserve the non-retryable type: a turn that ran and failed must not - // be retried (it would reopen the now-registered session id). - case e: AgentTurnFailed => throw e - case e: OrcaFlowException => - throw OrcaFlowException(s"claude CLI failed: ${e.getMessage}") - // Commit only after a successful drain: a subprocess that crashed before - // claude could register the session id (e.g. exit before `system.init`) - // would otherwise leave the registry wedged, forcing a retry to - // `--resume` a session claude never created. - sessions.commitSuccess(session, session) - result + ): AgentResult[BackendTag.ClaudeCode.type] = + // Self-scoped: the conversation forks its workers into this per-call Ox, the + // drain consumes them, and `cancel` (the `finally`) tears the subprocess + + // forks down before the scope joins. `drainAndCommit` doesn't tear down, so + // the `finally` is load-bearing (and a no-op on the happy path). + supervised: + val conv = openConversation( + prompt = prompt, + mode = SessionMode.Autonomous, + session = session, + config = config, + workDir = workDir, + outputSchema = outputSchema + ) + // drainAndCommit commits only after a successful drain: a subprocess that + // crashed before claude could register the session id (e.g. exit before + // `system.init`) would otherwise leave the registry wedged, forcing a + // retry to `--resume` a session claude never created. + try + Conversations.drainAndCommit("claude", conv, session, sessions, events) + finally conv.cancel() def runInteractive( prompt: String, session: SessionId[BackendTag.ClaudeCode.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.ClaudeCode.type] = + )(using Ox): Conversation[BackendTag.ClaudeCode.type] = val conv = openConversation( prompt = prompt, mode = SessionMode.Interactive(displayPrompt), @@ -147,10 +182,10 @@ private[orca] class ClaudeBackend( prompt: String, mode: SessionMode, session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.ClaudeCode.type] = + )(using Ox): Conversation[BackendTag.ClaudeCode.type] = // Allocate ask_user resources up front so we can close them // deterministically on a downstream failure. `None` for autonomous — // those calls don't expose the tool. Claude's `extras` deletes the @@ -165,7 +200,7 @@ private[orca] class ClaudeBackend( ) (Some(resources), p) case SessionMode.Autonomous => (None, "") - try + SubprocessSpawn.open("claude stream-json", askUser.toList) { val systemPromptFile = writeSystemPromptIfPresent( config, @@ -181,7 +216,7 @@ private[orca] class ClaudeBackend( // leave the registry wedged — a retry will still try `--session-id`. // Callers must not share a session id across concurrent calls; // `reviewAndFixLoop`'s parallel reviewer fan-out is safe because each - // reviewer mints its own distinct id via `LlmTool.newSession`. + // reviewer mints its own distinct id via `Agent.newSession`. val args = ClaudeArgs.streamJson( effectiveConfig, systemPromptFile, @@ -190,35 +225,20 @@ private[orca] class ClaudeBackend( mcpConfig = askUser.map(r => mcpConfigPath(r.server, workDir)), networkTools = networkTools ) - val process = cli.spawnPiped(args, cwd = workDir) - try - process.writeLine( - OutboundMessage.toJson(OutboundMessage.UserText(prompt)) - ) - process.closeStdin() - new ClaudeConversation( - process, - config, - initialPrompt = displayPrompt, - outputSchema = outputSchema, - askUser = askUser - ) - catch - case e: Exception => - // SIGINT the process; the outer catch closes askUser. - process.sendSigInt() - throw OrcaFlowException( - s"Failed to open claude stream-json session: ${e.getMessage}" - ) - catch - case e: Throwable => - // Any failure between resource allocation and a fully-constructed - // ClaudeConversation: tear down the MCP server (and delete the - // config file) so we don't leak a Netty binding or workDir - // artefact. Once the conversation owns the resources they ride - // through `onFinalize`. - askUser.foreach(_.close()) - throw e + cli.spawnPiped(args, cwd = workDir) + } { process => + process.writeLine( + OutboundMessage.toJson(OutboundMessage.UserText(prompt)) + ) + process.closeStdin() + new ClaudeConversation( + process, + config, + initialPrompt = displayPrompt, + outputSchema = outputSchema, + askUser = askUser + ) + } /** Path of the workDir-local MCP config file advertising the host's MCP * server. Named with the bound port so two interactive conversations sharing @@ -255,7 +275,7 @@ private[orca] class ClaudeBackend( * once on startup via `--append-system-prompt-file`). */ private def writeSystemPromptIfPresent( - config: LlmConfig, + config: AgentConfig, includeAskUserHint: Boolean = false ): Option[os.Path] = val hint = Option.when(includeAskUserHint)(AskUserMcpServer.Hint) @@ -266,6 +286,13 @@ private[orca] class ClaudeBackend( object ClaudeBackend: + /** Derives the project-directory slug that claude uses under + * `~/.claude/projects/`: replaces every `/` in the absolute path with `-`. + * E.g. `/home/foo/bar` → `-home-foo-bar`. + */ + private[claude] def cwdSlug(cwd: os.Path): String = + cwd.toString.replace('/', '-') + /** Read-only network tools pre-approved on [[ToolSet.NetworkOnly]] turns, so * an autonomous planner can read issues/PRs/web without a permission prompt * it can't answer. Command-scoped, so plan mode still blocks general bash diff --git a/claude/src/main/scala/orca/tools/claude/ClaudeConversation.scala b/claude/src/main/scala/orca/tools/claude/ClaudeConversation.scala index 57753a02..7dfe4bf3 100644 --- a/claude/src/main/scala/orca/tools/claude/ClaudeConversation.scala +++ b/claude/src/main/scala/orca/tools/claude/ClaudeConversation.scala @@ -1,11 +1,12 @@ package orca.tools.claude -import orca.llm.{AutoApprove, BackendTag, LlmConfig, Model, SessionId} +import orca.agents.{AutoApprove, BackendTag, AgentConfig, Model, SessionId} import orca.events.{Usage} import orca.{OrcaFlowException} -import orca.backend.{ApprovalDecision, ConversationEvent, LlmResult} -import orca.backend.{StreamConversation, StreamSource} +import orca.backend.{ApprovalDecision, ConversationEvent, AgentResult} +import orca.backend.{ForkedConversation, StreamSource} import orca.subprocess.PipedCliProcess +import ox.Ox import orca.tools.claude.streamjson.{ ContentBlock, ControlDecision, @@ -17,8 +18,8 @@ import orca.tools.claude.streamjson.{ /** Drives a stream-json conversation with claude to completion. * - * Boilerplate (reader thread, event queue, outcome lifecycle, stderr drain) - * lives in [[StreamConversation]]; this class supplies the claude-specific + * Boilerplate (reader fork, event queue, outcome lifecycle, stderr drain) + * lives in [[ForkedConversation]]; this class supplies the claude-specific * protocol translation: NDJSON → [[InboundMessage]] → `ConversationEvent`s, * plus auto-approve policy for tools listed in `config.autoApprove`. Outbound * writes (user turns, tool-approval responses) happen on the channel's thread @@ -26,18 +27,17 @@ import orca.tools.claude.streamjson.{ */ private[claude] class ClaudeConversation( process: PipedCliProcess, - config: LlmConfig, + config: AgentConfig, initialPrompt: String = "", val outputSchema: Option[String] = None, override val askUser: Option[orca.backend.mcp.AskUserSession] = None -) extends StreamConversation[BackendTag.ClaudeCode.type]( +)(using Ox) + extends ForkedConversation[BackendTag.ClaudeCode.type]( source = StreamSource.fromProcess(process), backendName = "claude", initialPrompt = initialPrompt ): - import StreamConversation.Outcome - // The reader thread is the sole writer for all three fields below. // No cross-thread visibility concerns: reads happen on the same thread // immediately after writes, within `handle(...)` dispatch. Plain `var`s @@ -59,24 +59,15 @@ private[claude] class ClaudeConversation( /** Tool-use ids of `ask_user` calls suppressed in `handleAssistantTurn`. * `handleUserTurn` drops the matching `tool_result` so the user's typed * answer doesn't re-render as `⎿ <answer>` right after the UserQuestion - * prompt already surfaced it. + * prompt already surfaced it. See [[orca.backend.AskUserEchoes]]. */ - private var suppressedToolUseIds: Set[String] = Set.empty - - // --- Conversation surface (only the bit not covered by the base) --- - - def sendUserMessage(text: String): Unit = - writeOutbound(OutboundMessage.UserText(text)) + private val askUserEchoes = new orca.backend.AskUserEchoes // `canAskUser` + ask_user bridge drainer + onFinalize close are owned by // the base; this subclass just declares `askUser` on the ctor param // above. Stdin-as-user-channel is closed right after the initial prompt, // so mid-session input flows through the MCP tool result either way. - // Subclass fields above are assigned now; safe to spin up the reader + - // stderr workers. See [[StreamConversation.start]]. - start() - // --- Reader hook --- override protected def handleLine(line: String): Unit = @@ -142,7 +133,7 @@ private[claude] class ClaudeConversation( // `⎿ <answer>` after the prompt already surfaced it). case ContentBlock.ToolUse(id, name, _) if name == ClaudeBackend.AskUserToolName => - suppressedToolUseIds = suppressedToolUseIds + id + askUserEchoes.suppress(id) case ContentBlock.ToolUse(_, name, rawInput) => eventQueue.enqueue(ConversationEvent.AssistantToolCall(name, rawInput)) case ContentBlock.Text(text) if !sawDeltasThisTurn => @@ -159,10 +150,11 @@ private[claude] class ClaudeConversation( private def handleUserTurn(content: List[ContentBlock]): Unit = content.foreach: case ContentBlock.ToolResult(toolUseId, _, _) - if suppressedToolUseIds.contains(toolUseId) => + if askUserEchoes.consume(toolUseId) => // Paired with a suppressed `ask_user` ToolUse; the user has already // seen their own typed answer at the prompt, so don't echo it back. - suppressedToolUseIds = suppressedToolUseIds - toolUseId + // `consume` removes the id as it matches. + () case ContentBlock.ToolResult(_, body, isError) => eventQueue.enqueue( ConversationEvent.ToolResult( @@ -180,7 +172,7 @@ private[claude] class ClaudeConversation( usage: Usage, model: Option[String] ): Unit = - val result = LlmResult( + val result = AgentResult( sessionId = SessionId[BackendTag.ClaudeCode.type](sid), output = structured.orElse(output).getOrElse(""), usage = usage, @@ -188,16 +180,16 @@ private[claude] class ClaudeConversation( // result message omits it. model = model.orElse(initModel).map(Model.apply) ) - val _ = outcomeRef.compareAndSet(None, Some(Outcome.Success(result))) + succeedWith(result) /** Claude sets `is_error: true` for out-of-band failures (API errors, rate * limits, auth problems) that happen at the CLI boundary rather than inside * a turn. Treat these as session-ending failures rather than feeding the * error body into the downstream response parser, which might otherwise * accept a `{"type":"error",...}` payload as a structurally valid agent - * output. `Outcome.Failed` carries the full message for `awaitResult` to - * surface; the in-stream `Error` event is short if the user already saw the - * body as part of a streamed turn. + * output. `failWith` carries the full message for `awaitResult` to surface; + * the in-stream `Error` event is short if the user already saw the body as + * part of a streamed turn. */ private def handleResultError(output: Option[String]): Unit = val message = @@ -206,14 +198,7 @@ private[claude] class ClaudeConversation( if deltasSinceTurnBoundary then "session failed (see message above)" else message eventQueue.enqueue(ConversationEvent.Error(displayed)) - val _ = outcomeRef.compareAndSet( - None, - Some( - Outcome.Failed( - new OrcaFlowException(s"claude session failed: $message") - ) - ) - ) + failWith(new OrcaFlowException(s"claude session failed: $message")) private def handleControlRequest( requestId: String, diff --git a/claude/src/main/scala/orca/tools/claude/DefaultClaudeTool.scala b/claude/src/main/scala/orca/tools/claude/DefaultClaudeAgent.scala similarity index 67% rename from claude/src/main/scala/orca/tools/claude/DefaultClaudeTool.scala rename to claude/src/main/scala/orca/tools/claude/DefaultClaudeAgent.scala index 9c136165..f73fa828 100644 --- a/claude/src/main/scala/orca/tools/claude/DefaultClaudeTool.scala +++ b/claude/src/main/scala/orca/tools/claude/DefaultClaudeAgent.scala @@ -1,14 +1,14 @@ package orca.tools.claude -import orca.llm.{BackendTag, ClaudeTool, LlmConfig, Model, Prompts} +import orca.agents.{BackendTag, ClaudeAgent, AgentConfig, Model, Prompts} import orca.events.{OrcaListener} import orca.backend.Interaction -import orca.llm.BaseLlmTool +import orca.agents.BaseAgent -/** Default ClaudeTool implementation. Inherits the autonomous-text + - * `resultAs[O]` plumbing from [[BaseLlmTool]] and only adds the - * Claude-specific model accessors (`haiku` / `sonnet` / `opus`). +/** Default ClaudeAgent implementation. Inherits the autonomous-text + + * `resultAs[O]` plumbing from [[BaseAgent]] and only adds the Claude-specific + * model accessors (`haiku` / `sonnet` / `opus`). * * Free-form text `autonomous.run` and structured `resultAs[O].autonomous.run` * go through the backend's headless mode. Interactive structured calls @@ -16,15 +16,15 @@ import orca.llm.BaseLlmTool * the subprocess in a `Conversation` that the supplied `interaction` drives to * completion. */ -private[orca] class DefaultClaudeTool( +private[orca] class DefaultClaudeAgent( backend: ClaudeBackend, - config: LlmConfig, + config: AgentConfig, prompts: Prompts, workDir: os.Path, events: OrcaListener, interaction: Interaction, val name: String = "main" -) extends BaseLlmTool[BackendTag.ClaudeCode.type, ClaudeTool]( +) extends BaseAgent[BackendTag.ClaudeCode.type, ClaudeAgent]( backend, config, prompts, @@ -32,20 +32,20 @@ private[orca] class DefaultClaudeTool( events, interaction ) - with ClaudeTool: + with ClaudeAgent: - def haiku: ClaudeTool = withModel(Model("claude-haiku-4-5")) - def sonnet: ClaudeTool = withModel(Model("claude-sonnet-4-6")) - def opus: ClaudeTool = withModel(DefaultClaudeTool.Opus1M) - def fable: ClaudeTool = withModel(DefaultClaudeTool.Fable) + def haiku: ClaudeAgent = withModel(Model("claude-haiku-4-5")) + def sonnet: ClaudeAgent = withModel(Model("claude-sonnet-4-6")) + def opus: ClaudeAgent = withModel(DefaultClaudeAgent.Opus1M) + def fable: ClaudeAgent = withModel(DefaultClaudeAgent.Fable) /** Per the trait: configure the read-only network allowlist by swapping in a * reconfigured backend (the allowlist is claude-specific, so it lives there - * rather than in the shared `LlmConfig`). Constructs directly rather than + * rather than in the shared `AgentConfig`). Constructs directly rather than * via [[copyTool]] because the latter threads the current backend unchanged. */ - def withNetworkTools(tools: Seq[String]): ClaudeTool = - new DefaultClaudeTool( + def withNetworkTools(tools: Seq[String]): ClaudeAgent = + new DefaultClaudeAgent( backend.withNetworkTools(tools), config, prompts, @@ -56,10 +56,10 @@ private[orca] class DefaultClaudeTool( ) protected def copyTool( - config: LlmConfig = config, + config: AgentConfig = config, name: String = name - ): ClaudeTool = - new DefaultClaudeTool( + ): ClaudeAgent = + new DefaultClaudeAgent( backend, config, prompts, @@ -69,7 +69,7 @@ private[orca] class DefaultClaudeTool( name ) -private[orca] object DefaultClaudeTool: +private[orca] object DefaultClaudeAgent: /** The default coding model: Opus with the 1M-token context window, selected * via the documented `[1m]` model-alias suffix (Claude Code model-config; no * beta header needed). The main implementer session is long-lived and diff --git a/claude/src/test/scala/orca/tools/claude/ClaudeArgsTest.scala b/claude/src/test/scala/orca/tools/claude/ClaudeArgsTest.scala index 2af61b03..93df01c7 100644 --- a/claude/src/test/scala/orca/tools/claude/ClaudeArgsTest.scala +++ b/claude/src/test/scala/orca/tools/claude/ClaudeArgsTest.scala @@ -1,7 +1,14 @@ package orca.tools.claude import orca.backend.Dispatch -import orca.llm.{AutoApprove, BackendTag, LlmConfig, Model, SessionId, ToolSet} +import orca.agents.{ + AutoApprove, + BackendTag, + AgentConfig, + Model, + SessionId, + ToolSet +} class ClaudeArgsTest extends munit.FunSuite: private val testSid = @@ -10,7 +17,7 @@ class ClaudeArgsTest extends munit.FunSuite: ) private def streamJson( - config: LlmConfig, + config: AgentConfig, dispatch: Dispatch[BackendTag.ClaudeCode.type] = Dispatch.Fresh(testSid), networkTools: Seq[String] = Seq.empty ): Seq[String] = @@ -22,24 +29,24 @@ class ClaudeArgsTest extends munit.FunSuite: ) test("stream-json shape: --print, --input/--output-format stream-json, etc."): - val args = streamJson(LlmConfig.default) + val args = streamJson(AgentConfig.default) assert(args.contains("--print")) assert(args.containsSlice(Seq("--input-format", "stream-json"))) assert(args.containsSlice(Seq("--output-format", "stream-json"))) assert(args.contains("--verbose")) assert(args.contains("--include-partial-messages")) - test("model flag is emitted when LlmConfig.model is set"): - val args = streamJson(LlmConfig(model = Some(Model("sonnet-4")))) + test("model flag is emitted when AgentConfig.model is set"): + val args = streamJson(AgentConfig(model = Some(Model("sonnet-4")))) assert(args.containsSlice(Seq("--model", "sonnet-4"))) - test("model flag is absent when LlmConfig.model is None"): - assert(!streamJson(LlmConfig.default).contains("--model")) + test("model flag is absent when AgentConfig.model is None"): + assert(!streamJson(AgentConfig.default).contains("--model")) test("system-prompt-file flag is emitted when a file is supplied"): val file = os.temp(contents = "content") val args = ClaudeArgs.streamJson( - config = LlmConfig.default, + config = AgentConfig.default, systemPromptFile = Some(file), dispatch = Dispatch.Fresh(testSid) ) @@ -48,20 +55,22 @@ class ClaudeArgsTest extends munit.FunSuite: ) test("AutoApprove.All maps to --permission-mode bypassPermissions"): - val args = streamJson(LlmConfig(autoApprove = AutoApprove.All)) + val args = streamJson(AgentConfig(autoApprove = AutoApprove.All)) assert(args.containsSlice(Seq("--permission-mode", "bypassPermissions"))) assert(!args.contains("--allowedTools")) test("AutoApprove.Only(tools) maps to acceptEdits + sorted --allowedTools"): val args = streamJson( - LlmConfig(autoApprove = AutoApprove.Only(Set("Zeta", "Alpha", "Middle"))) + AgentConfig(autoApprove = + AutoApprove.Only(Set("Zeta", "Alpha", "Middle")) + ) ) assert(args.containsSlice(Seq("--permission-mode", "acceptEdits"))) assert(args.containsSlice(Seq("--allowedTools", "Alpha,Middle,Zeta"))) test("AutoApprove.Only(empty) maps to acceptEdits with no --allowedTools"): val args = - streamJson(LlmConfig(autoApprove = AutoApprove.Only(Set.empty))) + streamJson(AgentConfig(autoApprove = AutoApprove.Only(Set.empty))) assert(args.containsSlice(Seq("--permission-mode", "acceptEdits"))) assert(!args.contains("--allowedTools")) @@ -72,7 +81,7 @@ class ClaudeArgsTest extends munit.FunSuite: // Edit/Write/Bash unavailable, not just non-auto-approved. It wins over // `autoApprove` (the agent is verifying claims, not editing). val args = streamJson( - LlmConfig(autoApprove = AutoApprove.All, tools = ToolSet.ReadOnly) + AgentConfig(autoApprove = AutoApprove.All, tools = ToolSet.ReadOnly) ) assert(args.containsSlice(Seq("--permission-mode", "plan")), args) assert(!args.contains("bypassPermissions"), args) @@ -82,7 +91,7 @@ class ClaudeArgsTest extends munit.FunSuite: "ToolSet.NetworkOnly layers networkTools onto plan mode via --allowedTools" ): val args = streamJson( - LlmConfig(tools = ToolSet.NetworkOnly), + AgentConfig(tools = ToolSet.NetworkOnly), networkTools = Seq("WebFetch", "Bash(gh api:*)") ) assert(args.containsSlice(Seq("--permission-mode", "plan")), args) @@ -92,21 +101,22 @@ class ClaudeArgsTest extends munit.FunSuite: ) test("ToolSet.NetworkOnly with no networkTools stays plain plan mode"): - val args = streamJson(LlmConfig(tools = ToolSet.NetworkOnly)) + val args = streamJson(AgentConfig(tools = ToolSet.NetworkOnly)) assert(args.containsSlice(Seq("--permission-mode", "plan")), args) assert(!args.contains("--allowedTools"), args) test("ToolSet.ReadOnly never emits networkTools even when supplied"): // Reviewers/triage use ReadOnly and must stay network-free. val args = streamJson( - LlmConfig(tools = ToolSet.ReadOnly), + AgentConfig(tools = ToolSet.ReadOnly), networkTools = Seq("WebFetch") ) assert(args.containsSlice(Seq("--permission-mode", "plan")), args) assert(!args.contains("--allowedTools"), args) test("Dispatch.Fresh emits --session-id <uuid>"): - val args = streamJson(LlmConfig.default, dispatch = Dispatch.Fresh(testSid)) + val args = + streamJson(AgentConfig.default, dispatch = Dispatch.Fresh(testSid)) assert( args.containsSlice(Seq("--session-id", SessionId.value(testSid))), args @@ -115,14 +125,14 @@ class ClaudeArgsTest extends munit.FunSuite: test("Dispatch.Resume emits --resume <uuid>"): val args = - streamJson(LlmConfig.default, dispatch = Dispatch.Resume(testSid)) + streamJson(AgentConfig.default, dispatch = Dispatch.Resume(testSid)) assert(args.containsSlice(Seq("--resume", SessionId.value(testSid))), args) assert(!args.contains("--session-id"), args) test("--json-schema is emitted when an output schema is supplied"): val schema = """{"type":"object"}""" val args = ClaudeArgs.streamJson( - config = LlmConfig.default, + config = AgentConfig.default, systemPromptFile = None, dispatch = Dispatch.Fresh(testSid), jsonSchema = Some(schema) @@ -132,7 +142,7 @@ class ClaudeArgsTest extends munit.FunSuite: test("--mcp-config <file> is emitted when supplied"): val cfg = os.temp() val args = ClaudeArgs.streamJson( - config = LlmConfig.default, + config = AgentConfig.default, systemPromptFile = None, dispatch = Dispatch.Fresh(testSid), mcpConfig = Some(cfg) @@ -142,7 +152,7 @@ class ClaudeArgsTest extends munit.FunSuite: test("all mappings compose: model + session + autoApprove + system-prompt"): val file = os.temp() val args = ClaudeArgs.streamJson( - config = LlmConfig( + config = AgentConfig( model = Some(Model("opus-4")), autoApprove = AutoApprove.Only(Set("Read")) ), diff --git a/claude/src/test/scala/orca/tools/claude/ClaudeBackendTest.scala b/claude/src/test/scala/orca/tools/claude/ClaudeBackendTest.scala index c55d98ba..569c129d 100644 --- a/claude/src/test/scala/orca/tools/claude/ClaudeBackendTest.scala +++ b/claude/src/test/scala/orca/tools/claude/ClaudeBackendTest.scala @@ -1,7 +1,7 @@ package orca.tools.claude import orca.backend.SupervisedBackend -import orca.llm.{BackendTag, LlmConfig, SessionId, ToolSet} +import orca.agents.{BackendTag, AgentConfig, SessionId, ToolSet} import orca.{OrcaFlowException} import orca.subprocess.{FakePipedCliProcess, SpawnStubCliRunner} @@ -33,7 +33,7 @@ class ClaudeBackendTest extends munit.FunSuite: p private def withBackend[T](runner: SpawnStubCliRunner)( - body: ClaudeBackend => T + body: ox.Ox ?=> ClaudeBackend => T ): T = SupervisedBackend.using(new ClaudeBackend(runner))(body) private def freshSid: SessionId[BackendTag.ClaudeCode.type] = @@ -48,7 +48,7 @@ class ClaudeBackendTest extends munit.FunSuite: backend.runAutonomous( "summarize", freshSid, - LlmConfig.default, + AgentConfig.default, os.temp.dir() ) val args = runner.calls.head @@ -63,7 +63,7 @@ class ClaudeBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "x", freshSid, - LlmConfig.default.copy(tools = ToolSet.NetworkOnly), + AgentConfig.default.copy(tools = ToolSet.NetworkOnly), os.temp.dir() ) val args = runner.calls.head @@ -80,7 +80,7 @@ class ClaudeBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "x", freshSid, - LlmConfig.default.copy(tools = ToolSet.NetworkOnly), + AgentConfig.default.copy(tools = ToolSet.NetworkOnly), os.temp.dir() ) val args = runner.calls.head @@ -97,7 +97,7 @@ class ClaudeBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "x", freshSid, - LlmConfig.default, + AgentConfig.default, os.temp.dir(), outputSchema = Some("""{"type":"object"}""") ) @@ -111,7 +111,7 @@ class ClaudeBackendTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(successfulProcess())) withBackend(runner): backend => val result = - backend.runAutonomous("x", freshSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("x", freshSid, AgentConfig.default, os.temp.dir()) assertEquals(SessionId.value(result.sessionId), "sess-123") assertEquals(result.output, "hello world") assertEquals(result.usage.inputTokens, 10L) @@ -131,7 +131,7 @@ class ClaudeBackendTest extends munit.FunSuite: p.sendSigInt() withBackend(new SpawnStubCliRunner(List(p))): backend => intercept[OrcaFlowException]: - backend.runAutonomous("x", freshSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("x", freshSid, AgentConfig.default, os.temp.dir()) test("runAutonomous throws when the subprocess exits non-zero"): val p = new FakePipedCliProcess(initiallyAlive = false): @@ -140,14 +140,14 @@ class ClaudeBackendTest extends munit.FunSuite: p.closeStderr() withBackend(new SpawnStubCliRunner(List(p))): backend => intercept[OrcaFlowException]: - backend.runAutonomous("x", freshSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("x", freshSid, AgentConfig.default, os.temp.dir()) test( "runAutonomous passes a --append-system-prompt-file pointing at the config's prompt" ): val runner = new SpawnStubCliRunner(List(successfulProcess())) withBackend(runner): backend => - val config = LlmConfig(systemPrompt = Some("you are a poet")) + val config = AgentConfig(systemPrompt = Some("you are a poet")) val _ = backend.runAutonomous("x", freshSid, config, os.temp.dir()) val args = runner.calls.head val flagIdx = args.indexOf("--append-system-prompt-file") @@ -171,9 +171,9 @@ class ClaudeBackendTest extends munit.FunSuite: ) withBackend(runner): backend => val _ = - backend.runAutonomous("first", sid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("first", sid, AgentConfig.default, os.temp.dir()) val _ = - backend.runAutonomous("again", sid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("again", sid, AgentConfig.default, os.temp.dir()) val first = runner.calls(0) val second = runner.calls(1) assert( @@ -185,6 +185,51 @@ class ClaudeBackendTest extends munit.FunSuite: second ) + test( + "registerSession (rehydrate on resume) makes the first call use --resume, not --session-id" + ): + // Regression for the cross-process resume bug: claude's sessions are + // durable on disk, so a resumed run re-claims the recorded id via + // `registerSession` (what `rehydrateSessions` calls). The very first call in + // THIS process must then `--resume` the existing session rather than + // re-create it with `--session-id` (which the CLI rejects as "already in + // use"). Before the fix, claude wired neither hook, so it always re-created. + val sid = SessionId[BackendTag.ClaudeCode.type]( + "44444444-4444-4444-4444-444444444444" + ) + val runner = new SpawnStubCliRunner(List(successfulProcess())) + withBackend(runner): backend => + backend.registerSession(sid, sid) + val _ = + backend.runAutonomous( + "continue", + sid, + AgentConfig.default, + os.temp.dir() + ) + val args = runner.calls.head + assert(args.containsSlice(Seq("--resume", SessionId.value(sid))), args) + assert(!args.contains("--session-id"), args) + + test( + "resumeWireId reflects the claim so the runtime records the resumable id" + ): + // `resumeWireId` is the source the runtime reads to write the resume wire id + // into the progress log; without it (the old default `None`) the claim was + // never persisted and resume re-created the session. + val sid = SessionId[BackendTag.ClaudeCode.type]( + "55555555-5555-5555-5555-555555555555" + ) + val runner = new SpawnStubCliRunner(List(successfulProcess())) + withBackend(runner): backend => + assertEquals(backend.resumeWireId(sid), None) // unclaimed + val _ = + backend.runAutonomous("hi", sid, AgentConfig.default, os.temp.dir()) + assertEquals( + backend.resumeWireId(sid), + Some(sid) + ) // claimed → persistable + test( "failed first call leaves the session unclaimed; retry still uses --session-id" ): @@ -208,11 +253,66 @@ class ClaudeBackendTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(failing, successfulProcess())) withBackend(runner): backend => val _ = intercept[OrcaFlowException]: - backend.runAutonomous("first", sid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("first", sid, AgentConfig.default, os.temp.dir()) val _ = - backend.runAutonomous("retry", sid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("retry", sid, AgentConfig.default, os.temp.dir()) val second = runner.calls(1) assert( second.containsSlice(Seq("--session-id", SessionId.value(sid))), s"retry after failure must re-claim with --session-id; got: $second" ) + + test("sessionExists returns true when the transcript file exists"): + val tmpProjects = os.temp.dir() + val cwd = os.temp.dir() + val slug = ClaudeBackend.cwdSlug(cwd) + os.makeDir.all(tmpProjects / slug) + os.write(tmpProjects / slug / s"${SessionId.value(freshSid)}.jsonl", "") + SupervisedBackend.using( + new ClaudeBackend( + new SpawnStubCliRunner(Nil), + projectsDir = tmpProjects, + cwdForProbe = cwd + ) + ): backend => + assert(backend.sessionExists(freshSid)) + + test("sessionExists returns false when the transcript file is absent"): + val tmpProjects = os.temp.dir() + val cwd = os.temp.dir() + SupervisedBackend.using( + new ClaudeBackend( + new SpawnStubCliRunner(Nil), + projectsDir = tmpProjects, + cwdForProbe = cwd + ) + ): backend => + assert(!backend.sessionExists(freshSid)) + + test("sessionExists returns false when the projects dir is absent"): + val missing = os.temp.dir() / "no-such-dir" + val cwd = os.temp.dir() + SupervisedBackend.using( + new ClaudeBackend( + new SpawnStubCliRunner(Nil), + projectsDir = missing, + cwdForProbe = cwd + ) + ): backend => + assert(!backend.sessionExists(freshSid)) + + test( + "sessionExists returns false for a malicious id with path traversal chars" + ): + val tmpProjects = os.temp.dir() + val cwd = os.temp.dir() + SupervisedBackend.using( + new ClaudeBackend( + new SpawnStubCliRunner(Nil), + projectsDir = tmpProjects, + cwdForProbe = cwd + ) + ): backend => + val maliciousId = + SessionId[BackendTag.ClaudeCode.type]("../../etc/passwd") + assert(!backend.sessionExists(maliciousId)) diff --git a/claude/src/test/scala/orca/tools/claude/ClaudeConversationTest.scala b/claude/src/test/scala/orca/tools/claude/ClaudeConversationTest.scala index db6a08f1..da89bbe0 100644 --- a/claude/src/test/scala/orca/tools/claude/ClaudeConversationTest.scala +++ b/claude/src/test/scala/orca/tools/claude/ClaudeConversationTest.scala @@ -1,16 +1,25 @@ package orca.tools.claude -import orca.llm.{AutoApprove, BackendTag, LlmConfig} +import orca.agents.{AutoApprove, BackendTag, AgentConfig} import orca.events.{Usage} import orca.{OrcaFlowException, OrcaInteractiveCancelled} import orca.backend.{ApprovalDecision, ConversationEvent} import orca.subprocess.FakePipedCliProcess +import ox.{Ox, supervised} class ClaudeConversationTest extends munit.FunSuite: - test("stream_event text_delta becomes AssistantTextDelta"): + /** `ClaudeConversation` forks its reader/stderr/ask-user workers into the + * caller's per-turn Ox, so construction needs a `using Ox`. Run each test + * body in a fresh supervised scope that provides it (and joins the forks on + * exit). The ask-user test manages its own scope and stays on plain `test`. + */ + private def convTest(name: String)(body: Ox ?=> Unit): Unit = + test(name)(supervised(body)) + + convTest("stream_event text_delta becomes AssistantTextDelta"): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout( """{"type":"stream_event","event":{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hello"}}}""" @@ -24,9 +33,9 @@ class ClaudeConversationTest extends munit.FunSuite: assertEquals(events, List(ConversationEvent.AssistantTextDelta("hello"))) val _ = conv.awaitResult() - test("result message finishes the session and carries usage"): + convTest("result message finishes the session and carries usage"): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout( """{"type":"result","subtype":"success","session_id":"sid-2","result":"done","usage":{"input_tokens":5,"output_tokens":7}}""" @@ -38,9 +47,11 @@ class ClaudeConversationTest extends munit.FunSuite: assertEquals(result.output, "done") assertEquals(result.usage, Usage(5L, 7L, None)) - test("is_error after streaming deltas emits a short marker, not a duplicate"): + convTest( + "is_error after streaming deltas emits a short marker, not a duplicate" + ): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout( """{"type":"stream_event","event":{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"API Error: 400 quota exceeded"}}}""" @@ -67,11 +78,11 @@ class ClaudeConversationTest extends munit.FunSuite: s"awaitResult should still carry the full body; got: ${failure.getMessage}" ) - test( + convTest( "result message with is_error=true fails the session and surfaces the message" ): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout( """{"type":"result","subtype":"error","session_id":"sid-err","result":"API Error: 400 rate limited","is_error":true}""" @@ -89,9 +100,11 @@ class ClaudeConversationTest extends munit.FunSuite: val failure = intercept[OrcaFlowException](conv.awaitResult()) assert(failure.getMessage.contains("rate limited")) - test("cancel surfaces as Left(OrcaInteractiveCancelled) from awaitResult"): + convTest( + "cancel surfaces as Left(OrcaInteractiveCancelled) from awaitResult" + ): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) conv.cancel() conv.awaitResult() match @@ -100,13 +113,13 @@ class ClaudeConversationTest extends munit.FunSuite: fail(s"expected Left(OrcaInteractiveCancelled), got: $other") assertEquals(process.sigIntCount, 1) - test( + convTest( "can_use_tool with autoApprove=All responds allow without emitting an event" ): val process = new FakePipedCliProcess() val conv = new ClaudeConversation( process, - LlmConfig.default.copy(autoApprove = AutoApprove.All) + AgentConfig.default.copy(autoApprove = AutoApprove.All) ) process.enqueueStdout( @@ -126,13 +139,13 @@ class ClaudeConversationTest extends munit.FunSuite: s"expected allow response, got: ${process.writes.head}" ) - test( + convTest( "can_use_tool with autoApprove=Only not matching emits ApproveTool for the channel" ): val process = new FakePipedCliProcess() val conv = new ClaudeConversation( process, - LlmConfig.default.copy(autoApprove = AutoApprove.Only(Set("Read"))) + AgentConfig.default.copy(autoApprove = AutoApprove.Only(Set("Read"))) ) process.enqueueStdout( @@ -162,23 +175,7 @@ class ClaudeConversationTest extends munit.FunSuite: ) assert(denyLine.get.contains("too risky")) - test("sendUserMessage writes a stream-json user turn to stdin"): - val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) - - conv.sendUserMessage("keep going") - val injected = process.writes.headOption - assert(injected.isDefined, "expected a stdin write") - assert(injected.get.contains(""""type":"user"""")) - assert(injected.get.contains(""""text":"keep going"""")) - - process.enqueueStdout( - """{"type":"result","subtype":"success","session_id":"sid-5"}""" - ) - process.closeStdout() - val _ = conv.awaitResult() - - test( + convTest( "tool_use surrounding streaming events are ignored; emission comes from the full-turn message" ): // In claude's live protocol the `assistant` message arrives BEFORE @@ -190,7 +187,7 @@ class ClaudeConversationTest extends munit.FunSuite: // (if it ever flipped). Pin both: feed the streaming events, then // the full-turn message, and expect exactly one AssistantToolCall. val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout( """{"type":"stream_event","event":{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"id-1","name":"Bash","input":{}}}}""" @@ -223,11 +220,11 @@ class ClaudeConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test( + convTest( "assistant turn with text falls back to an AssistantTextDelta when no partials streamed" ): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout( """{"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"no-partials"}]}}""" @@ -247,9 +244,9 @@ class ClaudeConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("user turn with tool_result blocks emits ToolResult events"): + convTest("user turn with tool_result blocks emits ToolResult events"): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout( """{"type":"user","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"id-1","content":"output","is_error":false}]}}""" @@ -272,11 +269,11 @@ class ClaudeConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test( + convTest( "malformed NDJSON line surfaces as ConversationEvent.Error and the loop continues" ): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) process.enqueueStdout("this is not json") process.enqueueStdout( @@ -294,11 +291,11 @@ class ClaudeConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("autoApprove.Only matches the tool → silent allow"): + convTest("autoApprove.Only matches the tool → silent allow"): val process = new FakePipedCliProcess() val conv = new ClaudeConversation( process, - LlmConfig.default.copy(autoApprove = AutoApprove.Only(Set("Read"))) + AgentConfig.default.copy(autoApprove = AutoApprove.Only(Set("Read"))) ) process.enqueueStdout( @@ -314,13 +311,13 @@ class ClaudeConversationTest extends munit.FunSuite: val _ = conv.awaitResult() assert(process.writes.head.contains(""""behavior":"allow"""")) - test( + convTest( "multiple back-to-back ApproveTool events carry distinct respond closures" ): val process = new FakePipedCliProcess() val conv = new ClaudeConversation( process, - LlmConfig.default.copy(autoApprove = AutoApprove.Only(Set.empty)) + AgentConfig.default.copy(autoApprove = AutoApprove.Only(Set.empty)) ) process.enqueueStdout( @@ -368,7 +365,7 @@ class ClaudeConversationTest extends munit.FunSuite: val askUser = AskUserSession.allocate() val conv = new ClaudeConversation( process, - LlmConfig.default, + AgentConfig.default, askUser = Some(askUser) ) val bridge = askUser.bridge @@ -396,18 +393,18 @@ class ClaudeConversationTest extends munit.FunSuite: val _ = conv.events.toList val _ = conv.awaitResult() - test("canAskUser is false when no bridge is provided"): + convTest("canAskUser is false when no bridge is provided"): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) assertEquals(conv.canAskUser, false) process.closeStdout() val _ = conv.events.toList - test( + convTest( "handleAssistantTurn suppresses the agent's ToolUse for ask_user" ): val process = new FakePipedCliProcess() - val conv = new ClaudeConversation(process, LlmConfig.default) + val conv = new ClaudeConversation(process, AgentConfig.default) // Assistant turn carrying a tool_use block for the MCP-prefixed // ask_user tool name. Our renderer-side suppression should drop the diff --git a/claude/src/test/scala/orca/tools/claude/ClaudeIntegrationTest.scala b/claude/src/test/scala/orca/tools/claude/ClaudeIntegrationTest.scala index 0824ba2d..32cc8827 100644 --- a/claude/src/test/scala/orca/tools/claude/ClaudeIntegrationTest.scala +++ b/claude/src/test/scala/orca/tools/claude/ClaudeIntegrationTest.scala @@ -1,6 +1,6 @@ package orca.tools.claude -import orca.llm.{AutoApprove, BackendTag, LlmConfig, SessionId} +import orca.agents.{AutoApprove, BackendTag, AgentConfig, SessionId} import orca.backend.{ApprovalDecision, ConversationEvent, SupervisedBackend} import orca.subprocess.OsProcCliRunner @@ -19,7 +19,7 @@ class ClaudeIntegrationTest extends munit.FunSuite: import scala.concurrent.duration.DurationInt 2.minutes - private def withBackend(body: ClaudeBackend => Unit): Unit = + private def withBackend(body: ox.Ox ?=> ClaudeBackend => Unit): Unit = SupervisedBackend.using(new ClaudeBackend(OsProcCliRunner))(body) private def fresh = SessionId.fresh[BackendTag.ClaudeCode.type] @@ -29,7 +29,7 @@ class ClaudeIntegrationTest extends munit.FunSuite: val result = backend.runAutonomous( prompt = "Reply with the single word: READY", session = fresh, - config = LlmConfig.default, + config = AgentConfig.default, workDir = os.temp.dir() ) assert( @@ -45,13 +45,13 @@ class ClaudeIntegrationTest extends munit.FunSuite: val _ = backend.runAutonomous( prompt = "Remember the number 42. Reply with: stored.", session = session, - config = LlmConfig.default, + config = AgentConfig.default, workDir = workDir ) val second = backend.runAutonomous( prompt = "What number did I ask you to remember?", session = session, - config = LlmConfig.default, + config = AgentConfig.default, workDir = workDir ) assert( @@ -65,7 +65,7 @@ class ClaudeIntegrationTest extends munit.FunSuite: prompt = "Reply with just the number 7. Nothing else.", session = fresh, displayPrompt = "reply with 7", - config = LlmConfig.default, + config = AgentConfig.default, workDir = os.temp.dir(), outputSchema = None ) @@ -88,7 +88,7 @@ class ClaudeIntegrationTest extends munit.FunSuite: "Count from 1 to 5, one per line, then stop. Do not emit anything else.", session = fresh, displayPrompt = "count 1..5", - config = LlmConfig.default, + config = AgentConfig.default, workDir = os.temp.dir(), outputSchema = None ) @@ -119,7 +119,7 @@ class ClaudeIntegrationTest extends munit.FunSuite: session = fresh, displayPrompt = "read /etc/hostname", config = - LlmConfig.default.copy(autoApprove = AutoApprove.Only(Set.empty)), + AgentConfig.default.copy(autoApprove = AutoApprove.Only(Set.empty)), workDir = os.temp.dir(), outputSchema = None ) diff --git a/claude/src/test/scala/orca/tools/claude/DefaultLlmCallTest.scala b/claude/src/test/scala/orca/tools/claude/DefaultAgentCallTest.scala similarity index 87% rename from claude/src/test/scala/orca/tools/claude/DefaultLlmCallTest.scala rename to claude/src/test/scala/orca/tools/claude/DefaultAgentCallTest.scala index b7ec4b13..6a5c6465 100644 --- a/claude/src/test/scala/orca/tools/claude/DefaultLlmCallTest.scala +++ b/claude/src/test/scala/orca/tools/claude/DefaultAgentCallTest.scala @@ -1,11 +1,11 @@ package orca.tools.claude import orca.{AgentTurnFailed, OrcaFlowException} -import orca.llm.{BackendTag, JsonData, LlmConfig, SessionId} +import orca.agents.{BackendTag, JsonData, AgentConfig, SessionId} import orca.events.{OrcaListener, Usage} -import orca.backend.{Interaction, LlmBackend, LlmResult} -import orca.llm.{DefaultLlmCall, DefaultPrompts} +import orca.backend.{Interaction, AgentBackend, AgentResult} +import orca.agents.{DefaultAgentCall, DefaultPrompts} import ox.supervised import java.util.concurrent.atomic.{AtomicInteger, AtomicReference} @@ -17,7 +17,7 @@ case class Answer(value: Int) derives JsonData * test here. */ class SequencedBackend(outputs: List[String]) - extends LlmBackend[BackendTag.ClaudeCode.type]: + extends AgentBackend[BackendTag.ClaudeCode.type]: private val remaining: AtomicReference[List[String]] = AtomicReference(outputs) private val promptsRef: AtomicReference[List[String]] = @@ -36,19 +36,19 @@ class SequencedBackend(outputs: List[String]) def prompts: List[String] = promptsRef.get().reverse /** Listeners the backend was called with, in invocation order. Lets tests - * assert that `DefaultLlmCall` threaded its own `events` through rather than - * silently dropping it on the floor. + * assert that `DefaultAgentCall` threaded its own `events` through rather + * than silently dropping it on the floor. */ def events: List[orca.events.OrcaListener] = seenEvents.get().reverse /** `outputSchema` values the backend received, in invocation order. Lets - * tests assert that `DefaultLlmCall` actually passes `Some(<schema>)` rather - * than dropping to `None`. + * tests assert that `DefaultAgentCall` actually passes `Some(<schema>)` + * rather than dropping to `None`. */ def schemas: List[Option[String]] = seenSchemas.get().reverse /** `(clientSid, serverSid)` pairs the framework passed to `registerSession`, - * in invocation order. Lets tests assert that `DefaultLlmCall` wired the + * in invocation order. Lets tests assert that `DefaultAgentCall` wired the * post-drain hook through to the backend. */ def registered: List[ @@ -67,11 +67,11 @@ class SequencedBackend(outputs: List[String]) def runAutonomous( prompt: String, session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: orca.events.OrcaListener, outputSchema: Option[String] - ): LlmResult[BackendTag.ClaudeCode.type] = + ): AgentResult[BackendTag.ClaudeCode.type] = val _ = seenEvents.updateAndGet(events :: _) val _ = seenSchemas.updateAndGet(outputSchema :: _) nextResult(prompt).copy(sessionId = session) @@ -80,36 +80,39 @@ class SequencedBackend(outputs: List[String]) prompt: String, session: SessionId[BackendTag.ClaudeCode.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): orca.backend.Conversation[BackendTag.ClaudeCode.type] = + )(using ox.Ox): orca.backend.Conversation[BackendTag.ClaudeCode.type] = // Minimal stand-in: the conversation is not actually driven — the test's - // `Interaction.drive` ignores it and returns a canned `LlmResult`. We + // `Interaction.drive` ignores it and returns a canned `AgentResult`. We // still need *something* to return so the interactive path compiles. new orca.backend.Conversation[BackendTag.ClaudeCode.type]: val outputSchema: Option[String] = None val events: Iterator[orca.backend.ConversationEvent] = Iterator.empty def awaitResult() = throw new UnsupportedOperationException("test stub") - def sendUserMessage(text: String): Unit = () def canAskUser: Boolean = false def cancel(): Unit = () private def nextResult( prompt: String - ): LlmResult[BackendTag.ClaudeCode.type] = + ): AgentResult[BackendTag.ClaudeCode.type] = val _ = promptsRef.updateAndGet(prompt :: _) val next = remaining .getAndUpdate(rs => rs.drop(1)) .headOption .getOrElse(throw new IllegalStateException("ran out of canned outputs")) - LlmResult( + AgentResult( sessionId = SessionId[BackendTag.ClaudeCode.type]("sess-test"), output = next, usage = Usage.empty ) -class DefaultLlmCallTest extends munit.FunSuite: +class DefaultAgentCallTest extends munit.FunSuite: + + // LLM `run` is now gated on `InStage`; mint the token once for the suite + // (package `orca.tools.claude` can reach `InStage.unsafe`). + private given orca.InStage = orca.InStage.unsafe import scala.concurrent.duration.DurationInt @@ -121,12 +124,12 @@ class DefaultLlmCallTest extends munit.FunSuite: val listeners: List[OrcaListener] = Nil def drive[B <: BackendTag]( conversation: orca.backend.Conversation[B] - ): LlmResult[B] = throw new UnsupportedOperationException("test stub") + ): AgentResult[B] = throw new UnsupportedOperationException("test stub") private def makeCall( backend: SequencedBackend - ): DefaultLlmCall[BackendTag.ClaudeCode.type, Answer] = - new DefaultLlmCall[BackendTag.ClaudeCode.type, Answer]( + ): DefaultAgentCall[BackendTag.ClaudeCode.type, Answer] = + new DefaultAgentCall[BackendTag.ClaudeCode.type, Answer]( backend = backend, effectiveConfig = cfg => cfg.copy(retrySchedule = fastRetry), prompts = DefaultPrompts, @@ -197,11 +200,11 @@ class DefaultLlmCallTest extends munit.FunSuite: // A specific Announce[Answer] wins over Announce.default; the call // emits a single StructuredResult event carrying both the raw // payload (the agent's JSON) and the summary derived from Announce. - given orca.llm.Announce[Answer] = - orca.llm.Announce.from(a => s"answer is ${a.value}") + given orca.agents.Announce[Answer] = + orca.agents.Announce.from(a => s"answer is ${a.value}") val backend = new SequencedBackend(List("""{"value":99}""")) val seen = AtomicReference[List[orca.events.OrcaEvent]](Nil) - val call = new DefaultLlmCall[BackendTag.ClaudeCode.type, Answer]( + val call = new DefaultAgentCall[BackendTag.ClaudeCode.type, Answer]( backend = backend, effectiveConfig = cfg => cfg.copy(retrySchedule = fastRetry), prompts = DefaultPrompts, @@ -246,11 +249,11 @@ class DefaultLlmCallTest extends munit.FunSuite: // Without this wiring, structured calls (which is what every reviewer // uses) lose tool-use / assistant-message visibility — the per-turn // events fire only when the backend gets the same listener the - // DefaultLlmCall was constructed with. + // DefaultAgentCall was constructed with. val backend = new SequencedBackend(List("""{"value":1}""")) val myListener: orca.events.OrcaListener = (_: orca.events.OrcaEvent) => () supervised: - val _ = new DefaultLlmCall[BackendTag.ClaudeCode.type, Answer]( + val _ = new DefaultAgentCall[BackendTag.ClaudeCode.type, Answer]( backend = backend, effectiveConfig = cfg => cfg.copy(retrySchedule = fastRetry), prompts = DefaultPrompts, @@ -265,12 +268,12 @@ class DefaultLlmCallTest extends munit.FunSuite: "autonomous emits StructuredResult with summary=None under default Announce" ): // The library's catch-all `Announce.default` returns an empty - // string, which DefaultLlmCall normalises to `None` so listeners + // string, which DefaultAgentCall normalises to `None` so listeners // can pattern-match without an empty-string sentinel. val backend = new SequencedBackend(List("""{"value":1}""")) val seen = AtomicReference[List[orca.events.OrcaEvent]](Nil) supervised: - val _ = new DefaultLlmCall[BackendTag.ClaudeCode.type, Answer]( + val _ = new DefaultAgentCall[BackendTag.ClaudeCode.type, Answer]( backend = backend, effectiveConfig = cfg => cfg.copy(retrySchedule = fastRetry), prompts = DefaultPrompts, @@ -301,7 +304,7 @@ class DefaultLlmCallTest extends munit.FunSuite: ) val seen = AtomicReference[List[orca.events.OrcaEvent]](Nil) supervised: - val _ = new DefaultLlmCall[BackendTag.ClaudeCode.type, Answer]( + val _ = new DefaultAgentCall[BackendTag.ClaudeCode.type, Answer]( backend = backend, effectiveConfig = cfg => cfg.copy(retrySchedule = fastRetry), prompts = DefaultPrompts, @@ -337,11 +340,11 @@ class DefaultLlmCallTest extends munit.FunSuite: override def runAutonomous( prompt: String, session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener, outputSchema: Option[String] - ): LlmResult[BackendTag.ClaudeCode.type] = + ): AgentResult[BackendTag.ClaudeCode.type] = val _ = calls.incrementAndGet() throw new AgentTurnFailed("Prompt is too long") supervised: @@ -362,11 +365,11 @@ class DefaultLlmCallTest extends munit.FunSuite: override def runAutonomous( prompt: String, session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener, outputSchema: Option[String] - ): LlmResult[BackendTag.ClaudeCode.type] = + ): AgentResult[BackendTag.ClaudeCode.type] = if calls.getAndIncrement() == 0 then throw new OrcaFlowException( "Failed to open claude stream-json session: Broken pipe" @@ -393,7 +396,7 @@ class DefaultLlmCallTest extends munit.FunSuite: // `interaction.drive` returns, and restamp the returned id to the // caller-supplied `session` so a follow-up `.run(prompt, sid)` resumes // the right thread. Removing the `backend.registerSession` line in - // `DefaultLlmCall.runInteractiveOnce` would fail this test. + // `DefaultAgentCall.runInteractiveOnce` would fail this test. val clientSid = SessionId[BackendTag.ClaudeCode.type]("client-uuid-aaaa") val serverSid = @@ -403,14 +406,14 @@ class DefaultLlmCallTest extends munit.FunSuite: val listeners: List[OrcaListener] = Nil def drive[B <: BackendTag]( conversation: orca.backend.Conversation[B] - ): LlmResult[B] = - LlmResult[B]( + ): AgentResult[B] = + AgentResult[B]( sessionId = SessionId[B](SessionId.value(serverSid)), output = """{"value":3}""", usage = Usage.empty ) supervised: - val (returned, answer) = new DefaultLlmCall[ + val (returned, answer) = new DefaultAgentCall[ BackendTag.ClaudeCode.type, Answer ]( diff --git a/codex/src/main/scala/orca/tools/codex/CodexArgs.scala b/codex/src/main/scala/orca/tools/codex/CodexArgs.scala index 08709caa..e81431d5 100644 --- a/codex/src/main/scala/orca/tools/codex/CodexArgs.scala +++ b/codex/src/main/scala/orca/tools/codex/CodexArgs.scala @@ -2,9 +2,9 @@ package orca.tools.codex import orca.backend.CliArgs import orca.backend.mcp.AskUserMcpServer -import orca.llm.{AutoApprove, BackendTag, LlmConfig, SessionId, ToolSet} +import orca.agents.{AutoApprove, BackendTag, AgentConfig, SessionId, ToolSet} -/** Maps `LlmConfig` fields to `codex exec` CLI flags. `systemPrompt` is not +/** Maps `AgentConfig` fields to `codex exec` CLI flags. `systemPrompt` is not * handled here — codex doesn't accept an `--append-system-prompt` equivalent * on `exec`, so the backend folds it into the user prompt before this method * runs. `onUnapproved` and `retrySchedule` have no CLI shape and live at the @@ -19,7 +19,7 @@ private[codex] object CodexArgs: /** Single-turn `codex exec --json [<prompt>]` invocation. */ def exec( prompt: String, - config: LlmConfig, + config: AgentConfig, outputSchemaFile: Option[os.Path], workDir: os.Path, mcpServerUrl: Option[String] = None @@ -40,13 +40,19 @@ private[codex] object CodexArgs: /** Multi-turn continuation: `codex exec resume <id> <prompt>`. * - * Two limitations vs. [[exec]]: + * Three limitations vs. [[exec]]: * - `exec resume` doesn't accept `--cd / -C`, so cwd is set on the OS * process spawn rather than the argv. * - `exec resume` doesn't accept `--output-schema`, so the resumed turn's * structured-output validation falls to the prompt template + the * post-hoc parser. The retry-with- corrective-prompt loop in - * `DefaultLlmCall` handles parse failures. + * `DefaultAgentCall` handles parse failures. + * - `exec resume` rejects `--sandbox <mode>` and `--full-auto` (it errors + * with "unexpected argument"): a resumed session inherits the sandbox it + * was created with. Only `--dangerously-bypass-approvals-and-sandbox` is + * still accepted, so [[resumeSandboxArgs]] keeps that one and drops the + * rest. Without this, every resumed turn (a fix iteration, a follow-up + * task on the same session, or a cross-process resume) fails. * * codex also enforces that the resumed session was not started with * `--ephemeral`; the backend never passes `--ephemeral` on `exec`, so resume @@ -55,18 +61,34 @@ private[codex] object CodexArgs: def execResume( sessionId: SessionId[BackendTag.Codex.type], prompt: String, - config: LlmConfig, + config: AgentConfig, mcpServerUrl: Option[String] = None ): Seq[String] = Seq("codex") ++ mcpServerArgs(mcpServerUrl) ++ networkConfigArgs(config) ++ Seq("exec", "resume", "--json", SessionId.value(sessionId)) ++ - sandboxArgs(config) ++ + resumeSandboxArgs(config) ++ CliArgs.modelArgs(config) ++ Seq("--skip-git-repo-check") ++ Seq(prompt) + /** Sandbox flags accepted by `exec resume` (a subset of [[sandboxArgs]]). The + * resumed session inherits its sandbox from creation, and `exec resume` + * rejects `--sandbox <mode>` / `--full-auto` outright, so those map to no + * flag here. Only `--dangerously-bypass-approvals-and-sandbox` (Full + + * [[AutoApprove.All]]) is accepted and is re-asserted each turn to keep + * approvals off. + */ + private def resumeSandboxArgs(config: AgentConfig): Seq[String] = + config.tools match + case ToolSet.Full => + config.autoApprove match + case AutoApprove.All => + Seq("--dangerously-bypass-approvals-and-sandbox") + case AutoApprove.Only(_) => Seq.empty + case ToolSet.ReadOnly | ToolSet.NetworkOnly => Seq.empty + /** Top-level `-c mcp_servers.<name>.{url,tool_timeout_sec}` overrides. Placed * BEFORE the subcommand so they land in codex's global-config slot (the * subcommand inherits them). URL value is wrapped in TOML double-quotes @@ -97,11 +119,11 @@ private[codex] object CodexArgs: private def outputSchemaArgs(file: Option[os.Path]): Seq[String] = file.toSeq.flatMap(p => Seq("--output-schema", p.toString)) - /** Maps [[LlmConfig.tools]] to codex's sandbox flags (placed after the `exec` - * subcommand). `ReadOnly` uses `--sandbox read-only` (no writes, no shell - * side-effects), matching claude's `--permission-mode plan`. `Full` follows - * [[LlmConfig.autoApprove]]; codex has no per-tool CLI allowlist, so - * [[AutoApprove.Only]] is approximated with `--full-auto`. + /** Maps [[AgentConfig.tools]] to codex's sandbox flags (placed after the + * `exec` subcommand). `ReadOnly` uses `--sandbox read-only` (no writes, no + * shell side-effects), matching claude's `--permission-mode plan`. `Full` + * follows [[AgentConfig.autoApprove]]; codex has no per-tool CLI allowlist, + * so [[AutoApprove.Only]] is approximated with `--full-auto`. * * - `ReadOnly` → `--sandbox read-only` * - `NetworkOnly` → `--full-auto` (workspace-write + non-interactive @@ -115,7 +137,7 @@ private[codex] object CodexArgs: * no-edit guarantee there is prompt-only (the planning prompts forbid * edits). */ - private def sandboxArgs(config: LlmConfig): Seq[String] = + private def sandboxArgs(config: AgentConfig): Seq[String] = config.tools match case ToolSet.ReadOnly => Seq("--sandbox", "read-only") case ToolSet.NetworkOnly => Seq("--full-auto") @@ -132,7 +154,7 @@ private[codex] object CodexArgs: * this the planner's `gh`/`curl` calls would be blocked. Empty for the other * tiers. */ - private def networkConfigArgs(config: LlmConfig): Seq[String] = + private def networkConfigArgs(config: AgentConfig): Seq[String] = config.tools match case ToolSet.NetworkOnly => Seq("-c", "sandbox_workspace_write.network_access=true") diff --git a/codex/src/main/scala/orca/tools/codex/CodexBackend.scala b/codex/src/main/scala/orca/tools/codex/CodexBackend.scala index 644448b9..a82f3460 100644 --- a/codex/src/main/scala/orca/tools/codex/CodexBackend.scala +++ b/codex/src/main/scala/orca/tools/codex/CodexBackend.scala @@ -1,21 +1,21 @@ package orca.tools.codex import orca.events.OrcaListener -import orca.llm.{BackendTag, LlmConfig, SessionId} -import orca.{AgentTurnFailed, OrcaFlowException} +import orca.agents.{BackendTag, AgentConfig, SessionId} import orca.backend.{ Conversation, Conversations, Dispatch, - LlmBackend, - LlmResult, + AgentBackend, + AgentResult, SessionMode, SessionRegistry, + SubprocessSpawn, SystemPromptComposer } import orca.backend.mcp.{AskUserMcpServer, AskUserSession} import orca.subprocess.CliRunner -import ox.Ox +import ox.{Ox, supervised} import ox.channels.BufferCapacity /** Codex backend. Both autonomous and interactive paths drive `codex exec @@ -38,8 +38,32 @@ import ox.channels.BufferCapacity * call `ask_user` to surface a clarifying question to the user. Autonomous * calls skip the bridge entirely. */ -private[orca] class CodexBackend(cli: CliRunner)(using Ox, BufferCapacity) - extends LlmBackend[BackendTag.Codex.type]: +private[orca] class CodexBackend( + cli: CliRunner, + private[codex] val sessionsDir: os.Path = os.home / ".codex" / "sessions" +)(using BufferCapacity) + extends AgentBackend[BackendTag.Codex.type]: + + /** Best-effort probe: walks [[sessionsDir]] looking for a file whose name + * matches `rollout-*-<id>.jsonl`. Returns `false` — safe re-seed — when no + * match is found, the sessions dir doesn't exist, or the id fails the + * [[orca.agents.isSafeSessionId]] guard (blocks regex injection; e.g. + * id=`.*` would match every file without the guard). + * + * Note: the installed codex on some machines uses SQLite + * (`~/.codex/state_5.sqlite`) rather than `rollout-*.jsonl` files. If no + * matching files exist, the probe returns `false` → re-seed, which is always + * safe. + */ + override def sessionExists( + session: SessionId[BackendTag.Codex.type] + ): Boolean = + probeGuarded(SessionId.value(session)): id => + os.exists(sessionsDir) && os.walk + .stream(sessionsDir) + .exists(p => + p.last.startsWith("rollout-") && p.last.endsWith(s"-$id.jsonl") + ) /** Maps the client-allocated session id (the UUID the caller passes around) * to codex's server-allocated thread id (learned from `thread.started`). @@ -52,47 +76,46 @@ private[orca] class CodexBackend(cli: CliRunner)(using Ox, BufferCapacity) def runAutonomous( prompt: String, session: SessionId[BackendTag.Codex.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener = OrcaListener.noop, outputSchema: Option[String] = None - ): LlmResult[BackendTag.Codex.type] = - val conv = openConversation( - prompt = prompt, - mode = SessionMode.Autonomous, - session = session, - config = config, - workDir = workDir, - // Forwarded so (a) `conv.outputSchema` signals structured mode to the - // drain (suppressing the raw JSON payload from the user log) and (b) - // `--output-schema` enforces the contract on the codex side too. - // `exec resume` rejects `--output-schema`, so retries against an - // existing session fall back to prompt-only enforcement; the - // retry-with-corrective-prompt loop in `DefaultLlmCall` handles a - // resume that produces malformed JSON. - outputSchema = outputSchema - ) - try - val result = Conversations.drainAutonomous(conv, events) - sessions.commitSuccess(session, result.sessionId) + ): AgentResult[BackendTag.Codex.type] = + // Self-scoped: the conversation forks its workers into this per-call Ox, the + // drain consumes them, and `cancel` (the `finally`) tears the subprocess + + // forks down before the scope joins. + supervised: + val conv = openConversation( + prompt = prompt, + mode = SessionMode.Autonomous, + session = session, + config = config, + workDir = workDir, + // Forwarded so (a) `conv.outputSchema` signals structured mode to the + // drain (suppressing the raw JSON payload from the user log) and (b) + // `--output-schema` enforces the contract on the codex side too. + // `exec resume` rejects `--output-schema`, so retries against an + // existing session fall back to prompt-only enforcement; the + // retry-with-corrective-prompt loop in `DefaultAgentCall` handles a + // resume that produces malformed JSON. + outputSchema = outputSchema + ) // Hide the server-allocated id from the caller — they keep using the // client id they passed in. Future calls resolve via the registry. - result.copy(sessionId = session) - catch - // Preserve the non-retryable type: a turn that ran and failed must not - // be retried (it would reopen the now-registered session id). - case e: AgentTurnFailed => throw e - case e: OrcaFlowException => - throw new OrcaFlowException(s"codex CLI failed: ${e.getMessage}") + try + Conversations + .drainAndCommit("codex", conv, session, sessions, events) + .copy(sessionId = session) + finally conv.cancel() def runInteractive( prompt: String, session: SessionId[BackendTag.Codex.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Codex.type] = + )(using Ox): Conversation[BackendTag.Codex.type] = openConversation( prompt, mode = SessionMode.Interactive(displayPrompt), @@ -125,15 +148,15 @@ private[orca] class CodexBackend(cli: CliRunner)(using Ox, BufferCapacity) prompt: String, mode: SessionMode, session: SessionId[BackendTag.Codex.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Codex.type] = + )(using Ox): Conversation[BackendTag.Codex.type] = val (askUser, displayPrompt): (Option[AskUserSession], String) = mode match case SessionMode.Interactive(p) => (Some(AskUserSession.allocate()), p) case SessionMode.Autonomous => (None, "") - try + SubprocessSpawn.open("codex", askUser.toList) { // codex `exec` has no `--system-prompt` flag (it picks up `AGENTS.md` // files for static instructions), so fold the composed system prompt into // the user prompt. @@ -160,45 +183,34 @@ private[orca] class CodexBackend(cli: CliRunner)(using Ox, BufferCapacity) workDir, mcpServerUrl = mcpUrl ) - val process = cli.spawnPiped(args, cwd = workDir, pipeStderr = true) - try - // codex doesn't accept user turns over stdin once the initial - // prompt has been argv-supplied; close immediately so the - // child stops waiting on stdin EOF. Same reflex as claude's - // single-shot stream-json path. - process.closeStdin() - new CodexConversation( - process, - initialPrompt = displayPrompt, - outputSchema = outputSchema, - askUser = askUser - ) - catch - case e: Exception => - // SIGINT the process; the outer catch closes askUser. - process.sendSigInt() - throw OrcaFlowException( - s"Failed to open codex session: ${e.getMessage}" - ) - catch - case e: Throwable => - // Any failure between resource allocation and a fully-constructed - // CodexConversation: tear down the MCP server so the Netty - // binding doesn't leak. Once the conversation owns the resources - // they ride through `onFinalize`. - askUser.foreach(_.close()) - throw e + cli.spawnPiped(args, cwd = workDir, pipeStderr = true) + } { process => + // codex doesn't accept user turns over stdin once the initial prompt has + // been argv-supplied; close immediately so the child stops waiting on + // stdin EOF. Same reflex as claude's single-shot stream-json path. + process.closeStdin() + new CodexConversation( + process, + initialPrompt = displayPrompt, + outputSchema = outputSchema, + askUser = askUser + ) + } /** Record the server-allocated thread id so subsequent calls with the same * client id resume that thread. Called by [[runAutonomous]] post-drain and - * by [[orca.llm.DefaultLlmCall]] post-`interaction.drive` on the interactive - * path; delegates to the registry's `commitSuccess`. + * by [[orca.agents.DefaultAgentCall]] post-`interaction.drive` on the + * interactive path; delegates to the registry's `commitSuccess`. */ override def registerSession( client: SessionId[BackendTag.Codex.type], server: SessionId[BackendTag.Codex.type] ): Unit = sessions.commitSuccess(client, server) + override def resumeWireId( + client: SessionId[BackendTag.Codex.type] + ): Option[SessionId[BackendTag.Codex.type]] = sessions.resumeWireId(client) + private def writeSchemaIfPresent( schema: Option[String], workDir: os.Path diff --git a/codex/src/main/scala/orca/tools/codex/CodexConversation.scala b/codex/src/main/scala/orca/tools/codex/CodexConversation.scala index 24b160f4..71dc8433 100644 --- a/codex/src/main/scala/orca/tools/codex/CodexConversation.scala +++ b/codex/src/main/scala/orca/tools/codex/CodexConversation.scala @@ -1,13 +1,18 @@ package orca.tools.codex -import orca.llm.{BackendTag, Model, SessionId} +import orca.agents.{BackendTag, Model, SessionId} import orca.events.{Usage} import orca.{OrcaFlowException} -import orca.backend.{ConversationEvent, LlmResult} -import orca.backend.{BufferedStderrDiagnostics, StreamConversation, StreamSource} +import orca.backend.{ConversationEvent, AgentResult} +import orca.backend.{ + BufferedStderrDiagnostics, + ForkedConversation, + StreamSource +} import orca.backend.mcp.{AskUserMcpServer, AskUserSession} import orca.subprocess.PipedCliProcess import orca.tools.codex.jsonl.{FileChangeDetail, InboundEvent, Item} +import ox.Ox import com.github.plokhotnyuk.jsoniter_scala.core.writeToString import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec @@ -15,8 +20,8 @@ import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec import java.util.concurrent.atomic.AtomicReference /** Drives a `codex exec --json` session to completion. Boilerplate lives in - * [[StreamConversation]]; this class supplies the codex-specific protocol - * translation: JSONL → [[InboundEvent]] → `ConversationEvent`s. + * [[orca.backend.ForkedConversation]]; this class supplies the codex-specific + * protocol translation: JSONL → [[InboundEvent]] → `ConversationEvent`s. * * Notable parity gaps vs. claude (deliberate, driven by codex's JSONL protocol * — see ADR 0007): @@ -26,7 +31,7 @@ import java.util.concurrent.atomic.AtomicReference * pre-baked into spawn args. `ApproveTool` is never emitted here. * - `codex exec` is one-shot; multi-turn happens via `codex exec resume` on * a fresh spawn, so `sendUserMessage` is a no-op. - * - **`LlmResult.output` is synthesised**: codex has no terminal message + * - **`AgentResult.output` is synthesised**: codex has no terminal message * carrying the structured payload, so [[lastAgentMessage]] snapshots each * agent message and the result builder reads the last one at * `turn.completed`. The prompt template makes the last message JSON. @@ -36,7 +41,8 @@ private[codex] class CodexConversation( initialPrompt: String = "", val outputSchema: Option[String] = None, override val askUser: Option[AskUserSession] = None -) extends StreamConversation[BackendTag.Codex.type]( +)(using Ox) + extends ForkedConversation[BackendTag.Codex.type]( source = StreamSource.fromProcess(process), backendName = "codex", initialPrompt = initialPrompt @@ -44,7 +50,6 @@ private[codex] class CodexConversation( with BufferedStderrDiagnostics[BackendTag.Codex.type]: import CodexConversation.* - import StreamConversation.Outcome private val sessionIdRef = new AtomicReference[String]("") private val modelRef = new AtomicReference[Option[String]](None) @@ -58,26 +63,17 @@ private[codex] class CodexConversation( * has already surfaced the corresponding `UserQuestion` event, so rendering * the tool call on top would be noise. The matching `item.completed` is also * suppressed: the user's typed answer would otherwise re-render as `⎿ - * <answer>` after the prompt surfaced it. - * - * Single-threaded reader (the JSONL reader thread is the sole writer), so a - * plain `var` Set is enough. + * <answer>` after the prompt surfaced it. See + * [[orca.backend.AskUserEchoes]]. */ - private var suppressedMcpItemIds: Set[String] = Set.empty + private val askUserEchoes = new orca.backend.AskUserEchoes - // Subclass fields above are assigned now; safe to spin up the reader + - // stderr workers. See [[StreamConversation.start]] — the base also - // spawns the ask_user drainer if one was wired. - start() + // No `start()`: the base spawns its reader / stderr / ask-user forks lazily + // on first touch of the conversation surface, after this subclass's fields + // are initialised. // --- Conversation surface --- - /** Codex exec consumes its prompt argv-side and ignores stdin thereafter; - * injecting more user turns mid-session isn't supported. The contract still - * requires a callable method — this is a no-op. - */ - def sendUserMessage(text: String): Unit = () - // `canAskUser` is owned by the base — true when this conversation was // constructed with `askUser = Some(...)`. Codex exec has no in-session // stdin channel (ADR 0007), but the agent CAN reach the user via the @@ -98,9 +94,9 @@ private[codex] class CodexConversation( * written correctly to `~/.codex/sessions/`; the message is harmless. * * Filter both, plus empty lines. Anything else passes through with the - * default backend-prefixed Error event AND is recorded in the bounded - * stderr buffer (see [[BufferedStderrDiagnostics]]) so the failure exception - * can include them. + * default backend-prefixed Error event AND is recorded in the bounded stderr + * buffer (see [[BufferedStderrDiagnostics]]) so the failure exception can + * include them. */ override protected def handleStderr(line: String): Unit = val trimmed = line.trim @@ -153,7 +149,7 @@ private[codex] class CodexConversation( // ask_user is surfaced through the host-side bridge as a // UserQuestion event; the matching item.completed echo is dropped // too — the user has already seen their typed answer at the prompt. - suppressedMcpItemIds = suppressedMcpItemIds + id + askUserEchoes.suppress(id) case Item.McpToolCall(_, server, tool, args, _, _) => eventQueue.enqueue( ConversationEvent.AssistantToolCall( @@ -190,11 +186,10 @@ private[codex] class CodexConversation( content = changes.map(c => s"${c.kind} ${c.path}").mkString("\n") ) ) - case Item.McpToolCall(id, _, _, _, _, _) - if suppressedMcpItemIds.contains(id) => - // Matched a suppressed ask_user call started above; drop the - // mirrored completion and clear the id. - suppressedMcpItemIds = suppressedMcpItemIds - id + case Item.McpToolCall(id, _, _, _, _, _) if askUserEchoes.consume(id) => + // Matched a suppressed ask_user call started above; drop the mirrored + // completion. `consume` clears the id as it matches. + () case Item.McpToolCall(_, server, tool, _, result, status) => eventQueue.enqueue( ConversationEvent.ToolResult( @@ -215,13 +210,13 @@ private[codex] class CodexConversation( s"$server.$tool" private def handleTurnCompleted(usage: Usage): Unit = - val result = LlmResult( + val result = AgentResult( sessionId = SessionId[BackendTag.Codex.type](sessionIdRef.get()), output = lastAgentMessage.get(), usage = usage, model = modelRef.get().map(Model.apply) ) - val _ = outcomeRef.compareAndSet(None, Some(Outcome.Success(result))) + succeedWith(result) private def toWire(c: FileChangeDetail): FileChangeWire = FileChangeWire(c.path, c.kind) diff --git a/codex/src/main/scala/orca/tools/codex/DefaultCodexAgent.scala b/codex/src/main/scala/orca/tools/codex/DefaultCodexAgent.scala new file mode 100644 index 00000000..e5fb4bca --- /dev/null +++ b/codex/src/main/scala/orca/tools/codex/DefaultCodexAgent.scala @@ -0,0 +1,50 @@ +package orca.tools.codex + +import orca.agents.{BackendTag, CodexAgent, AgentConfig, Model, Prompts} +import orca.events.{OrcaListener} + +import orca.backend.{Interaction, AgentBackend} +import orca.agents.BaseAgent + +/** Default [[CodexAgent]] implementation. Inherits the autonomous-text + + * `resultAs[O]` plumbing from [[BaseAgent]] and only adds the Codex-specific + * `mini` model accessor. + */ +private[orca] class DefaultCodexAgent( + backend: AgentBackend[BackendTag.Codex.type], + config: AgentConfig, + prompts: Prompts, + workDir: os.Path, + events: OrcaListener, + interaction: Interaction, + val name: String = "main" +) extends BaseAgent[BackendTag.Codex.type, CodexAgent]( + backend, + config, + prompts, + workDir, + events, + interaction + ) + with CodexAgent: + + /** Pin the cheap-and-fast model variant. The literal model id matches what's + * available in the installed `codex-cli` (gpt-5.4-mini in 0.125.0); newer + * codex versions may rename, in which case callers pin the right id with + * `codex.withModel(Model("..."))`. + */ + def mini: CodexAgent = withModel(Model("gpt-5.4-mini")) + + protected def copyTool( + config: AgentConfig = config, + name: String = name + ): CodexAgent = + new DefaultCodexAgent( + backend, + config, + prompts, + workDir, + events, + interaction, + name + ) diff --git a/codex/src/main/scala/orca/tools/codex/DefaultCodexTool.scala b/codex/src/main/scala/orca/tools/codex/DefaultCodexTool.scala deleted file mode 100644 index f814a302..00000000 --- a/codex/src/main/scala/orca/tools/codex/DefaultCodexTool.scala +++ /dev/null @@ -1,50 +0,0 @@ -package orca.tools.codex - -import orca.llm.{BackendTag, CodexTool, LlmConfig, Model, Prompts} -import orca.events.{OrcaListener} - -import orca.backend.{Interaction, LlmBackend} -import orca.llm.BaseLlmTool - -/** Default [[CodexTool]] implementation. Inherits the autonomous-text + - * `resultAs[O]` plumbing from [[BaseLlmTool]] and only adds the Codex-specific - * `mini` model accessor. - */ -private[orca] class DefaultCodexTool( - backend: LlmBackend[BackendTag.Codex.type], - config: LlmConfig, - prompts: Prompts, - workDir: os.Path, - events: OrcaListener, - interaction: Interaction, - val name: String = "main" -) extends BaseLlmTool[BackendTag.Codex.type, CodexTool]( - backend, - config, - prompts, - workDir, - events, - interaction - ) - with CodexTool: - - /** Pin the cheap-and-fast model variant. The literal model id matches what's - * available in the installed `codex-cli` (gpt-5.4-mini in 0.125.0); newer - * codex versions may rename, in which case callers override via - * `withConfig(LlmConfig(model = Some(Model("..."))))`. - */ - def mini: CodexTool = withModel(Model("gpt-5.4-mini")) - - protected def copyTool( - config: LlmConfig = config, - name: String = name - ): CodexTool = - new DefaultCodexTool( - backend, - config, - prompts, - workDir, - events, - interaction, - name - ) diff --git a/codex/src/test/scala/orca/tools/codex/CodexArgsTest.scala b/codex/src/test/scala/orca/tools/codex/CodexArgsTest.scala index d7968fee..621fe2d4 100644 --- a/codex/src/test/scala/orca/tools/codex/CodexArgsTest.scala +++ b/codex/src/test/scala/orca/tools/codex/CodexArgsTest.scala @@ -1,22 +1,29 @@ package orca.tools.codex -import orca.llm.{AutoApprove, BackendTag, LlmConfig, Model, SessionId, ToolSet} +import orca.agents.{ + AutoApprove, + BackendTag, + AgentConfig, + Model, + SessionId, + ToolSet +} class CodexArgsTest extends munit.FunSuite: test("exec emits codex exec --json with the prompt as the trailing arg"): val args = CodexArgs.exec( prompt = "summarize", - config = LlmConfig.default, + config = AgentConfig.default, outputSchemaFile = None, workDir = os.pwd ) assertEquals(args.take(3), Seq("codex", "exec", "--json")) assertEquals(args.last, "summarize") - test("exec passes --model when LlmConfig.model is set"): + test("exec passes --model when AgentConfig.model is set"): val args = CodexArgs.exec( prompt = "x", - config = LlmConfig.default.copy(model = Some(Model("gpt-5.4-mini"))), + config = AgentConfig.default.copy(model = Some(Model("gpt-5.4-mini"))), outputSchemaFile = None, workDir = os.pwd ) @@ -26,19 +33,20 @@ class CodexArgsTest extends munit.FunSuite: val workDir = os.temp.dir() val args = CodexArgs.exec( prompt = "x", - config = LlmConfig.default, + config = AgentConfig.default, outputSchemaFile = None, workDir = workDir ) assert(args.containsSlice(Seq("-C", workDir.toString))) test("exec includes --skip-git-repo-check"): - val args = CodexArgs.exec("x", LlmConfig.default, None, os.pwd) + val args = CodexArgs.exec("x", AgentConfig.default, None, os.pwd) assert(args.contains("--skip-git-repo-check")) test("exec passes --output-schema <file> when supplied"): val schemaFile = os.temp() / "schema.json" - val args = CodexArgs.exec("x", LlmConfig.default, Some(schemaFile), os.pwd) + val args = + CodexArgs.exec("x", AgentConfig.default, Some(schemaFile), os.pwd) assert(args.containsSlice(Seq("--output-schema", schemaFile.toString))) test( @@ -46,7 +54,7 @@ class CodexArgsTest extends munit.FunSuite: ): val args = CodexArgs.exec( "x", - LlmConfig.default.copy(autoApprove = AutoApprove.All), + AgentConfig.default.copy(autoApprove = AutoApprove.All), None, os.pwd ) @@ -56,7 +64,7 @@ class CodexArgsTest extends munit.FunSuite: test("AutoApprove.Only maps to --full-auto"): val args = CodexArgs.exec( "x", - LlmConfig.default.copy(autoApprove = AutoApprove.Only(Set("Bash"))), + AgentConfig.default.copy(autoApprove = AutoApprove.Only(Set("Bash"))), None, os.pwd ) @@ -67,12 +75,12 @@ class CodexArgsTest extends munit.FunSuite: "ToolSet.ReadOnly maps to --sandbox read-only and overrides autoApprove" ): // Pins the gate used by `.withReadOnly` callers — reviewers, - // ReviewerSelector.llmDriven, lint, Plan.autonomous.from. Without + // ReviewerSelector.agentDriven, lint, Plan.autonomous.from. Without // this mapping codex's reviewers inherit the base tool's permissions // and could edit files during a review turn. val args = CodexArgs.exec( "x", - LlmConfig.default.copy( + AgentConfig.default.copy( tools = ToolSet.ReadOnly, autoApprove = AutoApprove.All ), @@ -89,7 +97,7 @@ class CodexArgsTest extends munit.FunSuite: // which must precede the `exec` subcommand. val args = CodexArgs.exec( "x", - LlmConfig.default.copy(tools = ToolSet.NetworkOnly), + AgentConfig.default.copy(tools = ToolSet.NetworkOnly), None, os.pwd ) @@ -109,7 +117,7 @@ class CodexArgsTest extends munit.FunSuite: // internal MCP timeout and a duplicate follow-up question. val args = CodexArgs.exec( "x", - LlmConfig.default, + AgentConfig.default, None, os.pwd, mcpServerUrl = Some("http://127.0.0.1:9876/mcp") @@ -132,7 +140,7 @@ class CodexArgsTest extends munit.FunSuite: ) test("exec omits -c mcp_servers when no MCP url is supplied"): - val args = CodexArgs.exec("x", LlmConfig.default, None, os.pwd) + val args = CodexArgs.exec("x", AgentConfig.default, None, os.pwd) assert( !args.exists(_.startsWith("mcp_servers.")), s"args should not mention mcp_servers; got: $args" @@ -143,7 +151,7 @@ class CodexArgsTest extends munit.FunSuite: val args = CodexArgs.execResume( sid, "next step", - LlmConfig.default + AgentConfig.default ) assertEquals(args.take(4), Seq("codex", "exec", "resume", "--json")) assert(args.contains("019dc-thread")) @@ -151,15 +159,55 @@ class CodexArgsTest extends munit.FunSuite: test("execResume omits -C and --output-schema (codex doesn't accept them)"): val sid = SessionId[BackendTag.Codex.type]("sid") - val args = CodexArgs.execResume(sid, "x", LlmConfig.default) + val args = CodexArgs.execResume(sid, "x", AgentConfig.default) assert(!args.contains("-C")) assert(!args.contains("--output-schema")) - test("execResume propagates --model when LlmConfig.model is set"): + test( + "execResume omits --sandbox/--full-auto (exec resume rejects them; inherited)" + ): + // Regression: `codex exec resume` errors with "unexpected argument + // '--sandbox'"; the resumed session inherits its sandbox from creation. + val sid = SessionId[BackendTag.Codex.type]("sid") + val readOnly = + CodexArgs.execResume( + sid, + "x", + AgentConfig.default.copy(tools = ToolSet.ReadOnly) + ) + assert(!readOnly.contains("--sandbox"), readOnly) + val networkOnly = + CodexArgs.execResume( + sid, + "x", + AgentConfig.default.copy(tools = ToolSet.NetworkOnly) + ) + assert(!networkOnly.contains("--full-auto"), networkOnly) + val fullOnly = CodexArgs.execResume( + sid, + "x", + AgentConfig.default.copy(autoApprove = AutoApprove.Only(Set("Bash"))) + ) + assert(!fullOnly.contains("--full-auto"), fullOnly) + + test( + "execResume keeps --dangerously-bypass-approvals-and-sandbox (Full + All)" + ): + // The one sandbox flag `exec resume` accepts; re-asserted each turn to keep + // approvals off for an auto-approve-all coder session. + val sid = SessionId[BackendTag.Codex.type]("sid") + val args = CodexArgs.execResume( + sid, + "x", + AgentConfig.default.copy(autoApprove = AutoApprove.All) + ) + assert(args.contains("--dangerously-bypass-approvals-and-sandbox"), args) + + test("execResume propagates --model when AgentConfig.model is set"): val sid = SessionId[BackendTag.Codex.type]("sid") val args = CodexArgs.execResume( sid, "x", - LlmConfig.default.copy(model = Some(Model("gpt-5.4-mini"))) + AgentConfig.default.copy(model = Some(Model("gpt-5.4-mini"))) ) assert(args.containsSlice(Seq("--model", "gpt-5.4-mini"))) diff --git a/codex/src/test/scala/orca/tools/codex/CodexBackendTest.scala b/codex/src/test/scala/orca/tools/codex/CodexBackendTest.scala index 87fcf823..1d44e4bd 100644 --- a/codex/src/test/scala/orca/tools/codex/CodexBackendTest.scala +++ b/codex/src/test/scala/orca/tools/codex/CodexBackendTest.scala @@ -1,7 +1,7 @@ package orca.tools.codex import orca.backend.SupervisedBackend -import orca.llm.{BackendTag, LlmConfig, Model, SessionId} +import orca.agents.{BackendTag, AgentConfig, Model, SessionId} import orca.{OrcaFlowException} import orca.subprocess.{FakePipedCliProcess, SpawnStubCliRunner} @@ -11,7 +11,7 @@ class CodexBackendTest extends munit.FunSuite: SessionId[BackendTag.Codex.type]("00000000-0000-0000-0000-000000000000") private def withBackend[T](runner: SpawnStubCliRunner)( - body: CodexBackend => T + body: ox.Ox ?=> CodexBackend => T ): T = SupervisedBackend.using(new CodexBackend(runner))(body) private def successfulProcess( @@ -40,7 +40,12 @@ class CodexBackendTest extends munit.FunSuite: ) withBackend(runner): backend => val result = - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) // The returned session id is the client-allocated one — the server's // thr-42 is mapped internally so subsequent calls can resume it without // the caller having to thread a new id back in. @@ -66,7 +71,12 @@ class CodexBackendTest extends munit.FunSuite: p.sendSigInt() withBackend(new SpawnStubCliRunner(List(p))): backend => val result = - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) assertEquals(result.model, Some(Model("gpt-5"))) test("runAutonomous throws when codex exits without turn.completed"): @@ -77,7 +87,12 @@ class CodexBackendTest extends munit.FunSuite: p.sendSigInt() withBackend(new SpawnStubCliRunner(List(p))): backend => intercept[OrcaFlowException]: - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) test("runAutonomous throws with the exit code when codex exits non-zero"): val p = new FakePipedCliProcess(initiallyAlive = false): @@ -86,7 +101,12 @@ class CodexBackendTest extends munit.FunSuite: p.closeStderr() withBackend(new SpawnStubCliRunner(List(p))): backend => val ex = intercept[OrcaFlowException]: - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) assert( ex.getMessage.contains("exited with code 7"), s"expected the exit code in the failure message; got: ${ex.getMessage}" @@ -100,7 +120,12 @@ class CodexBackendTest extends munit.FunSuite: p.closeStderr() withBackend(new SpawnStubCliRunner(List(p))): backend => val ex = intercept[OrcaFlowException]: - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) assert( ex.getMessage.contains("thread/resume failed: not found"), s"expected stderr in the exception; got: ${ex.getMessage}" @@ -114,7 +139,12 @@ class CodexBackendTest extends munit.FunSuite: p.closeStderr() withBackend(new SpawnStubCliRunner(List(p))): backend => val ex = intercept[OrcaFlowException]: - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) assert( !ex.getMessage.contains("Reading additional input from stdin"), s"filtered noise leaked into the exception: ${ex.getMessage}" @@ -126,7 +156,7 @@ class CodexBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "list files", clientSid, - LlmConfig.default.copy(systemPrompt = Some("be terse")), + AgentConfig.default.copy(systemPrompt = Some("be terse")), os.temp.dir() ) val args = runner.calls.head @@ -146,9 +176,9 @@ class CodexBackendTest extends munit.FunSuite: withBackend(runner): backend => val workDir = os.temp.dir() val _ = - backend.runAutonomous("first", clientSid, LlmConfig.default, workDir) + backend.runAutonomous("first", clientSid, AgentConfig.default, workDir) val _ = - backend.runAutonomous("again", clientSid, LlmConfig.default, workDir) + backend.runAutonomous("again", clientSid, AgentConfig.default, workDir) val firstArgs = runner.calls(0) val secondArgs = runner.calls(1) assert(!firstArgs.contains("resume"), firstArgs) @@ -173,15 +203,15 @@ class CodexBackendTest extends munit.FunSuite: ) withBackend(runner): backend => val workDir = os.temp.dir() - // Simulate the post-interactive-drain registration that DefaultLlmCall + // Simulate the post-interactive-drain registration that DefaultAgentCall // performs (this test exercises the backend in isolation; the - // integration path is wired in LlmCall.runInteractiveOnce). + // integration path is wired in AgentCall.runInteractiveOnce). backend.registerSession( clientSid, SessionId[BackendTag.Codex.type]("thr-via-interactive") ) val _ = - backend.runAutonomous("after", clientSid, LlmConfig.default, workDir) + backend.runAutonomous("after", clientSid, AgentConfig.default, workDir) val args = runner.calls.head assert(args.contains("resume"), args) assert(args.contains("thr-via-interactive"), args) @@ -201,8 +231,8 @@ class CodexBackendTest extends munit.FunSuite: SessionId[BackendTag.Codex.type]("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa") val sidB = SessionId[BackendTag.Codex.type]("bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb") - val _ = backend.runAutonomous("for A", sidA, LlmConfig.default, workDir) - val _ = backend.runAutonomous("for B", sidB, LlmConfig.default, workDir) + val _ = backend.runAutonomous("for A", sidA, AgentConfig.default, workDir) + val _ = backend.runAutonomous("for B", sidB, AgentConfig.default, workDir) val secondArgs = runner.calls(1) assert( !secondArgs.contains("resume"), @@ -223,7 +253,7 @@ class CodexBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "q", clientSid, - LlmConfig.default, + AgentConfig.default, workDir, outputSchema = Some("""{"type":"object"}""") ) @@ -239,7 +269,12 @@ class CodexBackendTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(successfulProcess())) withBackend(runner): backend => val _ = - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) val args = runner.calls.head assert( !args.exists(_.startsWith("mcp_servers.")), @@ -254,7 +289,7 @@ class CodexBackendTest extends munit.FunSuite: "q", clientSid, displayPrompt = "q", - LlmConfig.default, + AgentConfig.default, workDir, Some("""{"type":"object"}""") ) @@ -277,7 +312,7 @@ class CodexBackendTest extends munit.FunSuite: "q", clientSid, displayPrompt = "q", - LlmConfig.default, + AgentConfig.default, os.temp.dir(), outputSchema = None ) @@ -308,7 +343,7 @@ class CodexBackendTest extends munit.FunSuite: "list files", clientSid, displayPrompt = "list files", - LlmConfig.default.copy(systemPrompt = Some("be terse")), + AgentConfig.default.copy(systemPrompt = Some("be terse")), os.temp.dir(), outputSchema = None ) @@ -322,3 +357,38 @@ class CodexBackendTest extends munit.FunSuite: terseIdx < askIdx && askIdx - terseIdx > "be terse".length + 2, s"systemPrompt and ask_user hint should be separated; got: $finalPrompt" ) + + test("sessionExists returns true when a matching rollout file exists"): + val sid = SessionId[BackendTag.Codex.type]("test-session-id-123") + val tmpSessions = os.temp.dir() + os.write(tmpSessions / "rollout-2024-01-01-test-session-id-123.jsonl", "") + SupervisedBackend.using( + new CodexBackend(new SpawnStubCliRunner(Nil), tmpSessions) + ): backend => + assert(backend.sessionExists(sid)) + + test("sessionExists returns false when no matching file exists"): + val tmpSessions = os.temp.dir() + SupervisedBackend.using( + new CodexBackend(new SpawnStubCliRunner(Nil), tmpSessions) + ): backend => + assert(!backend.sessionExists(clientSid)) + + test("sessionExists returns false when the sessions dir is absent"): + val missing = os.temp.dir() / "no-such-sessions" + SupervisedBackend.using( + new CodexBackend(new SpawnStubCliRunner(Nil), missing) + ): backend => + assert(!backend.sessionExists(clientSid)) + + test( + "sessionExists returns false for id `.*` even when rollout files exist (blocks regex injection)" + ): + val tmpSessions = os.temp.dir() + // Create a rollout file that the old regex `rollout-.*-.*\.jsonl` would match + os.write(tmpSessions / "rollout-2024-01-01-some-real-id.jsonl", "") + val maliciousId = SessionId[BackendTag.Codex.type](".*") + SupervisedBackend.using( + new CodexBackend(new SpawnStubCliRunner(Nil), tmpSessions) + ): backend => + assert(!backend.sessionExists(maliciousId)) diff --git a/codex/src/test/scala/orca/tools/codex/CodexConversationTest.scala b/codex/src/test/scala/orca/tools/codex/CodexConversationTest.scala index 3d83c586..1bba7569 100644 --- a/codex/src/test/scala/orca/tools/codex/CodexConversationTest.scala +++ b/codex/src/test/scala/orca/tools/codex/CodexConversationTest.scala @@ -1,14 +1,23 @@ package orca.tools.codex -import orca.llm.{SessionId} +import orca.agents.{SessionId} import orca.events.{Usage} import orca.{OrcaFlowException, OrcaInteractiveCancelled} import orca.backend.ConversationEvent import orca.subprocess.FakePipedCliProcess +import ox.{Ox, supervised} class CodexConversationTest extends munit.FunSuite: - test("agent_message item completes a turn with TextDelta + TurnEnd"): + /** `CodexConversation` forks its reader/stderr/ask-user workers into the + * caller's per-turn Ox, so construction needs a `using Ox`. Run each test + * body in a fresh supervised scope that provides it. Tests managing their + * own scope (the ask-user ones) stay on plain `test`. + */ + private def convTest(name: String)(body: Ox ?=> Unit): Unit = + test(name)(supervised(body)) + + convTest("agent_message item completes a turn with TextDelta + TurnEnd"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) @@ -38,7 +47,7 @@ class CodexConversationTest extends munit.FunSuite: assertEquals(result.output, "hello") assertEquals(result.usage, Usage(10L, 3L, None)) - test("initialPrompt becomes a UserMessage event before agent output"): + convTest("initialPrompt becomes a UserMessage event before agent output"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process, initialPrompt = "do the thing") @@ -59,7 +68,7 @@ class CodexConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("the LAST agent_message wins when a turn produces several"): + convTest("the LAST agent_message wins when a turn produces several"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) @@ -80,7 +89,7 @@ class CodexConversationTest extends munit.FunSuite: val Right(result) = conv.awaitResult(): @unchecked assertEquals(result.output, "final answer") - test( + convTest( "command_execution items become AssistantToolCall + ToolResult events" ): val process = new FakePipedCliProcess() @@ -114,7 +123,7 @@ class CodexConversationTest extends munit.FunSuite: case other => fail(s"expected ToolResult, got $other") val _ = conv.awaitResult() - test("command_execution with non-zero exit yields ok=false"): + convTest("command_execution with non-zero exit yields ok=false"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) @@ -142,7 +151,7 @@ class CodexConversationTest extends munit.FunSuite: assertEquals(toolResult.ok, false) val _ = conv.awaitResult() - test("file_change items become file_change tool calls and results"): + convTest("file_change items become file_change tool calls and results"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) @@ -177,7 +186,7 @@ class CodexConversationTest extends munit.FunSuite: assertEquals(toolResult.ok, true) val _ = conv.awaitResult() - test("reasoning items emit AssistantThinkingDelta when non-empty"): + convTest("reasoning items emit AssistantThinkingDelta when non-empty"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) @@ -201,7 +210,9 @@ class CodexConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("cancel surfaces as Left(OrcaInteractiveCancelled) from awaitResult"): + convTest( + "cancel surfaces as Left(OrcaInteractiveCancelled) from awaitResult" + ): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) conv.cancel() @@ -211,7 +222,7 @@ class CodexConversationTest extends munit.FunSuite: fail(s"expected Left(OrcaInteractiveCancelled), got: $other") assertEquals(process.sigIntCount, 1) - test( + convTest( "clean process exit without turn.completed surfaces as OrcaFlowException" ): val process = new FakePipedCliProcess(initiallyAlive = false) @@ -228,7 +239,7 @@ class CodexConversationTest extends munit.FunSuite: s"expected the missing-turn.completed message; got: ${ex.getMessage}" ) - test( + convTest( "malformed JSONL line surfaces as ConversationEvent.Error and the loop continues" ): val process = new FakePipedCliProcess() @@ -255,7 +266,7 @@ class CodexConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("stderr noise about reading stdin is filtered out"): + convTest("stderr noise about reading stdin is filtered out"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) @@ -280,7 +291,7 @@ class CodexConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test( + convTest( "`failed to record rollout items` shutdown noise is filtered out" ): // Codex emits this ERROR line on stderr during its shutdown sequence @@ -314,7 +325,7 @@ class CodexConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("real stderr lines surface as ConversationEvent.Error"): + convTest("real stderr lines surface as ConversationEvent.Error"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) @@ -337,28 +348,7 @@ class CodexConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("sendUserMessage is a documented no-op (no stdin write)"): - val process = new FakePipedCliProcess() - val conv = new CodexConversation(process) - conv.sendUserMessage("ignored") - assertEquals(process.writes, Nil) - - process.enqueueStdout( - """{"type":"thread.started","thread_id":"thr-noop"}""" - ) - process.enqueueStdout( - """{"type":"item.completed","item":{"id":"item_0","type":"agent_message","text":"ok"}}""" - ) - process.enqueueStdout( - """{"type":"turn.completed","usage":{"input_tokens":0,"output_tokens":0,"cached_input_tokens":0,"reasoning_output_tokens":0}}""" - ) - process.closeStdout() - process.closeStderr() - val _ = conv.events.toList - val _ = conv.awaitResult() - assertEquals(process.writes, Nil) - - test("mcp_tool_call emits AssistantToolCall + ToolResult"): + convTest("mcp_tool_call emits AssistantToolCall + ToolResult"): // A non-ask_user MCP tool: started and completed items round-trip // into matching AssistantToolCall + ToolResult events using the // dotted `server.tool` naming. @@ -397,7 +387,7 @@ class CodexConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test( + convTest( "mcp_tool_call ToolResult drops non-text content fragments" ): // The MCP content array can carry text + image + resource fragments; @@ -427,7 +417,7 @@ class CodexConversationTest extends munit.FunSuite: assertEquals(result, Some("text1text2")) val _ = conv.awaitResult() - test( + convTest( "mcp_tool_call ToolResult surfaces raw JSON when result fails to parse" ): // MCP servers in the wild emit non-standard result shapes too; the @@ -543,14 +533,15 @@ class CodexConversationTest extends munit.FunSuite: respond("magenta") assertEquals(askResult.join(), "magenta") - test("canAskUser is false when no bridge is provided"): + convTest("canAskUser is false when no bridge is provided"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) assertEquals(conv.canAskUser, false) process.closeStdout() + process.closeStderr() val _ = conv.events.toList - test("unknown top-level events are ignored without surfacing"): + convTest("unknown top-level events are ignored without surfacing"): val process = new FakePipedCliProcess() val conv = new CodexConversation(process) diff --git a/codex/src/test/scala/orca/tools/codex/CodexIntegrationTest.scala b/codex/src/test/scala/orca/tools/codex/CodexIntegrationTest.scala index f242711d..235a6b01 100644 --- a/codex/src/test/scala/orca/tools/codex/CodexIntegrationTest.scala +++ b/codex/src/test/scala/orca/tools/codex/CodexIntegrationTest.scala @@ -1,6 +1,6 @@ package orca.tools.codex -import orca.llm.{AutoApprove, BackendTag, LlmConfig, SessionId} +import orca.agents.{AutoApprove, BackendTag, AgentConfig, SessionId} import orca.backend.{ConversationEvent, SupervisedBackend} import orca.subprocess.OsProcCliRunner @@ -24,11 +24,11 @@ class CodexIntegrationTest extends munit.FunSuite: import scala.concurrent.duration.DurationInt 3.minutes - private def withBackend(body: CodexBackend => Unit): Unit = + private def withBackend(body: ox.Ox ?=> CodexBackend => Unit): Unit = SupervisedBackend.using(new CodexBackend(OsProcCliRunner))(body) - private val unsandboxed: LlmConfig = - LlmConfig.default.copy(autoApprove = AutoApprove.All) + private val unsandboxed: AgentConfig = + AgentConfig.default.copy(autoApprove = AutoApprove.All) private def fresh = SessionId.fresh[BackendTag.Codex.type] diff --git a/design.md b/design.md index cabd6aa3..197c4a58 100644 --- a/design.md +++ b/design.md @@ -91,8 +91,8 @@ implementers and advanced callers opt into explicitly. |---|---| | `orca` | Flow DSL — `flow`, `stage`, `fail`, accessors (`claude`, `codex`, `git`, `gh`, `fs`, `userPrompt`), `OrcaArgs`, `FlowContext`, `OrcaFlowException`, `OrcaInteractiveCancelled`. Plus a thin `exports.scala` re-exporting the user surface from sub-packages. | | `orca.events` | Flow-wide event log: `OrcaEvent`, `OrcaListener`, `EventDispatcher`, `CostTracker`, `Usage`. | -| `orca.llm` | LLM call API + default implementations: `LlmTool` / `ClaudeTool` / `CodexTool` / `LlmCall`, `LlmConfig`, `AutoApprove`, `UnapprovedPolicy`, `BackendTag`, `SessionId`, `JsonData`, `Announce`, `AgentInput`, `Prompts`, plus the `AbstractDefaultLlmTool`, `DefaultLlmCall`, `DefaultPrompts`, `ResponseParser` defaults backend tools extend. | -| `orca.backend` | LLM SPI (implemented per backend, never named in scripts): `LlmBackend`, `LlmResult`, `Conversation`, `ConversationEvent` (+ `ApprovalDecision`), `Interaction`, `StreamConversation`. | +| `orca.agents` | LLM call API + default implementations: `Agent` / `ClaudeAgent` / `CodexAgent` / `AgentCall`, `AgentConfig`, `AutoApprove`, `UnapprovedPolicy`, `BackendTag`, `SessionId`, `JsonData`, `Announce`, `AgentInput`, `Prompts`, plus the `AbstractDefaultAgent`, `DefaultAgentCall`, `DefaultPrompts`, `ResponseParser` defaults backend tools extend. | +| `orca.backend` | LLM SPI (implemented per backend, never named in scripts): `AgentBackend`, `AgentResult`, `Conversation`, `ConversationEvent` (+ `ApprovalDecision`), `Interaction`, `StreamConversation`. | | `orca.tools` | Tool traits + default `Os*` implementations in single files: `GitTool.scala` (incl. `OsGitTool` and the error/data types), `GitHubTool.scala`, `FsTool.scala`. User-facing GitHub data types (`Issue`, `IssueHandle`, `PrHandle`, `Comment`, `BuildStatus`, `BuildOutcome`) are re-exported from `orca`. | | `orca.plan`, `orca.review` | Higher-level workflow APIs and their domain types. The user-callable entry points are re-exported from `orca`. Bug-triage types live in `orca.plan` alongside `Plan`/`Task`. | | `orca.subprocess` | CLI-runner internals: `CliRunner`, `PipedCliProcess`, `OsProcCliRunner`, `QuietProc`. | @@ -101,9 +101,9 @@ implementers and advanced callers opt into explicitly. `orca/exports.scala` re-exports the user-facing subset of every sub-package into the root `orca` namespace, so the standard `import orca.{*, given}` -continues to pick up `LlmTool`, `Plan`, `IssueHandle`, etc. without +continues to pick up `Agent`, `Plan`, `IssueHandle`, etc. without sub-package imports. Sub-packages exist for navigability and to keep the -root namespace focused. Backend authors writing a new `LlmBackend` import +root namespace focused. Backend authors writing a new `AgentBackend` import `orca.backend.*`; channel authors writing a new `Interaction` import `orca.backend.{Conversation, ...}`; flow scripts import nothing more than `orca.{*, given}`. @@ -111,8 +111,8 @@ root namespace focused. Backend authors writing a new `LlmBackend` import ### Trait + canonical default: one file A trait and its single canonical implementation share one file, named after -the trait. Examples in this codebase: `LlmCall.scala` contains -`trait LlmCall` + `class DefaultLlmCall`; `FsTool.scala` contains +the trait. Examples in this codebase: `AgentCall.scala` contains +`trait AgentCall` + `class DefaultAgentCall`; `FsTool.scala` contains `trait FsTool` + `class OsFsTool`; `GitTool.scala` / `GitHubTool.scala` / `Prompts.scala` follow the same shape. @@ -128,7 +128,7 @@ Split into separate files when a second peer implementation appears (`ClaudeBackend` + `CodexBackend`), when the implementation lives in a different module from the trait (`Interaction` in `tools` vs. `TerminalInteraction` in `runner`), or when the "implementation" is an -abstract base others extend (`StreamConversation`, `AbstractDefaultLlmTool`). +abstract base others extend (`StreamConversation`, `AbstractDefaultAgent`). ### Tool traits @@ -173,7 +173,7 @@ claude.sonnet.resultAs[TaskPlan].prompt(input) // Claude Code, Sonnet claude.haiku.ask("quick question") // Haiku (untyped convenience) codex.resultAs[TaskPlan].prompt(input) // Codex, default model -claude.withConfig(LlmConfig( +claude.withConfig(AgentConfig( systemPrompt = Some("You are a performance reviewer...") )).resultAs[ReviewResult].prompt(diff) ``` @@ -190,33 +190,33 @@ enum BackendTag: The full traits: ```scala -trait LlmTool[B <: BackendTag]: - def resultAs[O: Schema: ConfiguredJsonValueCodec]: LlmCall[B, O] - def ask(prompt: String, config: LlmConfig = LlmConfig.default): String - def withConfig(config: LlmConfig): LlmTool[B] - def withSystemPrompt(prompt: String): LlmTool[B] +trait Agent[B <: BackendTag]: + def resultAs[O: Schema: ConfiguredJsonValueCodec]: AgentCall[B, O] + def ask(prompt: String, config: AgentConfig = AgentConfig.default): String + def withConfig(config: AgentConfig): Agent[B] + def withSystemPrompt(prompt: String): Agent[B] /** Model variants are backend-specific. */ -trait ClaudeTool extends LlmTool[BackendTag.ClaudeCode]: - def haiku: ClaudeTool - def sonnet: ClaudeTool - def opus: ClaudeTool - -trait CodexTool extends LlmTool[BackendTag.Codex]: - def mini: CodexTool - -trait LlmCall[B <: BackendTag, O]: - def prompt[I: AgentInput](input: I, config: LlmConfig = LlmConfig.default): O - def startSession[I: AgentInput](input: I, config: LlmConfig = LlmConfig.default): (SessionId[B], O) - def continueSession[I: AgentInput](sessionId: SessionId[B], input: I, config: LlmConfig = LlmConfig.default): O - def interactive[I: AgentInput](input: I, config: LlmConfig = LlmConfig.default): (SessionId[B], O) - def continueInteractive[I: AgentInput](sessionId: SessionId[B], input: I, config: LlmConfig = LlmConfig.default): O +trait ClaudeAgent extends Agent[BackendTag.ClaudeCode]: + def haiku: ClaudeAgent + def sonnet: ClaudeAgent + def opus: ClaudeAgent + +trait CodexAgent extends Agent[BackendTag.Codex]: + def mini: CodexAgent + +trait AgentCall[B <: BackendTag, O]: + def prompt[I: AgentInput](input: I, config: AgentConfig = AgentConfig.default): O + def startSession[I: AgentInput](input: I, config: AgentConfig = AgentConfig.default): (SessionId[B], O) + def continueSession[I: AgentInput](sessionId: SessionId[B], input: I, config: AgentConfig = AgentConfig.default): O + def interactive[I: AgentInput](input: I, config: AgentConfig = AgentConfig.default): (SessionId[B], O) + def continueInteractive[I: AgentInput](sessionId: SessionId[B], input: I, config: AgentConfig = AgentConfig.default): O ``` #### LLM configuration ```scala -case class LlmConfig( +case class AgentConfig( model: Option[String] = None, systemPrompt: Option[String] = None, autoApprove: AutoApprove = AutoApprove.All, @@ -233,7 +233,7 @@ enum UnapprovedPolicy: case AskUser // prompt the user (interactive stages only) ``` -Retries use Ox's `Schedule` directly. Failures that exhaust retries throw `LlmCallFailedException`. +Retries use Ox's `Schedule` directly. Failures that exhaust retries throw `AgentCallFailedException`. #### Structured types @@ -267,14 +267,14 @@ case class IgnoredIssues(issues: List[IgnoredIssue]) derives Schema, ConfiguredJ case class ReviewContext(summary: String, filesChanged: List[String]) derives Schema, ConfiguredJsonValueCodec case class SelectedReviewers(names: List[String]) derives Schema, ConfiguredJsonValueCodec: - def pick(all: List[LlmTool[?]]): List[LlmTool[?]] = all.filter(r => names.contains(r.name)) + def pick(all: List[Agent[?]]): List[Agent[?]] = all.filter(r => names.contains(r.name)) case class Usage(inputTokens: Long, outputTokens: Long, cost: Option[BigDecimal]) /** Build pre-configured reviewer agents on top of a base LLM tool. */ -def allReviewers[B <: BackendTag](base: LlmTool[B]): List[LlmTool[B]] +def allReviewers[B <: BackendTag](base: Agent[B]): List[Agent[B]] // performance, readability, test, code-functionality, abstraction, backend-architect, scala-fp -def minimalReviewers[B <: BackendTag](base: LlmTool[B]): List[LlmTool[B]] +def minimalReviewers[B <: BackendTag](base: Agent[B]): List[Agent[B]] // code-functionality, readability, test ``` @@ -291,9 +291,9 @@ def fixLoop( /** Review + fix loop with parallel reviewers and optional linter. Built on fixLoop. */ def reviewAndFix[B <: BackendTag]( - coder: LlmTool[B], + coder: Agent[B], sessionId: SessionId[B], - reviewers: List[LlmTool[?]], + reviewers: List[Agent[?]], task: String, lintCommand: Option[String] = None, confidenceThreshold: Double = 0.7 @@ -302,7 +302,7 @@ def reviewAndFix[B <: BackendTag]( /** Built-in lint: run a command, summarize errors via LLM, return as ReviewResult. */ def lint( command: String, - llm: LlmTool[?] = claude.haiku + agent: Agent[?] = claude.haiku )(using FlowContext): ReviewResult ``` @@ -312,8 +312,8 @@ Note: `reviewAndFix` uses a shared type parameter `B` for `coder` and `sessionId ```scala trait FlowContext: - val claude: ClaudeTool - val codex: CodexTool + val claude: ClaudeAgent + val codex: CodexAgent val git: GitTool val gh: GitHubTool val fs: FsTool @@ -327,8 +327,8 @@ The prompt sent to the backend is assembled from a pluggable `Prompts`: ```scala trait Prompts: - def autonomous(input: String, outputSchema: String, config: LlmConfig): String - def interactive(input: String, outputSchema: String, config: LlmConfig): String + def autonomous(input: String, outputSchema: String, config: AgentConfig): String + def interactive(input: String, outputSchema: String, config: AgentConfig): String ``` Custom prompt builders can be provided via `flow(..., prompts = ...)`. For **headless** calls, the backend returns JSON on stdout; a parse failure triggers a corrective-retry prompt (counts against the retry budget). For **interactive** calls, the backend runs a stream-json subprocess (ADR 0006) and emits typed `ConversationEvent`s; the final `result` message carries the validated `structured_output` when `--json-schema` is supplied. Interactive parse failures surface to the caller without retry — the user is steering the session. @@ -357,7 +357,7 @@ Events are dispatched synchronously. The library emits events automatically for ```scala trait Interaction: def listeners: List[OrcaListener] - def drive[B <: BackendTag](conversation: Conversation[B]): LlmResult[B] + def drive[B <: BackendTag](conversation: Conversation[B]): AgentResult[B] ``` Built-in: `TerminalInteraction` (default, JLine 3 + fansi), `SlackInteraction(channel)`. Additional listeners for telemetry: @@ -377,17 +377,17 @@ def fail(message: String)(using FlowContext): Nothing // emits Error, throws Or ### Backend abstraction ```scala -trait LlmBackend[B <: BackendTag]: - def runHeadless(prompt: String, config: LlmConfig, workDir: Path): LlmResult[B] - def continueHeadless(sessionId: SessionId[B], prompt: String, config: LlmConfig, workDir: Path): LlmResult[B] - def runInteractive(prompt: String, config: LlmConfig, workDir: Path, outputSchema: Option[String]): Conversation[B] - def continueInteractive(sessionId: SessionId[B], prompt: String, config: LlmConfig, workDir: Path, outputSchema: Option[String]): Conversation[B] +trait AgentBackend[B <: BackendTag]: + def runHeadless(prompt: String, config: AgentConfig, workDir: Path): AgentResult[B] + def continueHeadless(sessionId: SessionId[B], prompt: String, config: AgentConfig, workDir: Path): AgentResult[B] + def runInteractive(prompt: String, config: AgentConfig, workDir: Path, outputSchema: Option[String]): Conversation[B] + def continueInteractive(sessionId: SessionId[B], prompt: String, config: AgentConfig, workDir: Path, outputSchema: Option[String]): Conversation[B] -case class LlmResult[B <: BackendTag](sessionId: SessionId[B], output: String, usage: Usage) +case class AgentResult[B <: BackendTag](sessionId: SessionId[B], output: String, usage: Usage) trait Conversation[B <: BackendTag]: def events: Iterator[ConversationEvent] - def awaitResult(): LlmResult[B] + def awaitResult(): AgentResult[B] def sendUserMessage(text: String): Unit def cancel(): Unit ``` @@ -479,7 +479,7 @@ flow: val result = stage(s"Coding: ${task.description}"): claude.resultAs[CodingResult].continueSession( sessionId, task.description, - LlmConfig(autoApprove = AutoApprove.All) + AgentConfig(autoApprove = AutoApprove.All) ) if !result.success then fail(s"Coding failed: ${result.message}") diff --git a/examples/epic.sc b/examples/epic.sc index 5b3ac467..841ab3df 100644 --- a/examples/epic.sc +++ b/examples/epic.sc @@ -1,22 +1,21 @@ -//> using dep "org.virtuslab::orca:0.0.14" +//> using dep "org.virtuslab::orca:0.0.14+28-eb1a8993+20260624-0842-SNAPSHOT" //> using jvm 21 /** Run an epic: a multi-task workstream with cross-agent review. * * Two layers stack here: * - * - **On-disk epic.** `.orca/plan-<hash>.md` holds the task list — generated - * on a fresh run, recovered on a resume (pending edits stashed, branch - * re-attached) to restart from the first incomplete task. Each task's - * `Status: [x]` is committed as the task lands, so a crash loses no - * progress. - * - **Cross-agent review.** Claude implements; codex reviews — the - * implementer is its own worst critic, so a separate model widens coverage - * cheaply. Fixes go back to the same Claude session. Both CLIs must be - * logged in. + * - **Resumable stages.** The stage log (`.orca/progress-<hash>.json`) tracks + * the plan and every completed task. A re-run with the same prompt resumes + * from the first incomplete stage — the plan is not re-generated, already + * finished tasks are skipped, and the implementer session is reseeded from + * the recorded brief. + * - **Cross-agent review.** Claude (opus) plans and implements; codex reviews + * — the implementer is its own worst critic, so a separate model widens + * coverage cheaply. Fixes go back to the same Claude session. Both CLIs + * must be logged in. * - * On success the plan file is removed, then a docs step updates the README - * based on what changed. + * On success a docs step updates the README based on what changed. * * Run it from a git repository, with `claude` and `codex` logged in: * @@ -30,44 +29,36 @@ import orca.{*, given} -flow(OrcaArgs(args)): - val planFile = Plan.defaultPath(userPrompt) - - // Resume `.orca/plan-<hash>.md` if it exists; otherwise plan + branch. - val plan = stage("Acquire epic"): - Plan.recoverOrCreate(planFile): - // `.value` drops the planner's read-only session — the implementer - // below mints a fresh one. - Plan.autonomous.from(userPrompt, claude.opus).value +flow(OrcaArgs(args), _.claude): + val plan = stage("Plan"): + // `.value` drops the planner's read-only session — the implementer + // below mints a fresh one. + Plan.autonomous.from(userPrompt, claude.opus).value // Stable coder session reused across every task (and the docs pass) so the - // agent retains context. Fresh — not the planner's (read-only). The runtime - // owns git: the default system prompt tells the agent not to commit, so a - // stray `git commit` can't empty the tree before `implementTaskLoop` does. - val session = claude.newSession + // agent retains context. Seeded with the plan brief; replayed on resume if + // the backend session is lost. + val session = claude.session(seed = plan.brief) // Reviewers on codex; fixes go back to the Claude session that implemented. - val reviewers: List[LlmTool[?]] = allReviewers(codex) - - Plan.implementTaskLoop(planFile, plan): task => - stage(s"Implement task: ${task.title}"): - stage("Implementation"): - val _ = claude.autonomous.run(task.description, session) + val reviewers: List[Agent[?]] = allReviewers(codex) + for task <- plan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + claude.runSeeded(task.description, session) reviewAndFixLoop( - coder = claude, - sessionId = session, + coder = claude, sessionId = session, reviewers = reviewers, - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), + reviewerSelection = ReviewerSelector.agentDriven(claude.cheap), task = task.title.value, // Format after every edit; Spotless is wired into the seed pom. formatCommand = Some("mvn -q spotless:apply") ) + // one commit per task: code + progress entry stage("Update documentation"): - val _ = claude.autonomous.run( + claude.runSeeded( "All tasks done. Update project docs (README, doc-comments) based " + "on the changes made. Only update what's affected — no new sections.", session ) - git.commit("docs: update for completed work").orThrow diff --git a/examples/implement-enhanced.sc b/examples/implement-enhanced.sc index 90d52598..acf39df6 100644 --- a/examples/implement-enhanced.sc +++ b/examples/implement-enhanced.sc @@ -1,38 +1,32 @@ -//> using dep "org.virtuslab::orca:0.0.14" +//> using dep "org.virtuslab::orca:0.0.14+28-eb1a8993+20260624-0842-SNAPSHOT" //> using jvm 21 -/** Persistent planning + coding flow that lands the work on its own branch and +/** Autonomous planning + coding flow that lands the work on its own branch and * opens a pull request. * - * Same backbone as `implement.sc` (autonomous planning → persistent - * `.orca/plan-<hash>.md` → per-task implement + review-and-fix loop), enhanced - * the same way as before — both on the planner's read-only session, so they - * cost no extra exploration: + * Same backbone as `implement.sc` (autonomous planning → per-task implement + * + review-and-fix loop), enhanced with a self-review pass on the plan: * - * 1. **`.reviewed(claude)`** — the planner critiques its own draft and + * 1. **`.reviewed(agent)`** — the planner critiques its own draft and * returns an improved plan (missing/duplicated tasks, ordering, vague - * descriptions, steps that don't fit the code). - * 1. **`.briefed(claude)`** — the planner writes a one-off codebase brief - * (modules, paths, key APIs, conventions), producing a `PlanWithBrief`. - * `plan.taskPrompt(task)` prepends it to every task so the cold-starting - * coding agents don't re-discover what the planner already learned. + * descriptions, steps that don't fit the code). Runs read-only on the + * planning session; no extra exploration cost. * - * On top of that, like `issue-pr-bugfix.sc` but prompt-driven and with no - * failing-test/CI stages, the flow runs on a fresh branch and finishes with a - * PR: + * In addition, the plan always carries a `brief` — a codebase summary the + * planner writes as part of its structured output. It seeds the implementer + * session so the cold-starting coding agents don't re-discover what the + * planner already learned. * - * 1. On a fresh run it stashes any WIP, has haiku name a branch from the - * prompt, and switches to it. - * 1. Plans + implements all tasks (each committed on the branch). - * 1. Pushes and opens a PR with a haiku-generated title + description from - * the full branch diff. A human picks the PR up from there. + * The flow runtime handles the feature branch automatically: it creates a + * branch from the prompt, commits progress to the stage log, and returns to + * the starting branch on success. Resume is stage-log based — a re-run with + * the same prompt continues from the first incomplete stage. * - * Resume: an interrupted run leaves you on the orca branch with the committed - * plan file, so re-running continues the loop in place (the `recover` guard - * skips the stash/name/checkout). A resumed run ends on the orca branch — it - * can't recover the original starting branch; a fresh run returns to where it - * started. Re-running a finished flow is a no-op: `createPr` reports - * `PrAlreadyExists`, which the flow tolerates. + * On success the flow: + * + * 1. Pushes the feature branch. + * 1. Opens a PR with a haiku-generated title + description from the full + * branch diff. A human picks the PR up from there. * * ```bash * scala-cli run implement-enhanced.sc -- "Add a multiply function to the calculator crate" @@ -42,89 +36,51 @@ */ import orca.{*, given} -// Not in the `orca.*` export wildcard; imported by name to tolerate a re-run -// against a branch whose PR a prior run already opened. -import orca.tools.PrAlreadyExists - -/** Structured branch-name suggestion — a single kebab-case slug from haiku. */ -case class BranchName(name: String) derives JsonData - -flow(OrcaArgs(args)): - val planFile = Plan.defaultPath(userPrompt) - // Captured before any branch switch so the flow can return here at the end. - val startBranch = git.currentBranch() - // Fresh run only: stash WIP, name a branch, switch to it. On resume the plan - // file is already committed on the orca branch we're sitting on, so `recover` - // short-circuits this block and the loop below continues in place — naming a - // branch here would otherwise create a second one (haiku isn't deterministic). - if Plan.recover(planFile).isEmpty then - // Stash pre-existing edits before switching branches, or they'd ride onto - // the orca branch. `ensureClean` emits a Step the user can act on - // (`git stash pop`) once the flow finishes. - val _ = git.ensureClean("orca: pre-implement stash") - val branchName = stage("Name a branch"): - claude.haiku - .resultAs[BranchName] - .autonomous - .run( - s"""Suggest a git branch name for the task below. Use a short, - |kebab-case slug prefixed with `orca/` (e.g. - |`orca/add-multiply-fn`). Output the name only. - | - |Task: $userPrompt""".stripMargin - ) - ._2 - .name - stage(s"Create branch $branchName"): - git.checkoutOrCreate(branchName) - - // Plan → review → brief, all on one read-only planner session. On resume the - // persisted plan (with its brief) is reused without re-planning. - val plan = Plan.recoverOrCreate(planFile): +// Opens a PR at the end, so return to the starting branch afterward (the +// default is to stay on the feature branch, for no-PR flows like implement.sc). +// `_.claude` selects the leading agent; the body references it as `agent` (not +// `claude`) so the flow is backend-agnostic — switch the selector to `_.codex` / +// `_.opencode` / … and the whole flow follows. +flow(OrcaArgs(args), _.claude, returnToStartBranch = true): + // Plan → review, all on one read-only planner session. The Plan structured + // output always includes a brief, which seeds the implementer session below. + val plan = stage("Plan"): Plan.autonomous - .from(userPrompt, claude) - .reviewed(claude) - .briefed(claude) + .from(userPrompt, agent) + .reviewed(agent) .value - // Fresh implementer session — the planner's was read-only (plan mode). - val session = claude.newSession - - Plan.implementTaskLoop(planFile, plan): task => - stage(s"Implement task: ${task.title}"): - stage("Implementation"): - // taskPrompt prepends the shared brief. - val _ = claude.autonomous.run(plan.taskPrompt(task), session) + // Get-or-create the implementer session. Seeded with the brief so the agent + // has codebase context from the start; replayed on resume if the backend + // session is lost. + val session = agent.session(seed = plan.brief) + for task <- plan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + // The session seed already carries the brief, so send only the task + // description here — runSeeded re-prepends the seed on first use / resume. + agent.runSeeded(task.description, session) reviewAndFixLoop( - coder = claude, - sessionId = session, - reviewers = allReviewers(claude), - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), + coder = agent, sessionId = session, + reviewers = allReviewers(agent), + reviewerSelection = ReviewerSelector.agentDriven(agent.cheap), task = task.title.value, // Format after every edit (the implementation and each review fix). formatCommand = Some("cargo fmt"), lintCommand = Some("cargo check --tests"), - lintLlm = Some(claude.haiku) + lintAgent = Some(agent.cheap) ) + // one commit per task: code + progress entry - // The task loop has removed the plan file and committed that removal, so the - // branch now holds only the implementation. Push it and open the PR from the - // full branch diff. + // Push the branch and open the PR from the full branch diff. stage("Push branch"): git.push().orThrow val prSum = stage("Generate PR title and description"): - summarisePr(llm = claude.haiku, diff = git.diffVsBase(git.defaultBase())) + summarisePr(agent = agent.cheap, diff = git.diffVsBase(git.defaultBase())) stage("Open PR"): - gh.createPr(title = prSum.title, body = prSum.body) match - case Left(_: PrAlreadyExists) => () // a prior run already opened it - case Left(e) => throw e - case Right(_) => () // the tool logs the URL - - // Return to the start branch so any stashed WIP pops back onto it. On a - // resumed run this is the orca branch itself, so it's a no-op. - stage(s"Return to $startBranch"): - git.checkout(startBranch).orThrow + // gh.createPr is idempotent by head branch (R24): if the branch already has + // an open PR, the existing handle is returned rather than failing. + gh.createPr(title = prSum.title, body = prSum.body).orThrow diff --git a/examples/implement-interactive.sc b/examples/implement-interactive.sc index 35c02331..58b80a4e 100644 --- a/examples/implement-interactive.sc +++ b/examples/implement-interactive.sc @@ -1,15 +1,18 @@ -//> using dep "org.virtuslab::orca:0.0.14" +//> using dep "org.virtuslab::orca:0.0.14+28-eb1a8993+20260624-0842-SNAPSHOT" //> using jvm 21 -/** Interactive planning + coding flow (persistent). +/** Interactive planning + coding flow. * - * Same shape as `implement.sc`, but the planner can drive a conversation: on an - * underspecified prompt it calls the `ask_user` tool to clarify before - * producing the plan. The plan persists to `.orca/plan-<hash>.md` so a re-run - * resumes from the first incomplete task. + * Same shape as `implement.sc`, but the planner can drive a conversation: on + * an underspecified prompt it calls the `ask_user` tool to clarify before + * producing the plan. Progress and resume work identically — the stage log + * (`.orca/progress-<hash>.json`) is the sole resume mechanism. An interactive + * planning stage that already completed is replayed from its recorded result + * without re-prompting on a re-run. * - * `examples/runnable/02-interactive/create-test-project.sh` seeds the calculator - * crate into a temp dir and copies this script alongside it; from there: + * `examples/runnable/02-interactive/create-test-project.sh` seeds the + * calculator crate into a temp dir and copies this script alongside it; + * from there: * * ```bash * scala-cli run implement-interactive.sc -- "Add a new arithmetic operation to the calculator crate. Ask the user which." @@ -23,33 +26,26 @@ import orca.{*, given} -flow(OrcaArgs(args)): - val planFile = Plan.defaultPath(userPrompt) - - // Resume `.orca/plan-<hash>.md` if it exists; otherwise plan interactively - // (the planner can call `ask_user` to clarify) and branch. - val plan = stage("Acquire plan"): - Plan.recoverOrCreate(planFile): - // `.value` drops the planner's session; the implementer mints its own - // (ask_user was only needed for planning). - Plan.interactive.from(userPrompt, claude).value +flow(OrcaArgs(args), _.claude): + val plan = stage("Plan"): + // `.value` drops the planner's session; the implementer mints its own + // below (ask_user was only needed for planning). + Plan.interactive.from(userPrompt, claude).value // Stable autonomous session shared by implementer and fixer (ask_user was - // only needed for planning). - val session = claude.newSession - - Plan.implementTaskLoop(planFile, plan): task => - stage(s"Implement task: ${task.title}"): - stage("Implementation"): - val _ = claude.autonomous.run(task.description, session) + // only needed for planning). The seed primes it on first use and is + // replayed if the backend session is lost on resume. + val session = claude.session(seed = plan.brief) + for task <- plan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + claude.runSeeded(task.description, session) reviewAndFixLoop( - coder = claude, - sessionId = session, + coder = claude, sessionId = session, reviewers = allReviewers(claude), - // Haiku picks the per-task reviewer subset; swap for + // claude.cheap picks the per-task reviewer subset; swap for // `ReviewerSelector.allEveryRound` to run every reviewer. - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), + reviewerSelection = ReviewerSelector.agentDriven(claude.cheap), task = task.title.value, // Format after every edit so commits stay formatted and reviewers // skip style nits. @@ -57,5 +53,6 @@ flow(OrcaArgs(args)): // Cheap sanity gate; correctness is the reviewers' and CI's job, so // skip the heavier tests. lintCommand = Some("cargo check --tests"), - lintLlm = Some(claude.haiku) + lintAgent = Some(claude.cheap) ) + // one commit per task: code + progress entry diff --git a/examples/implement.sc b/examples/implement.sc index 4482611f..3820ef54 100644 --- a/examples/implement.sc +++ b/examples/implement.sc @@ -1,14 +1,17 @@ -//> using dep "org.virtuslab::orca:0.0.14" +//> using dep "org.virtuslab::orca:0.0.14+28-eb1a8993+20260624-0842-SNAPSHOT" //> using jvm 21 -/** Persistent planning + coding flow (autonomous planning). +/** Autonomous planning + coding flow. * - * Mirrors the README example. The agent breaks the prompt into tasks, persisted - * to `.orca/plan-<hash>.md` so a re-run with the same prompt resumes from the - * first incomplete task. Each task is implemented on a single epic branch with - * a review-and-fix loop; the work and the ticked checkbox are committed - * together. When every task is done the plan file is removed and that removal - * committed. + * Mirrors the README example. The agent breaks the prompt into tasks; the task + * list and implementation progress are tracked in the stage log + * (`.orca/progress-<hash>.json`), committed on the branch after every stage. A + * re-run with the same prompt resumes from the first incomplete stage — no + * plan file, no checkbox state to keep in sync. + * + * Each task is implemented on a single feature branch (auto-created from the + * prompt) with a review-and-fix loop; code + the updated progress entry are + * committed together. The branch is auto-deleted if no code landed. * * `examples/runnable/01-simple/create-test-project.sh` seeds the calculator * crate into a temp dir and copies this script alongside it; from there: @@ -25,32 +28,28 @@ import orca.{*, given} -flow(OrcaArgs(args)): - val planFile = Plan.defaultPath(userPrompt) - - // Resume `.orca/plan-<hash>.md` if it exists; otherwise plan + branch. - val plan = stage("Acquire plan"): - Plan.recoverOrCreate(planFile): - // `.value` drops the planner's read-only session; the implementer - // mints its own. - Plan.autonomous.from(userPrompt, claude).value - - // Stable session shared by implementer and fixer, so reviews land against - // the code's own context. Fresh — not the planner's (plan mode). - val session = claude.newSession +// `_.claude` selects the leading agent (the coding harness — claude, codex, pi, +// …). Inside the body, reference it as `agent`, not `claude`, so the flow is +// backend-agnostic: switch the selector to `_.codex` / `_.opencode` / … and the +// whole body follows. `agent.cheap` is the lead's own cheap tier. +flow(OrcaArgs(args), _.claude): + val plan = stage("Plan"): + Plan.autonomous.from(userPrompt, agent).value - Plan.implementTaskLoop(planFile, plan): task => - stage(s"Implement task: ${task.title}"): - stage("Implementation"): - val _ = claude.autonomous.run(task.description, session) + // Get-or-create the implementer session. The seed (plan brief) primes it on + // first use and is replayed if the backend session is lost on resume. + val session = agent.session(seed = plan.brief) - reviewAndFixLoop( - coder = claude, + for task <- plan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + agent.runSeeded(task.description, session) + reviewAndFixLoop( // runs under this stage + coder = agent, sessionId = session, - reviewers = allReviewers(claude), - // Haiku picks the per-task reviewer subset; swap for + reviewers = allReviewers(agent), + // agent.cheap picks the per-task reviewer subset; swap for // `ReviewerSelector.allEveryRound` to run every reviewer. - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), + reviewerSelection = ReviewerSelector.agentDriven(agent.cheap), task = task.title.value, // Format after every edit so commits stay formatted and reviewers // skip style nits. @@ -58,5 +57,6 @@ flow(OrcaArgs(args)): // Cheap sanity gate; correctness is the reviewers' and CI's job, so // skip the heavier tests. lintCommand = Some("cargo check --tests"), - lintLlm = Some(claude.haiku) + lintAgent = Some(agent.cheap) ) + // one commit per task: code + progress entry diff --git a/examples/issue-pr-bugfix.sc b/examples/issue-pr-bugfix.sc index d0493727..5e3be854 100644 --- a/examples/issue-pr-bugfix.sc +++ b/examples/issue-pr-bugfix.sc @@ -1,4 +1,4 @@ -//> using dep "org.virtuslab::orca:0.0.14" +//> using dep "org.virtuslab::orca:0.0.14+28-eb1a8993+20260624-0842-SNAPSHOT" //> using jvm 21 /** Bug-report → fix flow for Scala projects, autonomous. @@ -6,25 +6,32 @@ * Given a `<owner>/<repo>#<number>` reference to a Scala-project issue, the * flow: * - * 1. Reads the issue from GitHub. + * 1. Reads the issue from GitHub (pure read, outside any stage). * 1. Triages: actually a bug? can a unit test reproduce it? - * - Not a bug → comment the verdict on the issue and stop. - * - Bug, but not testable → comment reproduction steps on the issue - * and stop (no PR for docs-only repros). - * 1. For a testable bug: write the failing test, push, open a PR with a - * tentative haiku-generated description noting only the failing test - * has landed. - * 1. Wait `CiTimeout` for CI to go red — fail loudly on green (the - * reproduction is wrong). - * 1. Sonnet inspects the failed run via `gh` (the flow never reads the - * log into memory) and posts a short focused failure comment, then - * verifies the failure matches the report — also via `gh`, by run id. - * 1. Plan + implement the fix on the same branch (read/grep, write, - * `sbt scalafmtAll`, review per task). Implementation reuses the - * triage/failing-test session. - * 1. Push the fix, then regenerate the PR title + description from the - * full branch diff (so it reads as a fix, not "add a test"). Does NOT - * wait for CI green at the end — a human picks the PR up from there. + * - Not a bug → posts the verdict on the issue. The throwaway branch (no + * code committed) is auto-deleted by the runtime on exit. + * - Bug, but not testable → posts reproduction steps on the issue and + * stops (no PR for docs-only repros). Same branch cleanup. + * 1. For a testable bug: + * a. "Write failing test" stage: writes and commits the test. b. "Push + + * open tentative PR" stage: pushes the committed test, then opens a PR + * (`gh.createPr` is idempotent by branch). These are two separate + * stages because a stage commits only on completion — a push in the + * same stage as the edit would push nothing (ADR 0018 §3.2 R8). + * 1. Waits for CI to go red (pure polling read, outside a stage). Fails + * loudly on green — the reproduction is wrong. + * 1. Confirms the failure matches the report. + * 1. Plans + implements the fix on the same branch (each task a stage). + * 1. "Push fix + finalise PR" stage: pushes the fix and regenerates the PR + * title + description from the full branch diff. + * + * The feature branch is named deterministically from the issue number + * (`fix/issue-<n>`), so a re-run after a crash lands on the same branch. + * Resume is stage-log based — each completed stage is skipped on a re-run. + * + * The flow reads top-to-bottom below; the per-step helper methods it calls + * (`confirmReproductionMatches`, `planAndImplementFix`, `prSummary`) are + * defined at the bottom of the file. * * Usage: * @@ -39,214 +46,211 @@ import orca.{*, given} import scala.concurrent.duration.DurationInt -flow(OrcaArgs(args)): +// Parse the issue handle up-front so it can seed the deterministic branch +// naming strategy passed to `flow`. A parse failure exits before the flow. +val orcaArgs = OrcaArgs(args) +val issueHandle = IssueHandle.parseOrThrow(orcaArgs.userPrompt) - val CiTimeout = 30.minutes +val CiTimeout = 30.minutes - val issueHandle = IssueHandle.parseOrThrow(userPrompt) +// Opens a PR, so return to the starting branch afterward (the default is to +// stay on the feature branch, for no-PR flows). +flow( + orcaArgs, + _.claude, + branchNaming = Some(BranchNamingStrategy.issue(issueHandle)), + returnToStartBranch = true +): + // Pure read — outside any stage (reads don't need InStage). + val issue = gh.readIssue(issueHandle) - val issue = stage(s"Read issue ${issueHandle.shortRef}"): - gh.readIssue(issueHandle) + val issuePayload = + s"""Title: ${issue.title} + |Reporter: ${issue.author} + | + |${issue.body}""".stripMargin - // Autonomous triage, read-only (read/grep to verify the report). The session - // it returns is reused for the failing-test write and the fix: a writable - // call restores write access, and the implementer inherits the exploration. - val Sessioned(session, triage) = stage("Triage"): - Plan.autonomous.triage( - s"""Title: ${issue.title} - |Reporter: ${issue.author} - | - |${issue.body}""".stripMargin, - claude.opus - ) + // Get-or-create the implementer session before the triage stage (pure: + // reserves the session id, no LLM call). The seed primes it on first use + // and is replayed if the backend session is lost on resume. + val session = claude.session(seed = issue.body) - // ============================ pipeline phases ============================ - // One def per step of the testable-bug pipeline; the `Testable` branch at the - // bottom reads as their sequence. They close over `session`, `issue`, - // `issueHandle` and `CiTimeout`. - - /** PR title + body from the full branch diff, with issue context and a - * phase-specific `note`. Used for both the tentative (test-only) and final - * (test + fix) descriptions. `git.diffVsBase` (not `git.diff()` vs HEAD) - * because the changes are already committed. - */ - def prSummary(note: String): PrSummary = - summarisePr( - llm = claude.haiku, - diff = git.diffVsBase(git.defaultBase()), - context = Some( - s"""Originating issue: ${issueHandle.shortRef} - |Issue title: ${issue.title} - | - |$note""".stripMargin - ) - ) + val triage: Triage = stage("Triage"): + // Autonomous triage, read-only (read/grep to verify the report). + Plan.autonomous.triage(issuePayload, claude).value - def writeAndPushFailingTest(summary: String, failingTestPath: String): Unit = - stage("Write the failing test"): - val _ = claude.autonomous.run( - s"""Write the failing unit test at `$failingTestPath`. It MUST - |fail on the current code — that's how we confirm the bug. - |Run `sbt test` locally if you can to verify.""".stripMargin, - session = session - ) - git.commit(s"Add failing test: $summary").orThrow - stage("Push branch"): - git.push().orThrow - - /** Open the PR with a tentative description — only the failing test has - * landed, so the body says so explicitly. - */ - def openTentativePr(): PrHandle = - val prSum = stage("Generate tentative PR title and description"): - prSummary( - "Note: this is a tentative description — only a failing test has " + - "been added so far. The fix is still pending." - ) - stage("Open PR"): - gh.createPr( - title = prSum.title, - body = s"""${prSum.body} - | - |Closes ${issueHandle.shortRef}.""".stripMargin - ).orThrow - - /** The failing test must turn CI red; a green run means the reproduction is - * wrong, so fail loudly. - */ - def awaitRedCi(pr: PrHandle): Unit = - stage("Wait for CI to fail"): - val status = gh.waitForBuild(pr, CiTimeout).orThrow - if status.outcome == BuildOutcome.Success then + triage match + case Triage.NotABug(explanation) => + stage("Comment: not a bug"): + gh.upsertComment( + issueHandle, + orcaCommentMarker(userPrompt, "not-a-bug"), + explanation + ) + + case Triage.Untestable(_, reproductionSteps) => + stage("Comment: repro steps"): + gh.upsertComment( + issueHandle, + orcaCommentMarker(userPrompt, "repro-steps"), + s"""## Reproduction + | + |$reproductionSteps""".stripMargin + ) + + case Triage.Testable(summary, _, failingTestPath) => + // Write failing test: committed by the stage. + stage("Write failing test"): + claude.runSeeded( + s"""Write the failing unit test at `$failingTestPath`. It MUST + |fail on the current code — that's how we confirm the bug. + |Run `sbt test` locally if you can to verify.""".stripMargin, + session + ) + + // Push + open PR: a SEPARATE stage from the edit above. + // Authoring rule (ADR 0018 §3.2 R8): a stage commits only on completion, + // so a push in the same stage as the edit would push nothing — the push + // must be in a later stage than the code that produced it. + val pr = stage("Push + open tentative PR"): + git.push().orThrow + // gh.createPr is idempotent by head branch (R24): if the branch already + // has an open PR, the existing handle is returned. + gh.createPr( + title = summary, + body = s"""Failing test only — fix pending. + | + |Closes ${issueHandle.shortRef}.""".stripMargin + ).orThrow + + // `waitForBuild` is a pure polling read — outside any stage. + if gh.waitForBuild(pr, CiTimeout).orThrow.outcome == BuildOutcome.Success + then fail( - "CI passed on the failing-test commit. The reproduction " + - "doesn't actually reproduce — re-triage and try again." + "CI passed on the failing-test commit — the reproduction is wrong." ) + display(s"CI red on ${pr.shortRef} — reproduction confirmed") - /** Sonnet inspects the failed run via `gh` — the flow never pulls the log - * into memory. Each sonnet turn is a fresh one-shot session, so it - * re-inspects the run by id. Posts a focused failure comment, then strictly - * verifies the failure matches the original report. - */ - def confirmReproductionMatches(pr: PrHandle): Unit = - stage("Post focused failure comment"): - val (_, failureSummary) = claude.sonnet.autonomous.run( - s"""CI went red on PR ${pr.shortRef} (${pr.url}). Inspect the - |failed run via `gh` — start with: - | - | gh pr checks ${pr.number} --repo ${pr.owner}/${pr.repo} - | - |then drill into the failing check with `gh run view <run-id> - |--log-failed --repo ${pr.owner}/${pr.repo}`. Produce a - |short, focused failure summary — the most informative - |excerpt, not the whole log. Output only the summary text; - |it goes verbatim into a PR comment.""".stripMargin - ) - gh.writeComment(pr, failureSummary) + confirmReproductionMatches(pr, issue) + planAndImplementFix(session) - stage("Verify failure matches the report"): - val (_, verdict) = - claude.sonnet.resultAs[BugReportMatch].autonomous.run( - s"""Inspect the failed CI run on PR ${pr.shortRef} via `gh` - |(`gh pr checks ${pr.number} --repo ${pr.owner}/${pr.repo}`, - |then `gh run view <run-id> --log-failed`), then decide whether - |the failure matches the original report below. Be strict — a - |different stack trace or assertion error counts as a mismatch. - | - |Original report: - |${issue.body}""".stripMargin + // Push fix + update PR: again a LATER stage than the task edits above. + stage("Push fix + finalise PR"): + git.push().orThrow + val finalSum = prSummary( + "The branch now contains both the failing test and the fix " + + "that makes it pass.", + issue ) - if !verdict.matches then - fail(s"Reproduction doesn't match the report: ${verdict.explanation}") - - /** Plan + implement the fix on the same branch. No `.orca/plan-*.md` — the - * earlier stages (triage, CI red, repro verification) aren't restartable from - * a plan file alone, so use the in-memory `implementTaskLoop`. The draft is - * self-reviewed and briefed (`reviewed`/`briefed`, both read-only), then - * `.value` drops the planning session; the fix tasks carry the brief via - * `taskPrompt` and run on the triage `session`. - */ - def planAndImplementFix(branchName: String): Unit = - val fixPlan = stage("Plan the fix"): - Plan.autonomous.from( - s"""Implement the fix for ${issueHandle.shortRef}. A failing - |test is already on this branch (`$branchName`) — the fix - |must make it pass without regressing other tests.""".stripMargin, - claude - ).reviewed(claude).briefed(claude).value - - Plan.implementTaskLoop(fixPlan): task => - stage(s"Implement task: ${task.title}"): - stage("Implementation"): - val _ = claude.autonomous.run(fixPlan.taskPrompt(task), session) - reviewAndFixLoop( - coder = claude, - sessionId = session, - reviewers = allReviewers(claude), - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), - task = task.title.value, - // Format after every edit (the implementation and each review fix). - formatCommand = Some("sbt scalafmtAll"), - // Compile (main + test) is a cheap sanity gate; the failing test runs - // in CI and correctness is the reviewers' job, so skip the full suite. - lintCommand = Some("sbt Test/compile"), - lintLlm = Some(claude.haiku) + gh.updatePr( + pr, + title = finalSum.title, + body = s"""${finalSum.body} + | + |Closes ${issueHandle.shortRef}.""".stripMargin ) - /** Push the fix and regenerate the PR title + body from the full branch diff, - * so it reads as a fix, not "add a test". Doesn't wait for CI green — a human - * takes it from here. - */ - def pushAndFinalisePr(pr: PrHandle): Unit = - stage("Push the fix"): - git.push().orThrow - stage("Update PR title and description"): - val finalSum = prSummary( - "The branch now contains both the failing test and the fix " + - "that makes it pass." - ) - gh.updatePr( - pr, - title = finalSum.title, - body = s"""${finalSum.body} - | - |Closes ${issueHandle.shortRef}.""".stripMargin - ) +// ============================ pipeline helpers ============================ +// One def per step of the testable-bug pipeline; the `Testable` branch above +// reads as their sequence. Defined below the flow so the high-level logic comes +// first; each takes a `using` flow capability plus the `issue` / `session` it +// needs. - // ============================ the flow ============================ +/** PR title + body from the full branch diff, with issue context and a + * phase-specific `note`. Used for both the tentative (test-only) and final + * (test + fix) descriptions. `git.diffVsBase` (not `git.diff()` vs HEAD) + * because the changes are already committed. + */ +def prSummary(note: String, issue: Issue)(using + FlowContext, + InStage +): PrSummary = + summarisePr( + agent = claude.cheap, + diff = git.diffVsBase(git.defaultBase()), + context = Some( + s"""Originating issue: ${issueHandle.shortRef} + |Issue title: ${issue.title} + | + |$note""".stripMargin + ) + ) - triage match - case Triage.NotABug(explanation) => - stage("Comment 'not a bug' on the issue"): - gh.writeComment(issueHandle, explanation) +/** Confirm the CI failure matches the original report. Each sub-stage is a + * one-shot sonnet call — fresh session, no seed needed. + */ +def confirmReproductionMatches(pr: PrHandle, issue: Issue)(using + FlowControl +): Unit = + stage("Post focused failure comment"): + val (_, failureSummary) = claude.sonnet.autonomous.run( + s"""CI went red on PR ${pr.shortRef} (${pr.url}). Inspect the + |failed run via `gh` — start with: + | + | gh pr checks ${pr.number} --repo ${pr.owner}/${pr.repo} + | + |then drill into the failing check with `gh run view <run-id> + |--log-failed --repo ${pr.owner}/${pr.repo}`. Produce a + |short, focused failure summary — the most informative + |excerpt, not the whole log. Output only the summary text; + |it goes verbatim into a PR comment.""".stripMargin + ) + gh.upsertComment( + pr, + orcaCommentMarker(userPrompt, "ci-failure"), + failureSummary + ) - case Triage.Untestable(_, reproductionSteps) => - stage("Comment reproduction steps on the issue"): - gh.writeComment( - issueHandle, - s"""## Reproduction - | - |$reproductionSteps""".stripMargin + stage("Verify failure matches the report"): + val (_, verdict) = + claude.sonnet + .resultAs[BugReportMatch] + .autonomous + .run( + s"""Inspect the failed CI run on PR ${pr.shortRef} via `gh` + |(`gh pr checks ${pr.number} --repo ${pr.owner}/${pr.repo}`, + |then `gh run view <run-id> --log-failed`), then decide whether + |the failure matches the original report below. Be strict — a + |different stack trace or assertion error counts as a mismatch. + | + |Original report: + |${issue.body}""".stripMargin ) + if !verdict.matches then + fail(s"Reproduction doesn't match the report: ${verdict.explanation}") - case Triage.Testable(summary, branchName, failingTestPath) => - // Capture the start branch to return to at the end: the stash below is - // taken here, so `git stash pop` only lands WIP right if we come back. - val startBranch = git.currentBranch() - - // Stash pre-existing edits before switching branches, or they'd ride onto - // the bugfix branch into the failing-test commit. `ensureClean` emits a - // Step the user can act on (`git stash pop`) once the flow finishes. - val _ = git.ensureClean("orca: pre-bugfix stash") - git.checkoutOrCreate(branchName) - - writeAndPushFailingTest(summary, failingTestPath) - val pr = openTentativePr() - awaitRedCi(pr) - confirmReproductionMatches(pr) - planAndImplementFix(branchName) - pushAndFinalisePr(pr) - - // Return to the start branch so any stashed WIP pops back onto it. - stage(s"Return to $startBranch"): - git.checkout(startBranch).orThrow +/** Plan + implement the fix on the same branch. The plan always carries a + * brief (no separate `.briefed` step); `taskPrompt` prepends it to each task. + * Implementation reuses the triage `session`. + */ +def planAndImplementFix(session: SessionId[BackendTag.ClaudeCode.type])(using + FlowControl +): Unit = + val fixPlan = stage("Plan the fix"): + Plan.autonomous + .from( + s"""Implement the fix for ${issueHandle.shortRef}. A failing + |test is already on this branch — the fix must make it pass + |without regressing other tests.""".stripMargin, + claude + ) + .reviewed(claude) + .value + + for task <- fixPlan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + claude.runSeeded(fixPlan.taskPrompt(task), session) + reviewAndFixLoop( + coder = claude, + sessionId = session, + reviewers = allReviewers(claude), + reviewerSelection = ReviewerSelector.agentDriven(claude.cheap), + task = task.title.value, + // Format after every edit (the implementation and each review fix). + formatCommand = Some("sbt scalafmtAll"), + // Compile (main + test) is a cheap sanity gate; the failing test + // runs in CI and correctness is the reviewers' job. + lintCommand = Some("sbt Test/compile"), + lintAgent = Some(claude.cheap) + ) + // one commit per task: code + progress entry diff --git a/examples/issue-pr.sc b/examples/issue-pr.sc index 29324be7..c3b56c2c 100644 --- a/examples/issue-pr.sc +++ b/examples/issue-pr.sc @@ -1,22 +1,24 @@ -//> using dep "org.virtuslab::orca:0.0.14" +//> using dep "org.virtuslab::orca:0.0.14+28-eb1a8993+20260624-0842-SNAPSHOT" //> using jvm 21 /** GitHub-issue → PR flow, fully autonomous. * * Given a `<owner>/<repo>#<number>` reference (the user's prompt), the flow: * - * 1. Reads the issue from GitHub (title, body, author). - * 1. Resumes `.orca/plan-<hash>.md` if one exists (crash recovery); - * otherwise skeptically assesses the report against the repo — claims, - * missing detail, duplicates, scope. The agent returns a plan or a - * critique / follow-up question / rebuff. - * 1. On rejection: posts the agent's reply on the issue and exits. - * 1. On proceed: creates the epic branch, persists the plan, and runs - * `Plan.implementTaskLoop` — each task gets the review-and-fix loop, a - * checkbox tick, and a `task: <title>` commit; the plan file is removed - * once every task is done. - * 1. Pushes the branch, folds the diff into a PR title + description via a - * cheap model (`summarisePr`), and opens the PR via `gh`. + * 1. Reads the issue from GitHub (title, body, author) — outside any stage, + * since it is a pure read. + * 1. Skeptically assesses the report against the repo (claims, missing + * detail, duplicates, scope) and either proceeds with a plan or rejects. + * 1. On rejection: posts the agent's reply on the issue. The throwaway + * branch (no code committed) is auto-deleted by the runtime on exit. + * 1. On proceed: runs the per-task implement + review-and-fix loop. The + * task list and progress live in the stage log; a re-run resumes from + * the first incomplete stage. + * 1. Pushes the branch, folds the diff into a PR title + description via + * haiku (`summarisePr`), and opens the PR via `gh` (idempotent by branch). + * + * The feature branch is named deterministically from the issue number + * (`fix/issue-<n>`), so a re-run after a crash lands on the same branch. * * Usage — pass `<owner>/<repo>#<number>`: * @@ -29,16 +31,23 @@ import orca.{*, given} -flow(OrcaArgs(args)): - - val issueHandle = IssueHandle.parseOrThrow(userPrompt) +// Parse the issue handle up-front so it can seed the deterministic branch +// naming strategy passed to `flow`. A parse failure exits before the flow. +val orcaArgs = OrcaArgs(args) +val issueHandle = IssueHandle.parseOrThrow(orcaArgs.userPrompt) + +// returnToStartBranch: this flow opens a PR, so switch back to the starting +// branch afterward (ready for the next task) rather than staying on the feature +// branch — which is the default for no-PR flows like implement.sc. +flow( + orcaArgs, + _.claude, + branchNaming = Some(BranchNamingStrategy.issue(issueHandle)), + returnToStartBranch = true +): + // Pure read — outside any stage (reads don't need InStage). + val issue = gh.readIssue(issueHandle) - // 1. Pull the issue from GitHub. - val issue = stage(s"Read issue ${issueHandle.shortRef}"): - gh.readIssue(issueHandle) - - // Title + body. Comments are excluded by default — noisy threads pull the - // planner off-target. val issuePayload = s"""Issue: ${issue.title} | @@ -46,62 +55,51 @@ flow(OrcaArgs(args)): | |${issue.body}""".stripMargin - // Resume an in-progress plan for this issue if one exists; otherwise assess - // (Opus runs read-only to verify claims via Read/Grep). `Verdict.Proceed` - // persists the plan so a crash resumes from the first incomplete task; - // `Verdict.Rejection` posts the assessment on the issue and exits. - - // Capture the start branch to return to at the end: Plan.recover / - // checkoutOrCreate below switch away and stash WIP, so `git stash pop` - // only lands right if we come back. - val startBranch = git.currentBranch() - - val planFile = Plan.defaultPath(userPrompt) - val maybePlan = stage("Acquire plan"): - Plan - .recover(planFile) - .orElse: - Plan.autonomous.assessThenPlan(issuePayload, claude.opus).value match - case Verdict.Rejection(_, body) => - stage("Post assessment on the issue"): - gh.writeComment(issueHandle, body) - None - case Verdict.Proceed(plan) => - git.checkoutOrCreate(plan.epicId) - os.write.over(planFile, Plan.render(plan), createFolders = true) - Some(plan) + // Stage returns (plan, rejectionBody): exactly one of (Some(plan), "") or + // (None, body). Splitting the verdict and the comment into two stages means + // a crash between them doesn't double-post the comment on resume. + val (maybePlan, rejectionBody) = stage("Assess and plan"): + Plan.autonomous.assessThenPlan(issuePayload, claude.opus).value match + case Verdict.Rejection(_, body) => (None: Option[Plan], body) + case Verdict.Proceed(plan) => (Some(plan), "") + + if maybePlan.isEmpty then + stage("Comment: rejection"): + gh.upsertComment( + issueHandle, + orcaCommentMarker(userPrompt, "reject"), + rejectionBody + ) maybePlan.foreach: plan => - // Fresh session — the assess session was read-only (plan mode). Reused - // across tasks so the implementer retains context. - val session = claude.newSession - - Plan.implementTaskLoop(planFile, plan): task => - stage(s"Implement task: ${task.title}"): - stage("Implementation"): - val _ = claude.autonomous.run(task.description, session) + // Get-or-create the implementer session. Seeded with the brief so the + // agent has codebase context; replayed on resume if the session is lost. + val session = claude.session(seed = plan.brief) + for task <- plan.tasks do + stage(s"task: ${task.title}"): // skipped on resume if already done + claude.runSeeded(task.description, session) reviewAndFixLoop( - coder = claude, - sessionId = session, + coder = claude, sessionId = session, reviewers = allReviewers(claude), - // Haiku picks the per-task reviewer subset; swap for + // claude.cheap picks the per-task reviewer subset; swap for // `ReviewerSelector.allEveryRound` to run every reviewer. - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), + reviewerSelection = ReviewerSelector.agentDriven(claude.cheap), task = task.title.value, // Format after every edit; Prettier for a TS/JS project — swap for // your formatter. formatCommand = Some("npx prettier --write .") ) + // one commit per task: code + progress entry stage("Push branch"): git.push().orThrow val summary = stage("Generate PR title and description"): summarisePr( - llm = claude.haiku, + agent = claude.cheap, // Branch-vs-base diff — `git.diff()` (vs HEAD) would be empty, since - // `implementTaskLoop` already committed every task. + // every task is already committed. diff = git.diffVsBase(git.defaultBase()), context = Some( s"""Originating issue: ${issueHandle.shortRef} @@ -114,8 +112,4 @@ flow(OrcaArgs(args)): s"""${summary.body} | |Closes ${issueHandle.shortRef}.""".stripMargin - val _ = gh.createPr(title = summary.title, body = body).orThrow - - // Return to the start branch so any stashed WIP pops back onto it. - stage(s"Return to $startBranch"): - git.checkout(startBranch).orThrow + gh.createPr(title = summary.title, body = body).orThrow diff --git a/flow/src/main/resources/orca/plan/prompts/assess-then-plan.md b/flow/src/main/resources/orca/plan/prompts/assess-then-plan.md index 8eb664b8..52793cd4 100644 --- a/flow/src/main/resources/orca/plan/prompts/assess-then-plan.md +++ b/flow/src/main/resources/orca/plan/prompts/assess-then-plan.md @@ -19,11 +19,15 @@ Things to look for: Then return one of: - `verdict: "proceed"` with a `plan` (the same plan shape used by the - autonomous planner — epic id, description, ordered list of tasks). Each - task should be atomic (impl + tests together), independent of later tasks, - shippable on its own, and small enough for one focused implementer turn. - Do NOT edit files or run code during this assessment turn; planning is the - output. + autonomous planner — epic id, description, ordered list of tasks, and a + `brief`). Each task should be atomic (impl + tests together), independent of + later tasks, shippable on its own, and small enough for one focused + implementer turn. Fill the plan's `brief` field with a concise codebase + briefing the implementing agents (who start from a fresh context) will rely + on: the modules/files involved with paths, the key types and APIs to build + on, the conventions to follow, and anything non-obvious you learned while + verifying the report. Do NOT edit files or run code during this assessment + turn; planning is the output. - `verdict: "reject"` with `rejectKind` and `rejectBody`. `rejectKind` is one of: diff --git a/flow/src/main/resources/orca/plan/prompts/brief.md b/flow/src/main/resources/orca/plan/prompts/brief.md deleted file mode 100644 index 35bb290c..00000000 --- a/flow/src/main/resources/orca/plan/prompts/brief.md +++ /dev/null @@ -1,14 +0,0 @@ -The plan we just produced will be implemented by a separate coding agent, -starting from a fresh context — it has NOT seen your exploration of this -codebase. Write a single briefing it can rely on so it doesn't have to -rediscover it. - -Include, as concise notes: the modules and directories involved and what each is -responsible for; the specific files (with paths) it will read or change, and -why; the key types, functions, and APIs it will build on, with signatures; the -conventions to follow (error handling, naming, testing, build); and anything -non-obvious you learned that would otherwise cost a re-read. - -Do NOT restate the tasks or the plan — the agent already has those, and do NOT -edit files or run mutating commands. Output only the briefing, as plain -markdown. diff --git a/flow/src/main/resources/orca/plan/prompts/planning.md b/flow/src/main/resources/orca/plan/prompts/planning.md index 60e649ac..6de5b9de 100644 --- a/flow/src/main/resources/orca/plan/prompts/planning.md +++ b/flow/src/main/resources/orca/plan/prompts/planning.md @@ -9,4 +9,13 @@ the context an implementer would need before touching any single task. Keep it concrete and goal-oriented; do not restate the task list. Each task should be: atomic (impl + tests together), independent of later tasks, -shippable on its own, and small enough for one focused implementer turn. \ No newline at end of file +shippable on its own, and small enough for one focused implementer turn. + +Also fill the `brief` field: a single codebase briefing the implementing agents +will rely on, since they start from a fresh context and have NOT seen your +exploration. Include, as concise notes: the modules and directories involved and +what each is responsible for; the specific files (with paths) to read or change, +and why; the key types, functions, and APIs to build on, with signatures; the +conventions to follow (error handling, naming, testing, build); and anything +non-obvious you learned that would otherwise cost a re-read. Do NOT restate the +tasks in the brief. \ No newline at end of file diff --git a/flow/src/main/scala/orca/BranchNamingStrategy.scala b/flow/src/main/scala/orca/BranchNamingStrategy.scala new file mode 100644 index 00000000..a8b1b23b --- /dev/null +++ b/flow/src/main/scala/orca/BranchNamingStrategy.scala @@ -0,0 +1,119 @@ +package orca + +import orca.agents.Agent +import orca.tools.IssueHandle + +import java.nio.charset.StandardCharsets +import java.security.MessageDigest + +/** Strategy that produces a git-ref-safe feature-branch name. Resolved once per + * flow run, inside a stage (the `InStage` token gates the call). + */ +trait BranchNamingStrategy: + /** Resolve the feature-branch name. `userPrompt` is the flow's prompt; + * `agent` is the leading model (used only by the prompt-shortening + * strategy). + */ + def resolve(userPrompt: String, agent: Agent[?])(using InStage): String + +object BranchNamingStrategy: + + /** Git-ref-safe slug (PURE). Lower-case; keep only `[a-z0-9-]`; replace runs + * of other chars with a single `-`; strip leading/trailing `-`; cap to + * `maxLen` without leaving a trailing `-`. If the result is empty or still + * starts with `-`, return `"flow-<shorthash>"` where `<shorthash>` is a + * short hex hash of `text` — the ref is NEVER empty and NEVER begins with + * `-` (ADR 0018 §2.5). + * + * `maxLen` is clamped to a minimum of 1 so a zero/negative cap can't + * silently force the fallback. + */ + def slug(text: String, maxLen: Int = 50): String = + val cleaned = clean(text) + val capped = cap(cleaned, math.max(1, maxLen)) + if capped.isEmpty || capped.startsWith("-") then fallback(text) + else capped + + /** True iff `c` may appear in a slug: lower-case alphanumeric or `-`. */ + private[orca] def isSlugChar(c: Char): Boolean = + (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-' + + /** True iff `s` is a single git-ref-safe segment of the exact shape [[slug]] + * produces: non-empty, leading alphanumeric, only `[a-z0-9-]`. The producer + * ([[slug]]) and validators (e.g. recovery's untrusted-header check, which + * splits on `/` and checks each segment) share this one definition so they + * cannot drift. + */ + private[orca] def isSlugSegment(s: String): Boolean = + s.nonEmpty && isSlugChar(s.head) && s.head != '-' && s.forall(isSlugChar) + + /** `<prefix>/issue-<number>` — number is an Int, prefix slugged; safe by + * construction. `resolve` ignores `userPrompt` and `agent`. + */ + def issue( + handle: IssueHandle, + prefix: String = "fix" + ): BranchNamingStrategy = + // Slug the prefix once at construction, not on every `resolve` call. + val prefixSlug = slug(prefix) + new BranchNamingStrategy: + def resolve(userPrompt: String, agent: Agent[?])(using InStage): String = + s"$prefixSlug/issue-${handle.number}" + + /** Deterministic strategy: slugs `text` to produce the branch name. `resolve` + * ignores `userPrompt` and `agent`; `text` is evaluated once per `resolve` + * call (by-name for callers that want late binding). + */ + def fromText(text: => String): BranchNamingStrategy = + new BranchNamingStrategy: + def resolve(userPrompt: String, agent: Agent[?])(using InStage): String = + slug(text) + + /** Prompt-shortening strategy: asks `agent.cheap` for a 3–6 word lowercase + * branch label, then slugs it. Falls back to `slug(userPrompt)` on any + * failure (LLM throws, empty/blank result) so branch naming can never break + * the flow. Non-deterministic — computed once and persisted in the header; + * never recomputed on resume. + */ + val shortenPrompt: BranchNamingStrategy = + new BranchNamingStrategy: + def resolve(userPrompt: String, agent: Agent[?])(using InStage): String = + // `slug` is total (never empty), so the cheap-model reply OR the + // userPrompt fallback both produce a valid ref. + slug( + agent.cheapOneShot( + s"Reply with ONLY a 3–6 word lowercase branch label (hyphen-separated, no other punctuation) that summarises this task:\n\n$userPrompt", + fallback = userPrompt + ) + ) + + // -------------------------------------------------------------------------- + // Private helpers + // -------------------------------------------------------------------------- + + /** Lower-case, replace every run of non-`[a-z0-9]` chars (after downcasing) + * with a single `-`, and strip leading/trailing `-`. The `+` already + * collapses runs, so no second collapsing pass is needed. + */ + private def clean(text: String): String = + text.toLowerCase + .replaceAll("[^a-z0-9]+", "-") + .stripPrefix("-") + .stripSuffix("-") + + /** Truncate to at most `maxLen` chars, then strip any trailing `-` that the + * cut may have exposed. + */ + private def cap(s: String, maxLen: Int): String = + s.take(maxLen).stripSuffix("-") + + /** Deterministic `"flow-<8-hex-chars>"` fallback. Uses the first 8 chars of + * the SHA-256 hex digest of `text` (or of the empty string when `text` is + * blank) so the name is stable across calls with the same input. + */ + private def fallback(text: String): String = + val bytes = MessageDigest + .getInstance("SHA-256") + .digest(text.getBytes(StandardCharsets.UTF_8)) + val hex = bytes.map("%02x".format(_)).mkString.take(8) + s"flow-$hex" diff --git a/flow/src/main/scala/orca/Flow.scala b/flow/src/main/scala/orca/Flow.scala index 275cc108..13f1b36b 100644 --- a/flow/src/main/scala/orca/Flow.scala +++ b/flow/src/main/scala/orca/Flow.scala @@ -1,23 +1,86 @@ package orca +import com.github.plokhotnyuk.jsoniter_scala.core.{ + readFromString, + writeToString +} import orca.events.OrcaEvent +import orca.agents.JsonData +import orca.progress.StageEntry +import org.slf4j.LoggerFactory import scala.util.control.NonFatal -/** Wrap `body` as a named stage, emitting StageStarted before and - * StageCompleted after successful completion. Non-fatal exceptions from `body` - * trigger an Error event with the stage name and the exception is re-raised. - * Fatal errors (OOM, InterruptedException, control throwables) propagate - * without event emission, as they signal shutdown rather than a stage outcome. +private val log = LoggerFactory.getLogger("orca.flow") + +/** Run `body` as a named, resumable, committing stage (ADR 0018 §2.1). + * + * Control flow: * - * A body that calls `fail(...)` already emits its own Error, so - * OrcaFlowException is re-raised without a second Error event. + * - Compute the stage id `name#occurrence`, where `occurrence` counts prior + * same-named stages in this run. + * - Resume: if the progress log already holds an entry for this id whose + * stored JSON decodes to `T`, emit StageStarted/StageCompleted and return + * the decoded value without running `body`. A decode failure (the stage's + * result type changed under this id) is fail-safe: fall through and run. + * - Run: emit StageStarted, supply an `InStage` token, evaluate `body`. + * - Record & commit: append a `StageEntry(id, name, resultJson)` to the log, + * force-add the log file, then commit (the commit also `add -A`s code + * changes, so a stage yields one commit covering code + progress). Emit + * StageCompleted. + * + * Error handling mirrors `fail`/tool-adapter semantics: a non-fatal failure in + * `body` emits an Error event (once — `fail` and malformed-output already + * carry their own emission state) and re-raises. Fatal throwables propagate + * unreported, as they signal shutdown rather than a stage outcome. */ -def stage[T](name: String)(body: => T)(using ctx: FlowContext): T = - ctx.emit(OrcaEvent.StageStarted(name)) +def stage[T: JsonData]( + name: String, + commitMessage: Option[T => String] = None +)(body: InStage ?=> T)(using fc: FlowControl): T = + val id = s"$name#${fc.nextOccurrence(name)}" + resumeFrom(id, name).getOrElse(runStage(id, name, commitMessage)(body)) + +/** Try to skip the stage by replaying a recorded result. `Some(value)` when the + * log holds an entry for `id` that decodes to `T`; `None` when there's no + * entry or it no longer decodes (fail-safe: the caller then re-runs the body). + */ +private def resumeFrom[T: JsonData](id: String, name: String)(using + fc: FlowControl +): Option[T] = + fc.progressStore + .load() + .flatMap(_.entries.find(_.id == id)) + .flatMap: entry => + // Only the decode is fail-safe: a decode failure means the stage's result + // type changed under this id, so we fall through and re-run (None). The + // emits must NOT be inside the try — if `emit` threw, a catch here would + // spuriously re-run an already-recorded stage. + val decoded = + try + Some( + readFromString[T](entry.resultJson)(using summon[JsonData[T]].codec) + ) + catch case NonFatal(_) => None + decoded.map: value => + fc.emit(OrcaEvent.StageStarted(name)) + fc.emit(OrcaEvent.Step(s"Resuming '$name' from recorded result")) + fc.emit(OrcaEvent.StageCompleted(name)) + value + +/** Run the body fresh, then record its result and commit (steps 3–4 above). */ +private def runStage[T: JsonData]( + id: String, + name: String, + commitMessage: Option[T => String] +)(body: InStage ?=> T)(using fc: FlowControl): T = + fc.emit(OrcaEvent.StageStarted(name)) try - val result = body - ctx.emit(OrcaEvent.StageCompleted(name)) + val result = + given InStage = InStage.unsafe + body + recordAndCommit(id, name, result, commitMessage) + fc.emit(OrcaEvent.StageCompleted(name)) result catch case e: OrcaFlowException => @@ -29,38 +92,99 @@ def stage[T](name: String)(body: => T)(using ctx: FlowContext): T = // exception so an enclosing stage / the flow boundary doesn't // re-report it as it unwinds. e match - case mao: orca.llm.MalformedAgentOutputException => - ctx.emit(OrcaEvent.Error(formatMalformedOutput(name, mao))) + case mao: orca.agents.MalformedAgentOutputException => + fc.emit(OrcaEvent.Error(formatMalformedOutput(name, mao))) e.alreadyEmitted = true case _ if e.alreadyEmitted => () case _ => - ctx.emit( - OrcaEvent.Error(s"Stage '$name' failed: ${throwableMessage(e, firstLineOnly = true)}") + fc.emit( + OrcaEvent.Error( + s"Stage '$name' failed: ${throwableMessage(e, firstLineOnly = true)}" + ) ) e.alreadyEmitted = true throw e case NonFatal(e) => - ctx.emit( - OrcaEvent.Error(s"Stage '$name' failed: ${throwableMessage(e, firstLineOnly = true)}") + fc.emit( + OrcaEvent.Error( + s"Stage '$name' failed: ${throwableMessage(e, firstLineOnly = true)}" + ) ) throw e -/** A throwable's human message: its `getMessage` (or the class name when blank), - * optionally collapsed to its first line. Shared by `stage` (first line, for a - * tidy one-line `✖`) and the flow boundary (whole message, so multi-line - * diagnostics like opencode's start-failure stderr survive). +/** Append the stage's result to the log and commit code + log as one commit. + * The progress file is force-added (so it lands even when `.orca/` is + * gitignored); `git.commit`'s own `add -A` picks up any code changes. The log + * always changed, so a `NothingToCommit` is unexpected — logged at DEBUG (so + * it's observable) rather than failed, since the recorded result is what + * matters for resume. + */ +private def recordAndCommit[T: JsonData]( + id: String, + name: String, + result: T, + commitMessage: Option[T => String] +)(using fc: FlowControl): Unit = + val resultJson = writeToString(result)(using summon[JsonData[T]].codec) + // Deliberately mints a fresh runtime `InStage` rather than threading the + // body's token: recording + committing the stage result is the runtime's own + // privileged step, not part of the user body. + given InStage = InStage.unsafe + // Capture the code diff BEFORE force-adding the progress file so the LLM + // sees only the body's substantive changes, not the orca bookkeeping. + val message = + commitMessage.map(_(result)).getOrElse(defaultCommitMessage(name)) + fc.progressStore.appendEntry(StageEntry(id, name, resultJson)) + fc.git.forceAdd(fc.progressStore.path) + // The log always changed, so a clean tree here is unexpected (a prior partial + // run may already have committed this entry). Surface it at DEBUG so it's + // observable rather than silent; never fail the stage over it. + fc.git.commit(message) match + case Right(()) => () + case Left(_) => + log.debug("stage {} commit was empty (already recorded?)", name) + +/** Generate a commit message from the current working-tree diff via the leading + * agent's cheap model (`fc.agent.cheapOneShot`). The diff is captured before + * the progress file is force-added, so it reflects only code changes the stage + * body produced. Falls back to `"stage: <name>"` when the diff is empty, the + * agent returns blank, or any `NonFatal` is thrown — committing must never + * break. Only called when the caller supplied no explicit `commitMessage`. + */ +private def defaultCommitMessage( + name: String +)(using fc: FlowControl, ev: InStage): String = + val fallback = s"stage: $name" + // `git.diff` is a read and shouldn't fail, but stay defensive: a commit + // message must never break a stage. The cheap agent call is guarded by + // `cheapOneShot` itself. + val diff = + try fc.git.diff() + catch case NonFatal(_) => "" + if diff.isBlank then fallback + else + fc.agent.cheapOneShot( + s"Write a concise one-line git commit message (imperative mood, ≤72 chars) for this diff:\n\n$diff", + fallback + ) + +/** A throwable's human message: its `getMessage` (or the class name when + * blank), optionally collapsed to its first line. Shared by `stage` (first + * line, for a tidy one-line `✖`) and the flow boundary (whole message, so + * multi-line diagnostics like opencode's start-failure stderr survive). */ private[orca] def throwableMessage( e: Throwable, firstLineOnly: Boolean = false ): String = val msg = Option(e.getMessage).filter(_.nonEmpty) - val picked = if firstLineOnly then msg.flatMap(_.linesIterator.nextOption()) else msg + val picked = + if firstLineOnly then msg.flatMap(_.linesIterator.nextOption()) else msg picked.getOrElse(e.getClass.getName) private def formatMalformedOutput( stage: String, - e: orca.llm.MalformedAgentOutputException + e: orca.agents.MalformedAgentOutputException ): String = val snippet = val collapsed = e.rawOutput.replaceAll("\\s+", " ").trim @@ -72,6 +196,13 @@ private def formatMalformedOutput( | hint: tighten the system prompt to enforce JSON-only, or set | ORCA_DEBUG=1 to see the full response.""".stripMargin +/** Show a progress line without checkpointing: no stage, id, commit, or log + * entry (ADR 0018 §2.1). Needs only `FlowContext`, so it's callable anywhere — + * outside a stage, or inside a fork. + */ +def display(message: String)(using ctx: FlowContext): Unit = + ctx.emit(OrcaEvent.Step(message)) + def fail(message: String)(using ctx: FlowContext): Nothing = ctx.emit(OrcaEvent.Error(message)) throw new OrcaFlowException(message, alreadyEmitted = true) diff --git a/flow/src/main/scala/orca/FlowContext.scala b/flow/src/main/scala/orca/FlowContext.scala index 74938070..e35a4365 100644 --- a/flow/src/main/scala/orca/FlowContext.scala +++ b/flow/src/main/scala/orca/FlowContext.scala @@ -4,7 +4,17 @@ import orca.events.OrcaEvent import orca.tools.FsTool import orca.tools.GitTool import orca.tools.GitHubTool -import orca.llm.{ClaudeTool, CodexTool, GeminiTool, OpencodeTool, PiTool} +import orca.agents.{ + Agent, + BackendTag, + ClaudeAgent, + CodexAgent, + GeminiAgent, + OpencodeAgent, + PiAgent +} + +import scala.annotation.implicitNotFound /** Ambient context a flow script operates in. Bundles every tool the top- level * accessors (`claude`, `codex`, `opencode`, `pi`, `gemini`, `git`, `gh`, `fs`) @@ -17,12 +27,32 @@ import orca.llm.{ClaudeTool, CodexTool, GeminiTool, OpencodeTool, PiTool} * `flow(args): ...` block and let Scala 3's context functions resolve the * given instance. */ +@implicitNotFound( + "the flow tools (`claude`/`codex`/`git`/`gh`/`fs`/…), `display`, and `fail` are only available inside a `flow(...)` body. Wrap this code in `flow(OrcaArgs(args), _.claude): ...`." +) trait FlowContext: - def claude: ClaudeTool - def codex: CodexTool - def opencode: OpencodeTool - def pi: PiTool - def gemini: GeminiTool + /** Backend tag of the leading agent. You never write this — it's the backend + * of the agent your `flow(...)` selector picked, and it's why `agent` is + * precisely typed (`Agent[LeadB]`) so sessions thread. (A type *member* + * rather than a parameter, so `FlowContext` stays unparametrised — every + * `using FlowContext` site is unaffected; the runtime captures the concrete + * tag here at construction, with `flow` generic over it, inferred from the + * selector.) + */ + type LeadB <: BackendTag + + /** The leading agent. Reference it in a body instead of a concrete accessor + * (`claude`/`codex`) so the flow is backend-agnostic — switch the `flow` + * selector and the whole flow follows. A session from `agent.session` + * threads into `agent.runSeeded` and the reviewers because [[LeadB]] pins + * the backend. + */ + def agent: Agent[LeadB] + def claude: ClaudeAgent + def codex: CodexAgent + def opencode: OpencodeAgent + def pi: PiAgent + def gemini: GeminiAgent def git: GitTool def gh: GitHubTool def fs: FsTool diff --git a/flow/src/main/scala/orca/FlowControl.scala b/flow/src/main/scala/orca/FlowControl.scala new file mode 100644 index 00000000..b5beb0ab --- /dev/null +++ b/flow/src/main/scala/orca/FlowControl.scala @@ -0,0 +1,46 @@ +package orca + +import orca.progress.ProgressStore + +import scala.annotation.implicitNotFound + +/** Marker capability: the holder is permitted to start a new stage. + * + * `FlowControl` is a subtype of [[FlowContext]] so that any code requiring + * only a `FlowContext` can be given a `FlowControl` without an explicit cast. + * The reverse is not true — a plain `FlowContext` cannot be widened to + * `FlowControl` — which lets `flow` hand forks only the narrow type, + * preventing them from starting nested stages. + * + * Thread-affine: one `FlowControl` exists per top-level `flow(...)` invocation + * and must not be shared across threads (ADR 0018 §2.2). + * + * Not sealed: its implementation (`DefaultFlowContext`) lives in the `runner` + * module, which depends on `flow`, not the reverse. This is an accepted + * guard-rail — the open trait is not part of the public extension surface. + * + * Carries the run-state the `stage` primitive needs: the progress store to + * read/append against and a per-run occurrence counter that disambiguates + * same-named stages (ADR 0018 §2.1). `stage` requires `(using FlowControl)`; + * `flow` supplies it. + */ +@implicitNotFound( + "`stage(...)`, `agent.session(...)`, and `agent.runSeeded(...)` can only be called inside a `flow(...)` body — and not inside a `fork` (forks can read and emit, but can't start stages). If this is a helper that starts stages, declare it `(using FlowControl)` so its caller supplies it." +) +trait FlowControl extends FlowContext: + /** The store backing this run's progress log. */ + def progressStore: ProgressStore + + /** Next occurrence index for a stage `name` in this run: 0 for the first + * `stage(name)`, 1 for the second, and so on. Stages run sequentially, so + * this is plain per-run bookkeeping. Used to build a stage id (`name#index`) + * that is stable against inserting/removing *other* stages between runs. + */ + def nextOccurrence(stageName: String): Int + + /** Next session occurrence index in this run: 0 for the first + * `agent.session(...)`, 1 for the second, and so on. Independent of the + * stage counter so sessions can be obtained outside stages without + * perturbing stage ids. + */ + def nextSessionOccurrence(): Int diff --git a/flow/src/main/scala/orca/Session.scala b/flow/src/main/scala/orca/Session.scala new file mode 100644 index 00000000..ec9989fb --- /dev/null +++ b/flow/src/main/scala/orca/Session.scala @@ -0,0 +1,143 @@ +package orca + +import orca.agents.{BackendTag, Agent, SessionId} +import orca.events.OrcaEvent +import orca.progress.{ProgressLog, SessionRecord} + +/** Get-or-create session extension for `Agent`. Lives in the `flow` module so + * it can depend on [[FlowControl]] (which is in `flow`) while [[Agent]] + * remains in `tools` (which `flow` depends on, not the reverse). + */ +extension [B <: BackendTag](agent: Agent[B]) + /** Get-or-create a session keyed by call-occurrence in this run's log. + * + * Reserves/returns a [[SessionId]] and records `(id, seed)` in the progress + * log; the backend conversation is created lazily on the first gated `run`. + * On resume, returns the id recorded at this occurrence (does not mint a + * second). The seed is only recorded here — applying it on first use and + * replaying it on loss are separate later tasks. + * + * The key is **positional** (the n-th `session(...)` call in the run), like + * `stage`'s id. So across runs, keep `session(...)` calls in a stable order + * and unconditional — reordering them, or skipping one on some runs, shifts + * every later session's identity and loses resume continuity. + * + * No LLM call and no commit — so it is callable outside a stage. (The id is + * a fresh UUID, so it is not referentially transparent.) The store write + * uses a runtime-minted `InStage.unsafe` (the same pattern the `stage` + * runtime uses for setup-phase mutations). + */ + def session(seed: String)(using fc: FlowControl): SessionId[B] = + val idx = fc.nextSessionOccurrence() + fc.progressStore.load().flatMap(_.sessions.find(_.index == idx)) match + case Some(recorded) => + // Sessions are keyed by positional index (see the scaladoc), so a + // recorded seed differing from this call's means the `session(...)` + // sequence likely shifted between runs, or the author edited the seed. + // The recorded session is reused either way (re-seed is the safe + // fallback, ADR 0018 §2.6) — surface the divergence rather than resume + // the wrong session silently. + if recorded.seed != seed then + fc.emit( + OrcaEvent.Step( + s"warning: session #$idx resumed with a seed differing from the " + + "recorded one; using the recorded session (the call sequence " + + "may have shifted, or the seed was edited)" + ) + ) + SessionId[B](recorded.id) + case None => + // First run: mint a fresh id, record it, and return it. + val freshId = SessionId.fresh[B] + given InStage = InStage.unsafe + fc.progressStore.upsertSession( + SessionRecord(index = idx, id = freshId.value, seed = seed) + ) + freshId + + /** Run the agent autonomously against `session`, priming it with the recorded + * seed + a progress preamble IF the backend conversation isn't live (a fresh + * first use, or lost on resume). If the session is live, runs `prompt` as-is + * (continues the conversation). Returns the run's (SessionId, output). + * + * The seed is looked up from the progress log by matching `session`'s id; if + * no record is found the seed is treated as empty (does not throw). + * + * The progress preamble names completed stages and is only included when + * there is at least one completed entry — a true first use gets just `seed + + * prompt`, with no misleading "resuming" text. + */ + def runSeeded( + prompt: String, + session: SessionId[B] + )(using fc: FlowControl, ev: InStage): (SessionId[B], String) = + val result = + if agent.sessionExists(session) then agent.autonomous.run(prompt, session) + else + val log = fc.progressStore.load() + val seed = lookupSeed(log, session) + val preamble = progressPreamble(log) + val primedPrompt = composePrimedPrompt(preamble, seed, prompt) + agent.autonomous.run(primedPrompt, session) + persistResumeWireId(agent, session) + result + +/** After a run, persist the wire id to resume against that the backend has now + * learned (durable backends only — pi returns `None`), so a resumed run can + * rehydrate the map and continue/probe the right session. Upserts the matching + * [[SessionRecord]] only when the learned wire id differs from what is already + * recorded (and a record for `session` exists), so a no-op run writes nothing. + * The store write uses a runtime-minted `InStage.unsafe`, the same pattern + * [[session]] uses for the setup-phase write. + */ +private def persistResumeWireId[B <: BackendTag]( + agent: Agent[B], + session: SessionId[B] +)(using fc: FlowControl): Unit = + agent + .resumeWireId(session) + .foreach: wireId => + val log = fc.progressStore.load() + log + .flatMap(_.sessions.find(_.id == session.value)) + .foreach: record => + if !record.resumeWireId.contains(wireId.value) then + given InStage = InStage.unsafe + fc.progressStore.upsertSession( + record.copy(resumeWireId = Some(wireId.value)) + ) + +/** Look up the recorded seed for `session` from the log. Returns `None` if the + * log is absent, no record matches `session`, or the recorded seed is empty. + */ +private def lookupSeed[B <: BackendTag]( + log: Option[ProgressLog], + session: SessionId[B] +): Option[String] = + log.flatMap( + _.sessions.find(_.id == session.value).map(_.seed).filter(_.nonEmpty) + ) + +/** Compose the progress preamble from completed stage names in the log. Returns + * `None` if there are no completed entries (first run). + */ +private def progressPreamble(log: Option[ProgressLog]): Option[String] = + val completed = log.map(_.entries.map(_.name)).getOrElse(Nil) + if completed.isEmpty then None + else + Some( + s"Progress so far: completed ${completed.mkString(", ")}. Continue from here." + ) + +/** Assemble the final primed prompt from the optional preamble, optional seed, + * and the caller's prompt, omitting absent parts cleanly. The `---` separator + * appears ONLY when there is a non-empty context (preamble or seed); when + * neither is present the prompt is returned verbatim. + */ +private def composePrimedPrompt( + preamble: Option[String], + seed: Option[String], + prompt: String +): String = + val context = List(preamble, seed).flatten.filter(_.nonEmpty).mkString("\n\n") + if context.isEmpty then prompt else s"$context\n\n---\n\n$prompt" diff --git a/flow/src/main/scala/orca/accessors.scala b/flow/src/main/scala/orca/accessors.scala index 24e38d43..65188df5 100644 --- a/flow/src/main/scala/orca/accessors.scala +++ b/flow/src/main/scala/orca/accessors.scala @@ -1,20 +1,69 @@ package orca +import orca.progress.ProgressStore import orca.tools.FsTool import orca.tools.GitTool import orca.tools.GitHubTool -import orca.llm.{ClaudeTool, CodexTool, GeminiTool, OpencodeTool, PiTool} +import orca.agents.{ + Agent, + ClaudeAgent, + CodexAgent, + GeminiAgent, + OpencodeAgent, + PiAgent +} // Top-level accessors that resolve against the ambient FlowContext. // Flow scripts can write `git.checkout("main")` or `claude.ask(...)` // instead of `summon[FlowContext].git.checkout(...)`. -def claude(using ctx: FlowContext): ClaudeTool = ctx.claude -def codex(using ctx: FlowContext): CodexTool = ctx.codex -def opencode(using ctx: FlowContext): OpencodeTool = ctx.opencode -def pi(using ctx: FlowContext): PiTool = ctx.pi -def gemini(using ctx: FlowContext): GeminiTool = ctx.gemini +def claude(using ctx: FlowContext): ClaudeAgent = ctx.claude +def codex(using ctx: FlowContext): CodexAgent = ctx.codex +def opencode(using ctx: FlowContext): OpencodeAgent = ctx.opencode +def pi(using ctx: FlowContext): PiAgent = ctx.pi +def gemini(using ctx: FlowContext): GeminiAgent = ctx.gemini def git(using ctx: FlowContext): GitTool = ctx.git def gh(using ctx: FlowContext): GitHubTool = ctx.gh def fs(using ctx: FlowContext): FsTool = ctx.fs def userPrompt(using ctx: FlowContext): String = ctx.userPrompt + +/** The leading coding agent — the harness chosen by `flow`'s selector + * (`_.claude`, `_.codex`, …). Just another accessor over the ambient + * `FlowContext`, like `claude`/`git`. + * + * Two ways to drive a model in a flow: + * - **`agent`** — the leading agent, backend-agnostic. Use it for the flow's + * planning / implementing / reviewing and its session + * (`agent.session(seed)` → `agent.runSeeded`): switch the selector and the + * whole flow follows. A session threads because `agent` is + * `Agent[ctx.LeadB]` (pinned to the backend), not an erased `Agent[?]`. + * - **a concrete accessor + model** — `claude.opus`, `claude.sonnet`, + * `codex.mini`, `opencode.openaiGpt5Mini`. Use these for a specific + * backend or tier, or for interactive planning (`Plan.interactive` needs a + * concrete backend). The tier accessors (`.opus`/`.sonnet`/…) live on the + * concrete types, NOT on the agnostic `agent`, so `agent.opus` won't + * compile — that's the cue to name the backend. Name the model/tier + * **first**, then any constraints — `claude.opus.withReadOnly`, not + * `claude.withReadOnly.opus` — since only the tier accessors return the + * concrete agent that `.opus`/`.sonnet` hang off. + * + * Don't mix the two for one session: a `SessionId` is backend-typed, so a + * session minted from `claude` won't thread through `agent` once the selector + * is something else. + */ +def agent(using ctx: FlowContext): Agent[ctx.LeadB] = ctx.agent + +/** Build a stable, per-run HTML comment marker for use with + * [[GitHubTool.upsertComment]]. The marker is an HTML comment invisible in the + * rendered GitHub UI but detectable in the raw body, enabling a re-run to find + * and update its own prior comment instead of duplicating it. + * + * `userPrompt` is hashed so two different flow runs for different prompts + * produce distinct markers even with the same `purpose`. `purpose` further + * namespaces the marker within a single run (e.g. `"reject"`, `"triage"`). + * + * Example: `orcaCommentMarker(userPrompt, "reject")` produces a single-line + * marker: `<!-- orca:a1b2c3d4e5f6:reject -->` + */ +def orcaCommentMarker(userPrompt: String, purpose: String): String = + s"<!-- orca:${ProgressStore.hashPrompt(userPrompt)}:$purpose -->" diff --git a/flow/src/main/scala/orca/announceExtension.scala b/flow/src/main/scala/orca/announceExtension.scala index 7e491476..4c0c9d11 100644 --- a/flow/src/main/scala/orca/announceExtension.scala +++ b/flow/src/main/scala/orca/announceExtension.scala @@ -1,7 +1,7 @@ package orca import orca.events.OrcaEvent -import orca.llm.Announce +import orca.agents.Announce /** `value.announce` — manually emit an [[Announce]] message as a `Step`. */ extension [O](value: O)(using a: Announce[O]) diff --git a/flow/src/main/scala/orca/events/CostTracker.scala b/flow/src/main/scala/orca/events/CostTracker.scala index c51f7faa..476e502d 100644 --- a/flow/src/main/scala/orca/events/CostTracker.scala +++ b/flow/src/main/scala/orca/events/CostTracker.scala @@ -1,6 +1,6 @@ package orca.events -import orca.llm.Model +import orca.agents.Model import java.util.concurrent.atomic.AtomicReference @@ -85,7 +85,7 @@ class CostTracker(pricing: PriceList = Pricing.default) extends OrcaListener: def totalCost: Option[Cost] = state.get().byAgentCost.values.reduceOption(_ + _) - /** Per-agent usage breakdown — keyed by `LlmTool.name`. */ + /** Per-agent usage breakdown — keyed by `Agent.name`. */ def perAgent: Map[String, Usage] = state.get().byAgent /** Per-agent cost breakdown. Missing entry means that agent's calls had @@ -94,7 +94,7 @@ class CostTracker(pricing: PriceList = Pricing.default) extends OrcaListener: def perAgentCost: Map[String, Cost] = state.get().byAgentCost /** Per-model usage breakdown. `None` collects calls whose model the backend - * didn't report and the caller didn't pin in `LlmConfig`. + * didn't report and the caller didn't pin in `AgentConfig`. */ def perModel: Map[Option[Model], Usage] = state.get().byModel diff --git a/flow/src/main/scala/orca/events/Pricing.scala b/flow/src/main/scala/orca/events/Pricing.scala index 015ba95e..4a760894 100644 --- a/flow/src/main/scala/orca/events/Pricing.scala +++ b/flow/src/main/scala/orca/events/Pricing.scala @@ -1,6 +1,6 @@ package orca.events -import orca.llm.Model +import orca.agents.Model import java.time.LocalDate diff --git a/flow/src/main/scala/orca/plan/AssessedPlan.scala b/flow/src/main/scala/orca/plan/AssessedPlan.scala index a3b87016..a6701140 100644 --- a/flow/src/main/scala/orca/plan/AssessedPlan.scala +++ b/flow/src/main/scala/orca/plan/AssessedPlan.scala @@ -1,6 +1,6 @@ package orca.plan -import orca.llm.{Announce, JsonData, schemaFromJsonData, codecFromJsonData} +import orca.agents.{Announce, JsonData, schemaFromJsonData, codecFromJsonData} /** Wire shape the LLM produces for an assess-before-plan turn. Flattened * (rather than discriminated union) so jsoniter-scala's structured-output path diff --git a/flow/src/main/scala/orca/plan/BugReportMatch.scala b/flow/src/main/scala/orca/plan/BugReportMatch.scala index cb685ce5..e4ff5039 100644 --- a/flow/src/main/scala/orca/plan/BugReportMatch.scala +++ b/flow/src/main/scala/orca/plan/BugReportMatch.scala @@ -1,6 +1,6 @@ package orca.plan -import orca.llm.{Announce, JsonData} +import orca.agents.{Announce, JsonData} /** The agent's verdict on whether a CI failure (or other reproduction artefact) * actually matches the original bug report. Used after CI comes back red to diff --git a/flow/src/main/scala/orca/plan/BugTriage.scala b/flow/src/main/scala/orca/plan/BugTriage.scala index ce7411ba..e18be2da 100644 --- a/flow/src/main/scala/orca/plan/BugTriage.scala +++ b/flow/src/main/scala/orca/plan/BugTriage.scala @@ -1,6 +1,6 @@ package orca.plan -import orca.llm.JsonData +import orca.agents.JsonData /** Wire shape the LLM produces for a triage turn — a flat record with a boolean * discriminator (`isBug`) plus per-branch fields. Flattened (rather than a diff --git a/flow/src/main/scala/orca/plan/Plan.scala b/flow/src/main/scala/orca/plan/Plan.scala index ed797c01..3917ec32 100644 --- a/flow/src/main/scala/orca/plan/Plan.scala +++ b/flow/src/main/scala/orca/plan/Plan.scala @@ -1,40 +1,16 @@ package orca.plan -import orca.{FlowContext, OrcaFlowException} -import orca.llm.{Announce, BackendTag, CanAskUser, JsonData, LlmTool, given} -import orca.events.OrcaEvent - -/** The shared surface of a development plan, with or without a codebase brief. - * - * Implemented by [[Plan]] (bare) and [[PlanWithBrief]]. The brief is a - * distinct type, not an `Option[String]` field, so it stays out of [[Plan]]'s - * structured-output schema and the planner can't fabricate one. Persistence - * and the task loop work against this trait, so the brief rides in the single - * plan file and is cleaned up with it. - */ -sealed trait PlanLike: - def epicId: String - def description: String - def tasks: List[Task] - - /** First task whose `completed` flag is false, in declaration order. `None` - * means the plan is fully done. - */ - def firstIncomplete: Option[Task] = tasks.find(!_.completed) - - /** Mark the task with the given `title` complete, leaving the others - * untouched. Returns the same plan (same concrete type) if no task matches. - */ - def markComplete(title: Title): PlanLike - - /** Prompt for `task`: its description, with the shared brief (when present) - * prepended. - */ - def taskPrompt(task: Task): String +import orca.{FlowContext, InStage, OrcaFlowException} +import orca.agents.{Announce, BackendTag, CanAskUser, JsonData, Agent, given} /** A development plan: an ordered list of [[Task]]s the agent will work * through, all on a single branch named by `epicId` (kebab-case, used directly - * as the git branch name). + * as the git branch name), plus a `brief` — a concise codebase briefing the + * implementing agents rely on so they don't have to rediscover the layout. + * + * The brief is always present: it is produced as part of the planner's + * structured output, not a separate turn, and feeds the implementer session + * seed (ADR 0018 §2.6). * * ==Planning grid== * @@ -52,78 +28,50 @@ sealed trait PlanLike: * * Every cell returns a [[Sessioned]] — the result plus the agent session that * produced it. From a `Sessioned[B, Plan]` the same session can be continued - * read-only into [[Sessioned.reviewed]] (self-critique) and - * [[Sessioned.briefed]] (codebase brief), or discarded for a fresh implementer - * session. - * - * ==Persistence== - * - * [[recoverOrCreate]] + [[persistComplete]] + [[implementTaskLoop]] round-trip - * a plan through a `## Task: …` markdown file (parsed/rendered by [[parse]] / - * [[render]]) so a flow that crashes mid-loop resumes without re-planning. + * read-only into [[Sessioned.reviewed]] (self-critique), or discarded for a + * fresh implementer session. * * `derives JsonData` so the structured-output path works directly: the helper * methods consume Orca's auto-generated JSON schema; no caller-side - * serialization is needed. + * serialization is needed. As a single case class it is also a valid stage + * result (ADR 0018 §2.3) — the stage log, not a plan file, is what resume + * reads. */ case class Plan( epicId: String, description: String, - tasks: List[Task] -) extends PlanLike derives JsonData: + tasks: List[Task], + brief: String +) derives JsonData: + /** First task whose `completed` flag is false, in declaration order. `None` + * means the plan is fully done. + */ + def firstIncomplete: Option[Task] = tasks.find(!_.completed) + + /** Mark the task with the given `title` complete, leaving the others + * untouched. Returns the same plan if no task matches. + */ def markComplete(title: Title): Plan = copy(tasks = tasks.map(t => if t.title == title then t.markComplete else t)) - def taskPrompt(task: Task): String = task.description - -/** A [[Plan]] paired with a codebase brief for the implementing agents. - * - * Generated by [[Sessioned.briefed]] and prepended to every task via - * [[taskPrompt]]; persists as a trailing `## Brief` section. `derives - * JsonData` so [[Sessioned.reviewed]] can re-emit plan and brief together. - */ -case class PlanWithBrief(plan: Plan, brief: String) extends PlanLike - derives JsonData: - def epicId: String = plan.epicId - def description: String = plan.description - def tasks: List[Task] = plan.tasks - def markComplete(title: Title): PlanWithBrief = - copy(plan = plan.markComplete(title)) + /** Prompt for `task`: its description, with the shared brief prepended when + * present. An empty brief yields the description verbatim — no stray + * separator. + */ def taskPrompt(task: Task): String = - s"$brief\n\n---\n\n${task.description}" + if brief.isEmpty then task.description + else s"$brief\n\n---\n\n${task.description}" object Plan: - /** Subdirectory under `workDir` where persistent plans live. Hidden so plain - * `ls` doesn't show it; convention rather than configuration to keep - * resume-after-crash predictable across flows. - */ - val DefaultDir: os.SubPath = os.sub / ".orca" - - /** Default path for a persistent plan. `<workDir>/.orca/plan-<hash>.md` where - * `<hash>` is the first 12 hex chars of SHA-256(userPrompt). Two unrelated - * prompts in the same repo get different files; rerunning the same prompt - * resumes the same plan. - */ - def defaultPath(userPrompt: String, workDir: os.Path = os.pwd): os.Path = - workDir / DefaultDir / s"plan-${hashUserPrompt(userPrompt)}.md" - - /** First 6 bytes of SHA-256(userPrompt) rendered as 12 hex chars. Visible for - * testing; flow scripts should go through [[defaultPath]]. - */ - private[plan] def hashUserPrompt(userPrompt: String): String = - val md = java.security.MessageDigest.getInstance("SHA-256") - val digest = md.digest(userPrompt.getBytes("UTF-8")) - digest.iterator.take(6).map(b => f"${b & 0xff}%02x").mkString - /** Autonomous planning — a single agentic turn, no human in the loop. The * agent runs `NetworkOnly` (`.withNetworkOnly`): it can verify claims via * Read/Grep and read-only network (issues/PRs/web) but can't edit during the * planning turn (see [[autonomousResult]] for the per-backend guarantee). * Sibling of [[interactive]]; the choice between the two is visible at the * call site (`Plan.autonomous.from(...)` vs `Plan.interactive.from(...)`), - * mirroring `LlmTool`'s own `autonomous` / `interactive` split. + * mirroring `Agent`'s own `autonomous` / `interactive` split. * * Each operation returns a [[Sessioned]]: the read-only planning turn's * session is still resumable by a later writable call, so the caller can @@ -136,10 +84,10 @@ object Plan: /** Produce a [[Plan]] directly from `userPrompt`. */ def from[B <: BackendTag]( userPrompt: String, - llm: LlmTool[B], + agent: Agent[B], instructions: String = PlanPrompts.Planning - )(using FlowContext): Sessioned[B, Plan] = - autonomousResult[B, Plan, Plan](llm, userPrompt, instructions)(identity) + )(using FlowContext, InStage): Sessioned[B, Plan] = + autonomousResult[B, Plan, Plan](agent, userPrompt, instructions)(identity) /** Skeptically assess `userPrompt` (typically a bug/feature report) and * either proceed with a plan or reject with a [[Verdict.Rejection]] the @@ -147,11 +95,11 @@ object Plan: */ def assessThenPlan[B <: BackendTag]( userPrompt: String, - llm: LlmTool[B], + agent: Agent[B], instructions: String = PlanPrompts.AssessThenPlan - )(using FlowContext): Sessioned[B, Verdict[Plan]] = + )(using FlowContext, InStage): Sessioned[B, Verdict[Plan]] = autonomousResult[B, AssessedPlan, Verdict[Plan]]( - llm, + agent, userPrompt, instructions )(a => getOrFail(a.toVerdict)) @@ -161,26 +109,13 @@ object Plan: */ def triage[B <: BackendTag]( report: String, - llm: LlmTool[B], + agent: Agent[B], instructions: String = PlanPrompts.Triage - )(using FlowContext): Sessioned[B, Triage] = - autonomousResult[B, BugTriage, Triage](llm, report, instructions)(b => + )(using FlowContext, InStage): Sessioned[B, Triage] = + autonomousResult[B, BugTriage, Triage](agent, report, instructions)(b => getOrFail(b.toTriage) ) - /** Persistence convenience: parse `file` if it exists (resume), else - * generate via [[from]] and persist. Returns a bare [[PlanLike]] — unlike - * the grid operations above — because the resume branch has no session to - * expose. The implementer mints its own session. - */ - def loadOrGenerate[B <: BackendTag]( - file: os.Path, - userPrompt: String, - llm: LlmTool[B], - instructions: String = PlanPrompts.Planning - )(using FlowContext): PlanLike = - loadOrGenerateImpl(file, () => from(userPrompt, llm, instructions).value) - /** Interactive planning — the LLM call opens a conversation the user can * drive (clarifying questions, refinements) before the agent produces the * result. Not read-only: claude's plan mode would disable the `ask_user` MCP @@ -200,10 +135,12 @@ object Plan: /** Produce a [[Plan]] directly from `userPrompt`. */ def from[B <: BackendTag: CanAskUser]( userPrompt: String, - llm: LlmTool[B], + agent: Agent[B], instructions: String = PlanPrompts.Planning - )(using FlowContext): Sessioned[B, Plan] = - interactiveResult[B, Plan, Plan](llm, userPrompt, instructions)(identity) + )(using FlowContext, InStage): Sessioned[B, Plan] = + interactiveResult[B, Plan, Plan](agent, userPrompt, instructions)( + identity + ) /** Skeptically assess `userPrompt`, but able to ask the reporter clarifying * questions mid-turn rather than only rejecting with a @@ -211,11 +148,11 @@ object Plan: */ def assessThenPlan[B <: BackendTag: CanAskUser]( userPrompt: String, - llm: LlmTool[B], + agent: Agent[B], instructions: String = PlanPrompts.AssessThenPlan - )(using FlowContext): Sessioned[B, Verdict[Plan]] = + )(using FlowContext, InStage): Sessioned[B, Verdict[Plan]] = interactiveResult[B, AssessedPlan, Verdict[Plan]]( - llm, + agent, userPrompt, instructions )(a => getOrFail(a.toVerdict)) @@ -225,25 +162,13 @@ object Plan: */ def triage[B <: BackendTag: CanAskUser]( report: String, - llm: LlmTool[B], + agent: Agent[B], instructions: String = PlanPrompts.Triage - )(using FlowContext): Sessioned[B, Triage] = - interactiveResult[B, BugTriage, Triage](llm, report, instructions)(b => + )(using FlowContext, InStage): Sessioned[B, Triage] = + interactiveResult[B, BugTriage, Triage](agent, report, instructions)(b => getOrFail(b.toTriage) ) - /** Persistence convenience: parse `file` if it exists (resume), else - * generate via [[from]] and persist. Returns a bare [[PlanLike]] — see the - * autonomous counterpart for why no session is exposed. - */ - def loadOrGenerate[B <: BackendTag: CanAskUser]( - file: os.Path, - userPrompt: String, - llm: LlmTool[B], - instructions: String = PlanPrompts.Planning - )(using FlowContext): PlanLike = - loadOrGenerateImpl(file, () => from(userPrompt, llm, instructions).value) - /** Append the operation's instruction block to the caller's input. */ private def withInstructions(input: String, instructions: String): String = s"$input\n\n$instructions" @@ -255,26 +180,6 @@ object Plan: private def getOrFail[A](result: Either[String, A]): A = result.fold(msg => throw OrcaFlowException(msg), identity) - /** Common load-or-generate body: read+log on resume, otherwise call - * `generate` and persist its result. The two public variants differ only in - * which LLM-call shape they pass for `generate` (interactive vs autonomous). - */ - private def loadOrGenerateImpl(file: os.Path, generate: () => PlanLike)(using - ctx: FlowContext - ): PlanLike = - if os.exists(file) then - val parsed = parse(os.read(file)) - ctx.emit( - OrcaEvent.Step( - s"Reusing existing plan at $file (${parsed.tasks.size} task(s), ${parsed.tasks.count(_.completed)} already complete)" - ) - ) - parsed - else - val plan = generate() - os.write.over(file, render(plan), createFolders = true) - plan - /** Run one autonomous turn producing wire type `O`, convert it to the public * result `A`, and pair it with the session. Shared by every `autonomous.*` * operation (`from`, `assessThenPlan`, `triage`). @@ -282,16 +187,16 @@ object Plan: * Runs `NetworkOnly`: reads plus read-only network, so the planner can fetch * an issue/PR it was pointed at and verify external claims. Edits stay * blocked (hard on claude/gemini/opencode; prompt-only on pi/codex — the - * planning prompts forbid edits). Reviewers and the post-planning - * `reviewed`/`briefed` turns use plain `withReadOnly` instead — no network, - * hard no-edit everywhere. + * planning prompts forbid edits). Reviewers and the post-planning `reviewed` + * turn use plain `withReadOnly` instead — no network, hard no-edit + * everywhere. */ private def autonomousResult[B <: BackendTag, O: JsonData: Announce, A]( - llm: LlmTool[B], + agent: Agent[B], input: String, instructions: String - )(convert: O => A)(using FlowContext): Sessioned[B, A] = - val (sessionId, raw) = llm.withNetworkOnly + )(convert: O => A)(using FlowContext, InStage): Sessioned[B, A] = + val (sessionId, raw) = agent.withNetworkOnly .resultAs[O] .autonomous .run(withInstructions(input, instructions)) @@ -303,51 +208,34 @@ object Plan: O: JsonData: Announce, A ]( - llm: LlmTool[B], + agent: Agent[B], input: String, instructions: String - )(convert: O => A)(using FlowContext): Sessioned[B, A] = + )(convert: O => A)(using FlowContext, InStage): Sessioned[B, A] = val (sessionId, raw) = - llm.resultAs[O].interactive.run(withInstructions(input, instructions)) + agent.resultAs[O].interactive.run(withInstructions(input, instructions)) Sessioned(sessionId, convert(raw)) - // == Post-planning steps on a produced plan == + // == Post-planning step on a produced plan == // - // `reviewed` / `briefed` resume the planning session read-only, reusing the - // planner's exploration. Defined here (and in `PlanWithBrief`) so they're in - // the implicit scope of `Sessioned[B, Plan]` — no extra import needed. + // `reviewed` resumes the planning session read-only, reusing the planner's + // exploration. Defined here so it's in the implicit scope of + // `Sessioned[B, Plan]` — no extra import needed. extension [B <: BackendTag](sp: Sessioned[B, Plan]) /** Resume the planning session for a critical self-review, returning the - * improved plan paired with the (same) session. + * improved plan (brief included) paired with the (same) session. */ def reviewed( - llm: LlmTool[B], + agent: Agent[B], instructions: String = PlanPrompts.Review - )(using FlowContext): Sessioned[B, Plan] = - val (sessionId, improved) = llm.withReadOnly + )(using FlowContext, InStage): Sessioned[B, Plan] = + val (sessionId, improved) = agent.withReadOnly .resultAs[Plan] .autonomous .run(s"$instructions\n\n${render(sp.value)}", session = sp.sessionId) Sessioned(sessionId, improved) - /** Resume the planning session for a one-off codebase brief, attached as a - * [[PlanWithBrief]]. Brief last: a later bare-plan [[reviewed]] would drop - * it. - */ - def briefed( - llm: LlmTool[B], - instructions: String = PlanPrompts.Brief - )(using FlowContext): Sessioned[B, PlanWithBrief] = - val (sessionId, brief) = - llm.withReadOnly.autonomous.run(instructions, session = sp.sessionId) - val trimmed = brief.trim - // A blank brief is a planning failure, and would render a `## Brief` - // section that `parse` drops — fail rather than persist a degenerate plan. - if trimmed.isEmpty then - throw OrcaFlowException("brief turn produced an empty brief") - Sessioned(sessionId, PlanWithBrief(sp.value, trimmed)) - /** Empty plans render as nothing — surfacing "0 tasks planned" muddies the * picture; a planning failure is more useful as an explicit `fail(...)` from * the script. @@ -361,219 +249,17 @@ object Plan: val body = plan.tasks.map(t => s" - ${t.title}").mkString("\n") s"$header\n$body" - /** Mark the task with `title` complete in the plan stored at `file`. Reads - * the file, applies the change, writes it back. Use after a task is - * committed so a subsequent run resumes at the next pending task. - */ - def persistComplete(file: os.Path, title: Title): Unit = - val current = parse(os.read(file)) - val updated = current.markComplete(title) - os.write.over(file, render(updated)) - - /** Acquire a persistent plan: resume from `file` if it exists, otherwise - * evaluate `generate` (typically `Plan.autonomous.from(...).value`, or a - * `.reviewed(...).briefed(...).value` chain) and lay down the branch + - * on-disk plan for a fresh run. - * - * `generate` is a bare `=> PlanLike`, not a `Sessioned`: on the resume - * branch there's no planning session to carry, so this helper can't expose - * one consistently. Callers that want session reuse should drive the grid - * operation directly instead of through `recoverOrCreate`; flows here mint a - * fresh implementer session via `llm.newSession` (so `.value` the planning - * result). - * - * `stashMessage` is used when a fresh start finds a dirty tree; pass a - * flow-specific string so `git stash list` is searchable. - */ - def recoverOrCreate( - file: os.Path, - stashMessage: String = "orca: starting work" - )(generate: => PlanLike)(using ctx: FlowContext): PlanLike = - recover(file).getOrElse: - // ensureClean *before* generate so the planner sees a known-clean - // tree (and the "stashed pending changes" Step only fires when the - // user actually had pre-existing dirty edits, not when the planner - // itself wrote files — `Plan.autonomous.from` runs read-only). - val _ = ctx.git.ensureClean(stashMessage) - val plan = generate - ctx.git.checkoutOrCreate(plan.epicId) - os.write.over(file, render(plan), createFolders = true) - plan - - /** Resume from a previously-persisted plan. Returns `Some(plan)` when `file` - * exists, with the working tree cleaned (any pending edits stashed; the user - * can `git stash pop` afterwards) and the working copy attached to - * `plan.epicId`. Returns `None` when no file exists — the caller decides - * whether to generate a fresh plan ([[recoverOrCreate]] does that - * automatically) or treat the absence as a hard failure. - */ - def recover(file: os.Path)(using ctx: FlowContext): Option[PlanLike] = - if !os.exists(file) then None - else - // Snapshot the file before stashing so an untracked plan file (one - // created on disk but never committed — the common crash-before-first- - // task-commit case) survives `ensureClean -u`. After the stash, if the - // file is gone, restore it from the snapshot. If it's still there, - // we trust the post-stash version — a tracked+dirty plan file should - // resume from the committed contents, not the in-progress edits. - val snapshot = os.read(file) - val _ = ctx.git.ensureClean("orca: pre-recovery stash") - val source = - if os.exists(file) then os.read(file) - else - os.write.over(file, snapshot, createFolders = true) - snapshot - val plan = parse(source) - ctx.git.checkoutOrCreate(plan.epicId) - ctx.emit( - OrcaEvent.Step( - s"Recovered plan at $file (${plan.tasks.size} task(s), ${plan.tasks.count(_.completed)} already complete)" - ) - ) - Some(plan) - - /** Per-task implementation loop with on-disk progress + commits. - * - * For each incomplete task in `plan`: - * - * 1. Calls `body(task)` — the caller's implement-and-review work. 2. Ticks - * the task's `Status: [x]` in `file`. 3. Makes one `task: <title>` git - * commit covering both the body's changes and the checkbox tick. - * - * After the last task: removes `file` and makes a `chore: remove - * <file.last>` cleanup commit (skipped if the file was never tracked). - * Bodies that throw abort the loop with the partial plan still on disk, so a - * subsequent run resumes at the first incomplete task. - * - * Sibling of [[orca.review.reviewAndFixLoop]] — that one drives the - * review-and-fix iteration within a task; this one drives task-by-task - * progress across the whole plan. - */ - def implementTaskLoop(file: os.Path, plan: PlanLike)(body: Task => Unit)(using - ctx: FlowContext - ): Unit = - runTaskLoop( - initial = plan, - advance = (_, t) => - persistComplete(file, t.title) - val next = parse(os.read(file)) - // Defense in depth: persistComplete + a clean re-read should always - // advance past the just-processed title. If it didn't, the on-disk plan - // diverged from what we just wrote — surface it instead of spinning. - if next.firstIncomplete.map(_.title).contains(t.title) then - throw OrcaFlowException( - s"implementTaskLoop: task '${t.title.value}' is still the first incomplete entry " + - s"after persistComplete. The plan file at $file may have been " + - "edited so the title still matches an unchecked task." - ) - next - , - // Cleanup commit only fires if there's actually a file to remove — - // skipping the no-op branch avoids a wasted `git add -A` subprocess and - // a misleading "chore: remove …" commit-message intent when the file - // never existed (e.g. caller pre-removed it). - cleanup = () => - if os.exists(file) then - val _ = os.remove(file) - // `NothingToCommit` swallowed so a `.gitignore`d plan dir (untracked - // file → no diff after removal) doesn't crash the whole run. - val _ = ctx.git.commit(s"chore: remove ${file.last}") - )(body) - - /** In-memory variant of [[implementTaskLoop]] for flows that aren't - * resumable: same per-task `task: <title>` commit cadence, but completion is - * tracked in memory (no plan file, no chore-remove commit). Use when the - * surrounding flow has its own non-restartable state machine and a - * `.orca/plan-*.md` file would just be dead weight (e.g. a bugfix flow whose - * earlier stages — triage, CI red, repro verification — can't be resumed - * from a plan-file alone). - */ - def implementTaskLoop(plan: PlanLike)(body: Task => Unit)(using - FlowContext - ): Unit = - runTaskLoop( - initial = plan, - advance = (cur, t) => cur.markComplete(t.title), - cleanup = () => () - )(body) - - private def runTaskLoop( - initial: PlanLike, - advance: (PlanLike, Task) => PlanLike, - cleanup: () => Unit - )(body: Task => Unit)(using ctx: FlowContext): Unit = - @scala.annotation.tailrec - def loop(current: PlanLike): Unit = current.firstIncomplete match - case None => () - case Some(t) => - body(t) - val next = advance(current, t) - // `NothingToCommit` is non-fatal here: a body that produced only - // gitignored output (the plan-file tick when `.orca/` is in - // `.gitignore`, or a no-op task by design) shouldn't abort the - // loop and leave the next run skipping a task on the strength of - // an on-disk tick alone. Same swallow the cleanup commit already - // does. Surface the swallow as a Step so a body that silently - // produced nothing (e.g. an LLM turn that ended without edits) - // is still visible in the event log. - ctx.git.commit(s"task: ${t.title}") match - case Right(_) => () - case Left(_) => - ctx.emit( - OrcaEvent.Step( - s"task '${t.title.value}' produced no tracked changes — advancing without commit" - ) - ) - loop(next) - loop(initial) - cleanup() - - /** Parse a plan from its markdown representation, auto-detecting a trailing - * `## Brief` section: present → [[PlanWithBrief]], absent → [[Plan]]. Strict - * on the plan part — throws [[PlanParseException]] on any deviation from the - * `# Plan:` / `## Task:` schema. CRLF line endings and a leading BOM are - * normalised first. - */ - def parse(markdown: String): PlanLike = - val normalised = markdown.stripPrefix("\uFEFF").replace("\r\n", "\n") - val (planPart, brief) = splitBrief(normalised) - val plan = parsePlan(planPart) - brief.fold[PlanLike](plan)(b => PlanWithBrief(plan, b)) - - /** Render a plan back into the on-disk format. Output round-trips through - * [[parse]] without information loss; we use this to write the file when - * first generated and again when a task's status flips. A [[PlanWithBrief]] - * appends its brief as a trailing `## Brief` section. + /** Render a plan to markdown (tasks as a `[ ]`/`[x]` checklist, the brief as + * a trailing `## Brief` section). Used by [[Sessioned.reviewed]] to feed the + * plan back into the self-review prompt, and equally usable as a + * human-readable checklist. It is **never parsed back**: the stage log is + * the sole resume mechanism (ADR 0018 §2.8), so there is no inverse parser + * to keep in sync. */ - def render(plan: PlanLike): String = plan match - case p: Plan => renderPlan(p) - case PlanWithBrief(p, brief) => - s"${renderPlan(p)}\n## Brief\n\n${brief.stripLineEnd}\n" - - // --- Brief section (rendered last, parsed first) --- - - private val BriefHeaderPattern = "^##\\s+Brief\\s*$".r - - /** Split a normalised document into its plan part and optional brief. The - * brief is the text after the first `## Brief` heading that follows the - * tasks — searching only past the first `## Task:` stops a stray `## Brief` - * in the description from swallowing them, while still letting the brief - * body carry its own `##` headings. - */ - private def splitBrief(normalised: String): (String, Option[String]) = - val lines = normalised.linesIterator.toList - val afterFirstTask = lines.indexWhere(TaskHeaderPattern.matches) + 1 - lines.indexWhere(BriefHeaderPattern.matches, afterFirstTask) match - case -1 => (normalised, None) - case i => - // Drop the blank line `render` writes after the heading; keep the rest - // verbatim (a brief may be indented). - val brief = - lines.drop(i + 1).dropWhile(_.isEmpty).mkString("\n").stripLineEnd - ( - lines.take(i).mkString("\n"), - if brief.isEmpty then None else Some(brief) - ) + def render(plan: Plan): String = + val base = renderPlan(plan) + if plan.brief.trim.isEmpty then base + else s"$base\n## Brief\n\n${plan.brief.stripLineEnd}\n" private def renderPlan(plan: Plan): String = val header = s"# Plan: ${plan.epicId}\n" @@ -586,102 +272,3 @@ object Plan: s"\n## Task: ${t.title}\nStatus: $checkbox\n\n${t.description.stripLineEnd}\n" .mkString header + descriptionBlock + body - - // --- Parser internals --- - - private val HeaderPattern = "^# Plan:\\s*(\\S.*)$".r - private val TaskHeaderPattern = "^## Task:\\s*(\\S.*)$".r - private val StatusPattern = "^Status:\\s*\\[(.)\\]\\s*$".r - - private def parsePlan(planMarkdown: String): Plan = - val lines = planMarkdown.linesIterator.toList - val epicId = parseHeader(lines) - val description = parseDescription(lines) - val taskBlocks = splitTaskBlocks(lines) - if taskBlocks.isEmpty then throw PlanParseException("Plan has no tasks") - Plan(epicId, description, taskBlocks.map(parseTask)) - - private def parseHeader(lines: List[String]): String = - lines.find(_.trim.nonEmpty) match - case Some(HeaderPattern(id)) => id.trim - case other => - throw PlanParseException( - s"Expected first non-blank line to match `# Plan: <epicId>`; got: ${other.getOrElse("(empty file)")}" - ) - - /** Description sits between the `# Plan:` header and the first `## Task:` - * heading. Empty when the file goes straight from the header into tasks. - */ - private def parseDescription(lines: List[String]): String = - val afterHeader = lines.dropWhile(l => !HeaderPattern.matches(l)).drop(1) - afterHeader - .takeWhile(l => !TaskHeaderPattern.matches(l)) - .mkString("\n") - .trim - - private def splitTaskBlocks(lines: List[String]): List[List[String]] = - val blocks = collection.mutable.ListBuffer[List[String]]() - var current = collection.mutable.ListBuffer[String]() - var inTask = false - for line <- lines do - if TaskHeaderPattern.matches(line) then - if inTask then blocks += current.toList - current = collection.mutable.ListBuffer(line) - inTask = true - else if inTask then current += line - if inTask then blocks += current.toList - blocks.toList - - private def parseTask(block: List[String]): Task = - val title = block.headOption match - case Some(TaskHeaderPattern(t)) => t.trim - case _ => - throw PlanParseException( - s"Task block doesn't start with `## Task: <title>`: ${block.headOption.getOrElse("")}" - ) - val rest = block.tail.dropWhile(_.trim.isEmpty) - val (statusLine, afterStatus) = rest.headOption match - case Some(line @ StatusPattern(_)) => (line, rest.tail) - case _ => - throw PlanParseException( - s"Task '$title' is missing a `Status: [ ]` / `Status: [x]` line" - ) - val completed = statusLine match - case StatusPattern(" ") => false - case StatusPattern("x") => true - case StatusPattern(other) => - throw PlanParseException( - s"Task '$title' has unrecognised status checkbox '$other'" - ) - val description = afterStatus.mkString("\n").trim - if description.isEmpty then - throw PlanParseException(s"Task '$title' has no prompt body") - Task(title = Title(title), description = description, completed = completed) - -object PlanWithBrief: - - /** Reuse [[Plan]]'s announce — the brief is internal context, not worth - * surfacing in the event log on its own. - */ - given Announce[PlanWithBrief] = Announce.from: pwb => - summon[Announce[Plan]].message(pwb.plan).getOrElse("") - - extension [B <: BackendTag](sp: Sessioned[B, PlanWithBrief]) - /** Resume the planning session to review the plan *and* its brief together, - * returning the improved [[PlanWithBrief]]. The counterpart to - * [[Sessioned.reviewed]] on a bare plan, reachable as `…briefed.reviewed`. - */ - def reviewed( - llm: LlmTool[B], - instructions: String = PlanPrompts.Review - )(using FlowContext): Sessioned[B, PlanWithBrief] = - val (sessionId, improved) = llm.withReadOnly - .resultAs[PlanWithBrief] - .autonomous - .run( - s"$instructions\n\n${Plan.render(sp.value)}", - session = sp.sessionId - ) - Sessioned(sessionId, improved) - -class PlanParseException(message: String) extends RuntimeException(message) diff --git a/flow/src/main/scala/orca/plan/PlanPrompts.scala b/flow/src/main/scala/orca/plan/PlanPrompts.scala index 7a153942..bb399721 100644 --- a/flow/src/main/scala/orca/plan/PlanPrompts.scala +++ b/flow/src/main/scala/orca/plan/PlanPrompts.scala @@ -19,6 +19,8 @@ object PlanPrompts: /** Used by `Plan.interactive.*` and `Plan.autonomous.*` to brief the planner * agent. Without the opening clause agents tend to start editing files * during the planning turn — the implementation is the implementer's job. + * Also asks the planner to fill the `brief` field of its structured output + * with a codebase briefing for the implementing agents. */ val Planning: String = PromptResource.load("/orca/plan/prompts/planning.md") @@ -38,15 +40,8 @@ object PlanPrompts: val Triage: String = PromptResource.load("/orca/plan/prompts/triage.md") - /** Used by `Sessioned[B, Plan].reviewed` / `Sessioned[B, PlanWithBrief] - * .reviewed`. The current plan is appended after this block; the agent - * returns an improved plan (and refines the brief when one is present). + /** Used by `Sessioned[B, Plan].reviewed`. The current plan is appended after + * this block; the agent returns an improved plan, brief included. */ val Review: String = PromptResource.load("/orca/plan/prompts/review.md") - - /** Used by `Sessioned[B, Plan].briefed`. Asks the planner to write a codebase - * briefing for the implementing agents, without restating the plan. - */ - val Brief: String = - PromptResource.load("/orca/plan/prompts/brief.md") diff --git a/flow/src/main/scala/orca/plan/Sessioned.scala b/flow/src/main/scala/orca/plan/Sessioned.scala index d9ac43c9..595be187 100644 --- a/flow/src/main/scala/orca/plan/Sessioned.scala +++ b/flow/src/main/scala/orca/plan/Sessioned.scala @@ -1,6 +1,6 @@ package orca.plan -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} /** A planning-phase result paired with the agent session that produced it. * @@ -8,12 +8,12 @@ import orca.llm.{BackendTag, SessionId} * these so the caller can choose to continue the same conversation for the * downstream implementation phase (the agent keeps the context it built up * while planning / assessing / triaging) or discard the session and mint a - * fresh one via `llm.newSession`. + * resumable one via `agent.session(seed)`. * * Autonomous planning runs read-only, but the returned `sessionId` is still - * resumable: a later writable `llm.autonomous.run(task, sessionId)` reuses the - * same conversation thread with write access restored — read-only applied only - * to the planning turn, not to the thread. + * resumable: a later writable `agent.autonomous.run(task, sessionId)` reuses + * the same conversation thread with write access restored — read-only applied + * only to the planning turn, not to the thread. * * Destructure at the call site: * diff --git a/flow/src/main/scala/orca/plan/Task.scala b/flow/src/main/scala/orca/plan/Task.scala index 2384de02..ef91fe2a 100644 --- a/flow/src/main/scala/orca/plan/Task.scala +++ b/flow/src/main/scala/orca/plan/Task.scala @@ -1,17 +1,16 @@ package orca.plan import orca.plan.Title -import orca.llm.{JsonData} +import orca.agents.{JsonData} -/** A single task in a [[Plan]]. The same type covers both the in-memory - * (`Plan.from`) and markdown-backed (`Plan.loadOrGenerate`) shapes. +/** A single task in a [[Plan]]. * * - `title` is the one-line human-readable label rendered in the event log * and used as the `## Task: …` markdown section header. * - `description` is the longer instruction handed to the implementing * agent. - * - `completed` is the resume-state checkbox used by extended plans. Simple - * plans run end-to-end and leave it false. + * - `completed` is a checkbox surfaced in the cosmetic rendered checklist + * (ADR 0018 §2.8); resume reads the stage log, not this flag. */ case class Task( title: Title, diff --git a/flow/src/main/scala/orca/plan/Triage.scala b/flow/src/main/scala/orca/plan/Triage.scala index 5d8efdc5..f48c5a2a 100644 --- a/flow/src/main/scala/orca/plan/Triage.scala +++ b/flow/src/main/scala/orca/plan/Triage.scala @@ -1,6 +1,6 @@ package orca.plan -import orca.llm.Announce +import orca.agents.{Announce, JsonData} /** Outcome of triaging a bug report against a codebase. Three variants: * @@ -18,12 +18,22 @@ import orca.llm.Announce * with runtime `Option#get` / empty-string checks. * * Produced by [[Plan.autonomous.triage]] / [[Plan.interactive.triage]], both - * wrapped in a [[Sessioned]] so the triage agent's session can carry into the - * fix. + * wrapped in a [[Sessioned]] that carries the triage session id. Flows + * typically discard the triage session (calling `.value`) and start a FRESH + * implementer session seeded with the issue body (`agent.session(seed = + * issue.body)`) rather than continuing the triage session — so the `Sessioned` + * wrapper is available but no carry-over is guaranteed. + * + * `derives JsonData` so a `stage` can record and replay a `Triage` result — + * the triage stage is a checkpoint before the failing-test / fix pipeline (ADR + * 0018 §3.2). */ -enum Triage: +enum Triage derives JsonData: case NotABug(explanation: String) case Untestable(summary: String, reproductionSteps: String) + // `branchName` is the LLM-suggested name from the wire format. The actual + // feature branch is created by the flow runtime (via BranchNamingStrategy), + // not from this field — flows should wildcard it in pattern matches. case Testable(summary: String, branchName: String, failingTestPath: String) object Triage: diff --git a/flow/src/main/scala/orca/pr/summarisePr.scala b/flow/src/main/scala/orca/pr/summarisePr.scala index aa08f2b1..6b68c201 100644 --- a/flow/src/main/scala/orca/pr/summarisePr.scala +++ b/flow/src/main/scala/orca/pr/summarisePr.scala @@ -1,7 +1,7 @@ package orca.pr -import orca.FlowContext -import orca.llm.{Announce, JsonData, LlmTool} +import orca.{FlowContext, InStage} +import orca.agents.{Announce, JsonData, Agent} /** What [[summarisePr]] produces: a one-line PR title and a multi-paragraph * body. `gh.createPr(title = …, body = …)` accepts the two fields directly, so @@ -15,7 +15,7 @@ object PrSummary: */ given Announce[PrSummary] = Announce.from(_ => "") -/** Ask `llm` to fold `diff` (and optionally `context`) into a [[PrSummary]] — +/** Ask `agent` to fold `diff` (and optionally `context`) into a [[PrSummary]] — * the one-line title and multi-paragraph body that `gh.createPr` consumes. * * `context` is rendered above the diff as a preamble; typical contents are the @@ -28,11 +28,11 @@ object PrSummary: * diff dominates the prompt and would dwarf the event log. */ def summarisePr( - llm: LlmTool[?], + agent: Agent[?], diff: String, context: Option[String] = None, instructions: String = PrPrompts.Summarise -)(using FlowContext): PrSummary = +)(using FlowContext, InStage): PrSummary = val contextBlock = context.fold("")(c => s"$c\n\n") val prompt = s"""$instructions @@ -42,4 +42,4 @@ def summarisePr( |```diff |$diff |```""".stripMargin - llm.resultAs[PrSummary].autonomous.run(prompt, emitPrompt = false)._2 + agent.resultAs[PrSummary].autonomous.run(prompt, emitPrompt = false)._2 diff --git a/flow/src/main/scala/orca/progress/ProgressLog.scala b/flow/src/main/scala/orca/progress/ProgressLog.scala new file mode 100644 index 00000000..1008c76c --- /dev/null +++ b/flow/src/main/scala/orca/progress/ProgressLog.scala @@ -0,0 +1,89 @@ +package orca.progress + +import com.github.plokhotnyuk.jsoniter_scala.macros.{ + CodecMakerConfig, + ConfiguredJsonValueCodec +} +import orca.agents.{JsonData, given} +import sttp.tapir.Schema + +/** Header capturing the git context in which the progress log was started. */ +case class ProgressHeader( + startingBranch: String, + branch: String, + promptHash: String +) derives JsonData + +/** A single stage's outcome, stored as an already-serialised JSON string. + * + * `resultJson` is type-erased at rest — the log is heterogeneous across stage + * types. Deserialisation back to a typed value happens at the stage call site + * (C3), not here. + */ +case class StageEntry(id: String, name: String, resultJson: String) + derives JsonData + +/** A persisted session: the occurrence index, a minted UUID, the seed string + * the author supplied, and — when the session is durably resumable — the wire + * id to resume against. + * + * `id` is the stable client id the framework hands across calls; + * [[SessionRecord.resumeWireId]] is the id to put on the wire when resuming + * the live backend conversation (the same `wireId` notion as + * [[orca.backend.Dispatch]]). Its value depends on the backend, but its + * meaning is uniform: + * - codex/gemini/opencode: a backend-minted server-thread id (≠ `id`); + * - claude: equal to `id` itself — claude's sessions are durable on disk and + * its client id IS the wire id, so recording it re-claims the session + * (`--resume`) on a resumed run rather than re-creating it; + * - pi: `None` — pi's sessions live in a `deleteOnExit` temp dir, so nothing + * survives a restart and a resumed run always re-seeds. + * + * It is persisted so a resumed run can rehydrate the in-memory map and (a) + * resume against the right wire id and (b) probe it for existence. + * + * `resumeWireId` defaults to `None` and decodes to `None` when absent in older + * log files — the lenient [[ProgressLog]] codec tolerates the missing field. + * Stored in [[ProgressLog.sessions]] so that a resumed run reuses the same + * [[orca.agents.SessionId]] rather than minting a second one. + */ +case class SessionRecord( + index: Int, + id: String, + seed: String, + resumeWireId: Option[String] = None +) derives JsonData + +/** An append-only log of stage outcomes and session records for one flow run, + * keyed by its header. + * + * `sessions` defaults to `Nil` so that existing log files written before this + * field was introduced continue to decode. The custom [[JsonData]] instance + * below uses a lenient codec config that tolerates missing collection fields. + */ +case class ProgressLog( + header: ProgressHeader, + entries: List[StageEntry], + sessions: List[SessionRecord] = Nil +) + +object ProgressLog: + /** Custom `JsonData` instance for `ProgressLog` that does **not** require the + * `sessions` collection field to be present in the JSON. All other + * collection fields (`entries`) are similarly lenient here — `entries` is + * never absent in practice, but a uniform config keeps the rule simple. + * + * This intentionally diverges from `JsonData.strictCodecConfig`, which sets + * `withRequireCollectionFields(true)`. That strictness is appropriate for + * LLM-reply DTOs (where a missing list is almost certainly a model error), + * but wrong for the progress log, which must round-trip across software + * versions where new optional fields are added over time. + */ + given JsonData[ProgressLog] = JsonData( + Schema.derived[ProgressLog], + ConfiguredJsonValueCodec.derived[ProgressLog](using + CodecMakerConfig + .withRequireCollectionFields(false) + .withTransientEmpty(false) + ) + ) diff --git a/flow/src/main/scala/orca/progress/ProgressStore.scala b/flow/src/main/scala/orca/progress/ProgressStore.scala new file mode 100644 index 00000000..a98bba34 --- /dev/null +++ b/flow/src/main/scala/orca/progress/ProgressStore.scala @@ -0,0 +1,119 @@ +package orca.progress + +import com.github.plokhotnyuk.jsoniter_scala.core.{ + readFromString, + writeToString +} +import orca.{InStage} +import orca.agents.JsonData +import scala.util.control.NonFatal + +/** Persistent store for a single flow run's [[ProgressLog]]. + * + * Mutations are gated on [[InStage]] to mark them as working-tree-mutating + * operations (ADR 0018 §2.2). The token is not inspected at runtime — the type + * is the guard. + */ +trait ProgressStore: + /** The on-disk path of this store's JSON file. The stage commit force-adds + * this single path so the log is committed even when `.orca/` is gitignored. + */ + def path: os.Path + + def load(): Option[ProgressLog] + def writeHeader(header: ProgressHeader)(using InStage): Unit + + /** Upsert an entry by id: replaces an existing entry with the same id + * in-place, or appends if no entry with that id exists. Last write wins. + * + * Requires [[writeHeader]] to have been called first (a log must already + * exist); otherwise it throws. + */ + def appendEntry(entry: StageEntry)(using InStage): Unit + + /** Upsert a session record by [[SessionRecord.index]]: replaces an existing + * record at that index, or appends if none exists. Last write wins. + * + * Requires [[writeHeader]] to have been called first; otherwise it throws. + * Does NOT commit — the session is recorded in the log file only; the next + * stage commit will force-add the log and carry it. + */ + def upsertSession(record: SessionRecord)(using InStage): Unit + +object ProgressStore: + + /** Default OS-backed store: JSON at `<workDir>/.orca/progress-<hash>.json`. + * + * `hash` is the first 12 hex chars of SHA-256(userPrompt). Two unrelated + * prompts in the same repo produce different files; rerunning the same + * prompt resumes the same log. + */ + def default(workDir: os.Path, userPrompt: String): ProgressStore = + OsProgressStore( + workDir / os.sub / ".orca" / s"progress-${hashPrompt(userPrompt)}.json" + ) + + /** First 6 bytes of SHA-256(userPrompt) rendered as 12 hex chars. + * Package-private so the flow lifecycle can stamp the same hash into the + * progress header (ADR 0018 §2.4). + */ + private[orca] def hashPrompt(userPrompt: String): String = + val md = java.security.MessageDigest.getInstance("SHA-256") + val digest = md.digest(userPrompt.getBytes("UTF-8")) + digest.iterator.take(6).map(b => f"${b & 0xff}%02x").mkString + +private class OsProgressStore(val path: os.Path) extends ProgressStore: + + private val codec = summon[JsonData[ProgressLog]].codec + + def load(): Option[ProgressLog] = + if !os.exists(path) then None + else parseLog(os.read(path)) + + def writeHeader(header: ProgressHeader)(using InStage): Unit = + writeLog(ProgressLog(header, Nil)) + + def appendEntry(entry: StageEntry)(using InStage): Unit = + val current = load().getOrElse( + throw IllegalStateException( + s"appendEntry called before writeHeader: no log at $path" + ) + ) + writeLog(upsertEntry(current, entry)) + + def upsertSession(record: SessionRecord)(using InStage): Unit = + val current = load().getOrElse( + throw IllegalStateException( + s"upsertSession called before writeHeader: no log at $path" + ) + ) + writeLog(upsertSessionRecord(current, record)) + + private def upsertEntry(log: ProgressLog, entry: StageEntry): ProgressLog = + val idx = log.entries.indexWhere(_.id == entry.id) + val updated = + if idx >= 0 then log.entries.updated(idx, entry) + else log.entries :+ entry + log.copy(entries = updated) + + private def upsertSessionRecord( + log: ProgressLog, + record: SessionRecord + ): ProgressLog = + val idx = log.sessions.indexWhere(_.index == record.index) + val updated = + if idx >= 0 then log.sessions.updated(idx, record) + else log.sessions :+ record + log.copy(sessions = updated) + + // We rewrite the whole file each time rather than append JSONL: the log is a + // single structured *document* (a header + entries + sessions object), not a + // flat event stream — `upsertEntry`/`upsertSession` mutate existing elements, + // which an append-only log can't express. It's also small and bounded (a + // handful of stages + sessions per run), so a full rewrite is negligible. + private def writeLog(log: ProgressLog): Unit = + os.write.over(path, writeToString(log)(using codec), createFolders = true) + + private def parseLog(json: String): Option[ProgressLog] = + try Some(readFromString[ProgressLog](json)(using codec)) + catch case NonFatal(_) => None diff --git a/flow/src/main/scala/orca/progress/RecoveryCheck.scala b/flow/src/main/scala/orca/progress/RecoveryCheck.scala new file mode 100644 index 00000000..e42b20ac --- /dev/null +++ b/flow/src/main/scala/orca/progress/RecoveryCheck.scala @@ -0,0 +1,65 @@ +package orca.progress + +/** Validation of an untrusted [[ProgressHeader]] read back on resume (ADR 0018 + * §2.4/§2.5). + * + * The progress log is human-visible and pushable, so its header is untrusted + * input on load: it may have been hand-edited or carried onto the wrong branch + * by a merge. Before any destructive git action (checkout, reset --hard, + * delete) the runtime validates it here. A failure is a hard signal — the + * caller aborts the run rather than silently proceeding or starting fresh. + */ +object RecoveryCheck: + + /** Whether `s` is a safe git ref for orca's purposes: non-empty, and every + * `/`-separated segment matches the slug shape `^[a-z0-9][a-z0-9-]*$` (the + * same charset [[orca.BranchNamingStrategy.slug]] produces). Issue branches + * like `fix/issue-42` pass; ``, `-x`, `a/..`, `a b`, `Feat` are rejected. + * + * Forcing a leading alphanumeric blocks a name that begins with `-` (which + * `git`/`gh` would read as a CLI flag) and bans path-traversal/whitespace + * segments. + */ + def isSafeBranchRef(s: String): Boolean = + // A safe ref is a `/`-separated path of slug segments — the exact shape + // `BranchNamingStrategy.slug` produces. Referencing its predicate (rather + // than re-deriving the charset here) keeps the producer and this validator + // from drifting apart. + s.nonEmpty && s + .split("/", -1) + .forall(orca.BranchNamingStrategy.isSlugSegment) + + /** Branches that are always protected regardless of the repo's configured + * default — the floor [[validateHeader]] enforces (ADR 0018). The runtime + * adds the repo's actual default branch on top of these. + */ + val alwaysProtected: Set[String] = Set("main", "master") + + /** Validate the header before any destructive action. Returns `Left(reason)` + * on the first failure, `Right(())` when the header is trustworthy. + * + * - `branch` and `startingBranch` must be safe refs. + * - `branch` must not be a protected branch (`startingBranch` may be): + * `protectedBranches` (lower-cased) plus the `main`/`master` floor. + * - `promptHash` must equal the recomputed hash of the current prompt. + * + * `protectedBranches` lets the runtime pass the repo's ACTUAL default branch + * (e.g. `trunk`/`develop`), so a tampered header naming it as a feature + * branch is refused — not just `main`/`master`. Compared case-insensitively. + */ + def validateHeader( + header: ProgressHeader, + userPrompt: String, + protectedBranches: Set[String] + ): Either[String, Unit] = + val protectedLower = + (protectedBranches ++ alwaysProtected).map(_.toLowerCase) + if !isSafeBranchRef(header.branch) then + Left(s"branch '${header.branch}' is not a safe ref") + else if !isSafeBranchRef(header.startingBranch) then + Left(s"startingBranch '${header.startingBranch}' is not a safe ref") + else if protectedLower.contains(header.branch.toLowerCase) then + Left(s"branch '${header.branch}' is a protected branch") + else if header.promptHash != ProgressStore.hashPrompt(userPrompt) then + Left("promptHash does not match the current prompt") + else Right(()) diff --git a/flow/src/main/scala/orca/review/FixOutcome.scala b/flow/src/main/scala/orca/review/FixOutcome.scala index decd2b43..1c297c4d 100644 --- a/flow/src/main/scala/orca/review/FixOutcome.scala +++ b/flow/src/main/scala/orca/review/FixOutcome.scala @@ -1,6 +1,6 @@ package orca.review -import orca.llm.{JsonData, given} +import orca.agents.{JsonData, given} import orca.plan.Title /** What the fixing agent reports back per iteration: the titles of issues it diff --git a/flow/src/main/scala/orca/review/IgnoredIssue.scala b/flow/src/main/scala/orca/review/IgnoredIssue.scala index 7bae9b27..f6f8b8b7 100644 --- a/flow/src/main/scala/orca/review/IgnoredIssue.scala +++ b/flow/src/main/scala/orca/review/IgnoredIssue.scala @@ -1,6 +1,6 @@ package orca.review -import orca.llm.{Announce, JsonData, given} +import orca.agents.{Announce, JsonData, given} import orca.plan.Title case class IgnoredIssue(title: Title, reason: String) derives JsonData diff --git a/flow/src/main/scala/orca/review/ReviewContext.scala b/flow/src/main/scala/orca/review/ReviewContext.scala index afe7d9d0..49448652 100644 --- a/flow/src/main/scala/orca/review/ReviewContext.scala +++ b/flow/src/main/scala/orca/review/ReviewContext.scala @@ -1,6 +1,6 @@ package orca.review -import orca.llm.JsonData +import orca.agents.JsonData case class ReviewContext(summary: String, filesChanged: List[String]) derives JsonData diff --git a/flow/src/main/scala/orca/review/ReviewIssue.scala b/flow/src/main/scala/orca/review/ReviewIssue.scala index ac23e8e4..cd96231e 100644 --- a/flow/src/main/scala/orca/review/ReviewIssue.scala +++ b/flow/src/main/scala/orca/review/ReviewIssue.scala @@ -1,6 +1,6 @@ package orca.review -import orca.llm.JsonData +import orca.agents.JsonData import orca.plan.Title /** A single review finding. `title` is the one-line user-facing label (rendered diff --git a/flow/src/main/scala/orca/review/ReviewLoop.scala b/flow/src/main/scala/orca/review/ReviewLoop.scala index 57db906f..92c3cdf0 100644 --- a/flow/src/main/scala/orca/review/ReviewLoop.scala +++ b/flow/src/main/scala/orca/review/ReviewLoop.scala @@ -1,13 +1,13 @@ package orca.review -import orca.{FlowContext} +import orca.{FlowContext, InStage} import orca.plan.Title -import orca.llm.{ +import orca.agents.{ AgentInput, BackendTag, JsonData, - LlmConfig, - LlmTool, + AgentConfig, + Agent, SessionId, given } @@ -22,52 +22,89 @@ import ox.flow.Flow * cap is hit are folded into the returned `IgnoredIssues` with a `max * iterations reached` reason so callers can surface them. * - * Each call to `evaluate` runs inside a nested `Iteration N` stage. + * Each round emits an `Iteration N` progress marker (a `display`, not a + * committing stage — it runs under the caller's task stage, ADR 0018 §2.2). */ def fixLoop( evaluate: () => ReviewResult, fix: List[ReviewIssue] => FixOutcome, maxIterations: Int = 10 )(using ctx: FlowContext): IgnoredIssues = + // The stateless entry point: a caller with no cross-iteration state delegates + // to the threaded driver with `Unit` state. + fixLoopWithState[Unit]( + initial = (), + evaluate = _ => (evaluate(), ()), + fix = fix, + maxIterations = maxIterations + ) + +/** The iteration driver behind [[fixLoop]], generalised to thread a + * caller-supplied state `S` explicitly through each round instead of letting + * `evaluate` mutate a captured `var`. `evaluate` receives the state from the + * previous round and returns its result together with the next state; the + * driver feeds that forward. [[reviewAndFixLoop]] threads its + * [[ReviewLoopState]] (reviewer history + sessions) this way, so the + * cross-iteration data flow is visible in the type rather than hidden in a + * closure. + * + * Like [[fixLoop]], this calls no gated method itself — the gated work lives + * in the `evaluate`/`fix` thunks, which close over their own `InStage`. + */ +private def fixLoopWithState[S]( + initial: S, + evaluate: S => (ReviewResult, S), + fix: List[ReviewIssue] => FixOutcome, + maxIterations: Int +)(using ctx: FlowContext): IgnoredIssues = def emitStep(msg: String): Unit = ctx.emit(OrcaEvent.Step(msg)) - // Either keep going (and accumulate ignored entries) or stop. The Stop - // variant carries any final additions to the accumulator. + // Either keep going or stop, carrying any additions to the accumulator. Only + // `Continue` carries the next state — on a stop the loop ends, so the state + // produced by the final round is irrelevant. enum Step: - case Continue(addIgnored: IgnoredIssues) + case Continue(addIgnored: IgnoredIssues, next: S) case Stop(addIgnored: IgnoredIssues) - def runIteration(iteration: Int): Step = - orca.stage(s"Iteration ${iteration + 1}"): - val issues = evaluate().issues - if issues.isEmpty then - emitStep("No review comments") - Step.Stop(IgnoredIssues(Nil)) - else if iteration >= maxIterations then - emitStep(s"Reached max iterations ($maxIterations); bailing out") - Step.Stop( - IgnoredIssues( - issues.map(i => - IgnoredIssue(i.title, s"max iterations ($maxIterations) reached") - ) + def runIteration(iteration: Int, state: S): Step = + // A progress marker, not a committing stage: this runs under the caller's + // task stage (ADR 0018 §2.2), so it must not open its own stage. + orca.display(s"Iteration ${iteration + 1}") + val (result, nextState) = evaluate(state) + val issues = result.issues + if issues.isEmpty then + emitStep("No review comments") + Step.Stop(IgnoredIssues(Nil)) + else if iteration >= maxIterations then + emitStep(s"Reached max iterations ($maxIterations); bailing out") + Step.Stop( + IgnoredIssues( + issues.map(i => + IgnoredIssue(i.title, s"max iterations ($maxIterations) reached") ) ) - else - val outcome = fix(issues) - emitStep( - s"Fixed ${outcome.fixed.size}, ignored ${outcome.ignored.size}" - ) - if outcome.fixed.isEmpty then Step.Stop(IgnoredIssues(outcome.ignored)) - else Step.Continue(IgnoredIssues(outcome.ignored)) + ) + else + val outcome = fix(issues) + emitStep( + s"Fixed ${outcome.fixed.size}, ignored ${outcome.ignored.size}" + ) + if outcome.fixed.isEmpty then Step.Stop(IgnoredIssues(outcome.ignored)) + else Step.Continue(IgnoredIssues(outcome.ignored), nextState) @scala.annotation.tailrec - def loop(accumulated: IgnoredIssues, iteration: Int): IgnoredIssues = - runIteration(iteration) match - case Step.Stop(add) => accumulated ++ add - case Step.Continue(add) => loop(accumulated ++ add, iteration + 1) + def loop( + accumulated: IgnoredIssues, + iteration: Int, + state: S + ): IgnoredIssues = + runIteration(iteration, state) match + case Step.Stop(add) => accumulated ++ add + case Step.Continue(add, next) => + loop(accumulated ++ add, iteration + 1, next) - loop(IgnoredIssues(Nil), 0) + loop(IgnoredIssues(Nil), 0, initial) /** Format a single review comment as the body lines of a `Step`. * @@ -113,8 +150,8 @@ private[review] def formatReviewerOutcome( * `allReviewers(claude)` etc.), and lets the loop decide which reviewers to * re-run on the next iteration based on which ones found issues this time. */ -case class ReviewBatch(outcomes: List[(LlmTool[?], ReviewResult)]): - def reviewersWithIssues: List[LlmTool[?]] = +case class ReviewBatch(outcomes: List[(Agent[?], ReviewResult)]): + def reviewersWithIssues: List[Agent[?]] = outcomes.collect { case (r, rr) if rr.issues.nonEmpty => r } def allIssues: List[ReviewIssue] = outcomes.flatMap(_._2.issues) @@ -151,7 +188,7 @@ private object ReviewLoopState: /** Run reviewers in parallel against `task`, gather per-reviewer outcomes, hand * any issues above `confidenceThreshold` to `coder` via `run(session = * sessionId)`, and loop. `reviewerSelection` decides which reviewers run each - * iteration — typically [[ReviewerSelector.llmDriven]] wired against a cheap + * iteration — typically [[ReviewerSelector.agentDriven]] wired against a cheap * picker LLM; pass [[ReviewerSelector.allEveryRound]] to skip selection * entirely. * @@ -162,9 +199,9 @@ private object ReviewLoopState: * there's nothing new for the reviewers to find, so the loop halts. */ def reviewAndFixLoop[B <: BackendTag]( - coder: LlmTool[B], + coder: Agent[B], sessionId: SessionId[B], - reviewers: List[LlmTool[?]], + reviewers: List[Agent[?]], reviewerSelection: ReviewerSelector, task: String, /** Shell command run before each review round — after the implementation @@ -179,7 +216,7 @@ def reviewAndFixLoop[B <: BackendTag]( * `lintCommand` is set; ignored otherwise. Use a cheap model * (`claude.haiku`, `codex.mini`) — the lint summary is a small fold. */ - lintLlm: Option[LlmTool[?]] = None, + lintAgent: Option[Agent[?]] = None, confidenceThreshold: Double = 0.7, maxIterations: Int = 10, fixInstructions: String = ReviewLoopPrompts.Fix, @@ -194,10 +231,10 @@ def reviewAndFixLoop[B <: BackendTag]( * been committed and `git.diff()` would be empty). */ initialDiff: Option[String] = None -)(using ctx: FlowContext): IgnoredIssues = +)(using ctx: FlowContext, ev: InStage): IgnoredIssues = require( - lintCommand.isEmpty || lintLlm.isDefined, - "reviewAndFixLoop: lintCommand requires lintLlm" + lintCommand.isEmpty || lintAgent.isDefined, + "reviewAndFixLoop: lintCommand requires lintAgent" ) require( reviewers.map(_.name).distinct.size == reviewers.size, @@ -215,13 +252,6 @@ def reviewAndFixLoop[B <: BackendTag]( val taskTitle = Title(task) val changedFiles = ReviewLoop.extractChangedFiles(sampleDiff()) - // All loop state lives in one immutable case class threaded through - // a method-scope `var`. Within an iteration reviewers fan out via - // `par`, but the parallel forks read a stable snapshot and the - // var is reassigned exactly once after they all return — so no - // concurrent mutation, no `mutable.Map`, no `ConcurrentHashMap`. - var state: ReviewLoopState = ReviewLoopState.empty - def filterByConfidence(result: ReviewResult): ReviewResult = ReviewResult(issues = result.issues.filter(_.confidence >= confidenceThreshold) @@ -244,7 +274,7 @@ def reviewAndFixLoop[B <: BackendTag]( * that same `RB`. */ def reviewWithSession[RB <: BackendTag]( - r: LlmTool[RB], + r: Agent[RB], sessions: Map[String, SessionId.Untyped], currentDiff: String ): (ReviewResult, Option[(String, SessionId.Untyped)]) = @@ -274,7 +304,7 @@ def reviewAndFixLoop[B <: BackendTag]( */ enum AgentOutcome: case Reviewer( - tool: LlmTool[?], + tool: Agent[?], result: ReviewResult, entry: Option[(String, SessionId.Untyped)] ) @@ -291,10 +321,10 @@ def reviewAndFixLoop[B <: BackendTag]( * payload — pre-sampling also avoids redundant shell-outs. */ def runReviewersAndLint( - active: List[LlmTool[?]], + active: List[Agent[?]], currentState: ReviewLoopState ): ( - List[(LlmTool[?], ReviewResult)], + List[(Agent[?], ReviewResult)], Option[ReviewResult], ReviewLoopState ) = @@ -309,12 +339,12 @@ def reviewAndFixLoop[B <: BackendTag]( val lintTaskOpt: Option[() => AgentOutcome] = lintCommand - .zip(lintLlm) - .map: (cmd, llm) => + .zip(lintAgent) + .map: (cmd, agent) => () => // Group lint tokens under the same `reviewer: …` prefix as the // dimension reviewers; the renamed copy stays local to this call. - val labelled = llm.withName("reviewer: lint") + val labelled = agent.withName("reviewer: lint") AgentOutcome.Lint(filterByConfidence(lint(cmd, labelled))) val tasks = reviewerTasks ++ lintTaskOpt.toList @@ -346,7 +376,7 @@ def reviewAndFixLoop[B <: BackendTag]( ) (reviewerOutcomes, lintOutcome, nextState) - def evaluate(): ReviewResult = + def evaluate(state: ReviewLoopState): (ReviewResult, ReviewLoopState) = // Format before reviewing so the implementation's (and each prior fix's) // edits are cleaned up before reviewers and the lint see them, and the // committed tree stays formatted. Exit status ignored — a formatter failure @@ -366,10 +396,10 @@ def reviewAndFixLoop[B <: BackendTag]( // what the fixer receives — otherwise low-confidence issues are listed // per-reviewer but silently dropped from the fix payload. val (results, lintResult, nextState) = runReviewersAndLint(active, state) - state = nextState - ReviewResult(issues = + val result = ReviewResult(issues = results.flatMap(_._2.issues) ++ lintResult.toList.flatMap(_.issues) ) + (result, nextState) def fix(issues: List[ReviewIssue]): FixOutcome = coder @@ -382,14 +412,20 @@ def reviewAndFixLoop[B <: BackendTag]( ) ._2 - // The stage doesn't repeat `task` in its label — the enclosing - // implement-task stage already names it. - orca.stage("Review & fix"): - fixLoop( - evaluate = () => evaluate(), - fix = fix, - maxIterations = maxIterations - ) + // A progress marker, not a committing stage: the enclosing implement-task + // stage already names the work and owns the commit (ADR 0018 §2.2). + orca.display("Review & fix") + // All loop state lives in one immutable ReviewLoopState threaded explicitly + // through `fixLoopWithState` (no captured `var`): within an iteration the + // reviewers fan out via `par`, but each fork reads the snapshot it was handed + // and the next state is computed once after they all return — so no concurrent + // mutation, no `mutable.Map`, no `ConcurrentHashMap`. + fixLoopWithState[ReviewLoopState]( + initial = ReviewLoopState.empty, + evaluate = evaluate, + fix = fix, + maxIterations = maxIterations + ) private[review] object ReviewLoop: /** Parse a unified diff and return the changed file paths (the `b/` side of @@ -406,7 +442,7 @@ private[review] object ReviewLoop: .distinct /** Run `command` via `bash -c`, capture both stdout and stderr, write the - * combined output to a temp file, and ask `llm` to read that file and + * combined output to a temp file, and ask `agent` to read that file and * summarise it as a `ReviewResult`. An empty output short-circuits to * `ReviewResult.empty` so clean runs skip the round-trip to the LLM. Override * `instructions` when the lint produces unusual shapes the default phrasing @@ -426,9 +462,9 @@ private[review] object ReviewLoop: */ def lint( command: String, - llm: LlmTool[?], + agent: Agent[?], instructions: String = ReviewLoopPrompts.SummariseLint -)(using FlowContext): ReviewResult = +)(using FlowContext, InStage): ReviewResult = val proc = os .proc("bash", "-c", command) .call(check = false, mergeErrIntoOut = true) @@ -445,7 +481,7 @@ def lint( deleteOnExit = false ) try - llm.withReadOnly + agent.withReadOnly .resultAs[ReviewResult] .autonomous .run( diff --git a/flow/src/main/scala/orca/review/ReviewLoopPrompts.scala b/flow/src/main/scala/orca/review/ReviewLoopPrompts.scala index 5b4e8c3f..06d6e560 100644 --- a/flow/src/main/scala/orca/review/ReviewLoopPrompts.scala +++ b/flow/src/main/scala/orca/review/ReviewLoopPrompts.scala @@ -31,7 +31,7 @@ object ReviewLoopPrompts: val Fix: String = PromptResource.load("/orca/review/prompts/fix.md") - /** Used by [[ReviewerSelector.llmDriven]] to decide which reviewers to run + /** Used by [[ReviewerSelector.agentDriven]] to decide which reviewers to run * for a given task. Agents are picked from the supplied `availableReviewers` * list by name. */ diff --git a/flow/src/main/scala/orca/review/ReviewResult.scala b/flow/src/main/scala/orca/review/ReviewResult.scala index e96a63be..48b2bb2e 100644 --- a/flow/src/main/scala/orca/review/ReviewResult.scala +++ b/flow/src/main/scala/orca/review/ReviewResult.scala @@ -1,6 +1,6 @@ package orca.review -import orca.llm.{Announce, JsonData, given} +import orca.agents.{Announce, JsonData, given} case class ReviewResult( issues: List[ReviewIssue] diff --git a/flow/src/main/scala/orca/review/ReviewerSelector.scala b/flow/src/main/scala/orca/review/ReviewerSelector.scala index 87caf813..8c969265 100644 --- a/flow/src/main/scala/orca/review/ReviewerSelector.scala +++ b/flow/src/main/scala/orca/review/ReviewerSelector.scala @@ -1,8 +1,8 @@ package orca.review -import orca.FlowContext +import orca.{FlowContext, InStage} import orca.events.OrcaEvent -import orca.llm.{AgentInput, JsonData, LlmTool, given} +import orca.agents.{AgentInput, JsonData, Agent, given} import orca.plan.Title import scala.util.matching.Regex @@ -14,15 +14,15 @@ import scala.util.matching.Regex * first iteration when there's no history yet. * - `taskTitle` and `changedFiles` come from `reviewAndFixLoop`'s `task` and * the diff it sampled at loop entry. They're passed through on every call - * so that selectors which need them (e.g. [[llmDriven]]) don't have to be - * reconstructed per task. + * so that selectors which need them (e.g. [[agentDriven]]) don't have to + * be reconstructed per task. */ type ReviewerSelector = ( history: List[ReviewBatch], - all: List[LlmTool[?]], + all: List[Agent[?]], taskTitle: Title, changedFiles: List[String] -) => List[LlmTool[?]] +) => List[Agent[?]] object ReviewerSelector: @@ -43,8 +43,8 @@ object ReviewerSelector: */ val allEveryRound: ReviewerSelector = (_, all, _, _) => all - /** Asks `llm` to pick which reviewers are worth running for a given task. The - * selection is computed on the first call and cached for subsequent + /** Asks `agent` to pick which reviewers are worth running for a given task. + * The selection is computed on the first call and cached for subsequent * iterations — task context doesn't change mid-loop, so re-querying the * model would just burn tokens for the same answer. * @@ -75,13 +75,13 @@ object ReviewerSelector: * Pick a cheap model (e.g. `claude.haiku`); the request is small. Override * `instructions` to retune the selection brief. */ - def llmDriven( - llm: LlmTool[?], + def agentDriven( + agent: Agent[?], instructions: String = ReviewLoopPrompts.SelectReviewers, descriptions: Map[String, String] = ReviewerPrompts.descriptionsByToolName, filePatterns: Map[String, Regex] = ReviewerPrompts.filePatternsByToolName - )(using ctx: FlowContext): ReviewerSelector = + )(using ctx: FlowContext, ev: InStage): ReviewerSelector = var cached: Option[List[String]] = None (_, all, taskTitle, changedFiles) => val eligible = all.filter: r => @@ -115,7 +115,7 @@ object ReviewerSelector: // to run; it should never edit files during the selection // turn. If the model reads context (Cargo.toml, etc.) to // make a better choice, that's fine. - llm.withReadOnly + agent.withReadOnly .resultAs[SelectedReviewers] .autonomous .run( diff --git a/flow/src/main/scala/orca/review/Reviewers.scala b/flow/src/main/scala/orca/review/Reviewers.scala index 5ffad54c..5f8c91ef 100644 --- a/flow/src/main/scala/orca/review/Reviewers.scala +++ b/flow/src/main/scala/orca/review/Reviewers.scala @@ -1,13 +1,13 @@ package orca.review -import orca.llm.{BackendTag, LlmTool} +import orca.agents.{BackendTag, Agent} import orca.util.PromptResource import scala.util.matching.Regex /** A reviewer agent definition: a short slug name, a description suitable for - * LLM-driven selection ([[ReviewerSelector.llmDriven]]), and the system prompt - * that personalises the underlying LLM tool. `filePattern`, when set, + * LLM-driven selection ([[ReviewerSelector.agentDriven]]), and the system + * prompt that personalises the underlying LLM tool. `filePattern`, when set, * restricts the reviewer to changes that touch at least one matching file — * the selector drops the reviewer before the picker LLM sees it. */ @@ -83,7 +83,7 @@ private[review] object ReviewerPrompts: /** A small universally-applicable subset: correctness, test quality, clarity. * Useful as a starting point when the full set is overkill — e.g. a flow * that touches small diffs where performance/architecture concerns are - * rarely actionable. Pair with [[ReviewerSelector.llmDriven]] (the default + * rarely actionable. Pair with [[ReviewerSelector.agentDriven]] (the default * in [[reviewAndFixLoop]]) to let the picker narrow further. */ val minimal: List[Reviewer] = List( @@ -93,7 +93,7 @@ private[review] object ReviewerPrompts: ) /** Descriptions keyed by the prefixed tool name a builder produces - * (`reviewer: <slug>`). [[ReviewerSelector.llmDriven]] consults this by + * (`reviewer: <slug>`). [[ReviewerSelector.agentDriven]] consults this by * default so the picker LLM gets each reviewer's purpose alongside its name. * Covers every shipped reviewer, regardless of which preset list was used to * build the actual tools. @@ -109,18 +109,18 @@ private[review] object ReviewerPrompts: val filePatternsByToolName: Map[String, Regex] = all.flatMap(r => r.filePattern.map(p => s"$NamePrefix${r.name}" -> p)).toMap -/** Build LlmTools for every reviewer the library ships with. The picker in - * [[ReviewerSelector.llmDriven]] (the default in [[reviewAndFixLoop]]) narrows - * the active set per task, so passing the full list isn't wasteful. +/** Build Agents for every reviewer the library ships with. The picker in + * [[ReviewerSelector.agentDriven]] (the default in [[reviewAndFixLoop]]) + * narrows the active set per task, so passing the full list isn't wasteful. */ -def allReviewers[B <: BackendTag](base: LlmTool[B]): List[LlmTool[B]] = +def allReviewers[B <: BackendTag](base: Agent[B]): List[Agent[B]] = buildReviewers(base, ReviewerPrompts.all) -/** Build LlmTools for the small universally-applicable subset +/** Build Agents for the small universally-applicable subset * ([[ReviewerPrompts.minimal]] — correctness, test quality, clarity). Pick * this when the full set is overkill or the flow only touches small diffs. */ -def minimalReviewers[B <: BackendTag](base: LlmTool[B]): List[LlmTool[B]] = +def minimalReviewers[B <: BackendTag](base: Agent[B]): List[Agent[B]] = buildReviewers(base, ReviewerPrompts.minimal) /** Layer each reviewer's system prompt onto the base tool, prefix the name with @@ -135,9 +135,9 @@ def minimalReviewers[B <: BackendTag](base: LlmTool[B]): List[LlmTool[B]] = * permissions. */ private def buildReviewers[B <: BackendTag]( - base: LlmTool[B], + base: Agent[B], reviewers: List[Reviewer] -): List[LlmTool[B]] = +): List[Agent[B]] = reviewers.map: r => base .withSystemPrompt(r.systemPrompt) diff --git a/flow/src/main/scala/orca/review/SelectedReviewers.scala b/flow/src/main/scala/orca/review/SelectedReviewers.scala index 04514261..64a4fb03 100644 --- a/flow/src/main/scala/orca/review/SelectedReviewers.scala +++ b/flow/src/main/scala/orca/review/SelectedReviewers.scala @@ -1,6 +1,6 @@ package orca.review -import orca.llm.{JsonData, LlmTool} +import orca.agents.{JsonData, Agent} case class SelectedReviewers(names: List[String]) derives JsonData: /** Resolve the picker's reply to tools. Matches either the bare slug (what @@ -8,7 +8,7 @@ case class SelectedReviewers(names: List[String]) derives JsonData: * tool name, so a model that returns either form still resolves — the * `reviewer: ` cost-attribution prefix must not gate selection. */ - def pick(all: List[LlmTool[?]]): List[LlmTool[?]] = + def pick(all: List[Agent[?]]): List[Agent[?]] = all.filter: r => val slug = ReviewerPrompts.stripNamePrefix(r.name) names.contains(r.name) || names.contains(slug) diff --git a/flow/src/test/scala/orca/BranchNamingTest.scala b/flow/src/test/scala/orca/BranchNamingTest.scala new file mode 100644 index 00000000..60d05186 --- /dev/null +++ b/flow/src/test/scala/orca/BranchNamingTest.scala @@ -0,0 +1,307 @@ +package orca + +import orca.agents.{ + Announce, + AutonomousTextCall, + BackendTag, + JsonData, + AgentCall, + AgentConfig, + Agent, + SessionId, + ToolSet +} +import orca.tools.IssueHandle + +/** Tests for BranchNamingStrategy.slug (security-critical) and the + * issue/fromText/shortenPrompt factory strategies. + */ +class BranchNamingTest extends munit.FunSuite: + + /** Throwing stub — issue/fromText must never touch the LLM. */ + private object ThrowingAgent extends Agent[BackendTag.ClaudeCode.type]: + val name: String = "throwing-stub" + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + throw new AssertionError("LLM must not be called") + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + throw new AssertionError("LLM must not be called") + + /** Stub LLM that returns a fixed reply from `autonomous.run`. Used to test + * `shortenPrompt` without a real model. + */ + private def stubbedAgent(reply: String): Agent[BackendTag.ClaudeCode.type] = + new Agent[BackendTag.ClaudeCode.type]: + val name: String = "stubbed" + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + new AutonomousTextCall[BackendTag.ClaudeCode.type]: + def run( + prompt: String, + session: SessionId[BackendTag.ClaudeCode.type], + config: AgentConfig, + emitPrompt: Boolean + )(using + orca.InStage + ): (SessionId[BackendTag.ClaudeCode.type], String) = + (session, reply) + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = + this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? + + /** Stub LLM that throws on `autonomous.run`. */ + private val throwingAutonomousAgent: Agent[BackendTag.ClaudeCode.type] = + new Agent[BackendTag.ClaudeCode.type]: + val name: String = "throwing-autonomous" + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + new AutonomousTextCall[BackendTag.ClaudeCode.type]: + def run( + prompt: String, + session: SessionId[BackendTag.ClaudeCode.type], + config: AgentConfig, + emitPrompt: Boolean + )(using + orca.InStage + ): (SessionId[BackendTag.ClaudeCode.type], String) = + throw new RuntimeException("LLM unavailable") + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = + this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? + + private given InStage = InStage.unsafe + + // --------------------------------------------------------------------------- + // slug: basic transformation + // --------------------------------------------------------------------------- + + test("slug lower-cases input"): + assertEquals(BranchNamingStrategy.slug("Hello World"), "hello-world") + + test("slug keeps alphanumeric and hyphens"): + assertEquals(BranchNamingStrategy.slug("abc-123"), "abc-123") + + test("slug maps spaces to hyphens"): + assertEquals(BranchNamingStrategy.slug("add a feature"), "add-a-feature") + + test("slug maps punctuation to hyphens"): + assertEquals( + BranchNamingStrategy.slug("Add a Multiply Function!"), + "add-a-multiply-function" + ) + + test("slug maps unicode/emoji to hyphens"): + assertEquals(BranchNamingStrategy.slug("café résumé"), "caf-r-sum") + + // --------------------------------------------------------------------------- + // slug: collapse and strip + // --------------------------------------------------------------------------- + + test("slug collapses consecutive hyphens"): + assertEquals(BranchNamingStrategy.slug("a---b"), "a-b") + + test("slug strips leading hyphens"): + assertEquals(BranchNamingStrategy.slug("-foo"), "foo") + + test("slug strips trailing hyphens"): + assertEquals(BranchNamingStrategy.slug("foo-"), "foo") + + // --------------------------------------------------------------------------- + // slug: security — never starts with '-' + // --------------------------------------------------------------------------- + + test("slug with leading '-rf foo' does not start with '-'"): + val result = BranchNamingStrategy.slug("-rf foo") + assert(!result.startsWith("-"), s"started with '-': $result") + assert(result.nonEmpty, "must not be empty") + + test("slug with all-punctuation input does not start with '-'"): + val result = BranchNamingStrategy.slug("!!! x") + assert(!result.startsWith("-"), s"started with '-': $result") + assert(result.nonEmpty, "must not be empty") + + // --------------------------------------------------------------------------- + // slug: security — never empty + // --------------------------------------------------------------------------- + + test("slug with only emoji returns non-empty fallback"): + val result = BranchNamingStrategy.slug("💥✨") + assert(result.nonEmpty, "must not be empty") + assert(result.matches("^[a-z0-9][a-z0-9-]*$"), s"invalid ref: $result") + + test("slug with only punctuation returns non-empty fallback"): + val result = BranchNamingStrategy.slug("!!!") + assert(result.nonEmpty, "must not be empty") + assert(result.matches("^[a-z0-9][a-z0-9-]*$"), s"invalid ref: $result") + + test("slug with empty string returns non-empty fallback"): + val result = BranchNamingStrategy.slug("") + assert(result.nonEmpty, "must not be empty") + assert(result.matches("^[a-z0-9][a-z0-9-]*$"), s"invalid ref: $result") + + // --------------------------------------------------------------------------- + // slug: result always matches git-ref regex + // --------------------------------------------------------------------------- + + test("slug result matches ^[a-z0-9][a-z0-9-]*$"): + val result = BranchNamingStrategy.slug("Fix: Issue #42 — some weird text!") + assert( + result.matches("^[a-z0-9][a-z0-9-]*$"), + s"invalid ref: $result" + ) + + // --------------------------------------------------------------------------- + // slug: length cap + // --------------------------------------------------------------------------- + + test("slug caps result to maxLen"): + val long = "a" * 100 + val result = BranchNamingStrategy.slug(long, maxLen = 20) + assert(result.length <= 20, s"length ${result.length} > 20") + + test("slug cap does not produce trailing hyphen"): + // Input that produces hyphens near the cap boundary + val input = "abc-def-ghi-jkl-mno-pqr" + val result = BranchNamingStrategy.slug(input, maxLen = 10) + assert(!result.endsWith("-"), s"trailing hyphen: $result") + assert(result.length <= 10, s"too long: ${result.length}") + + test("slug default maxLen is 50"): + val long = "a" * 100 + val result = BranchNamingStrategy.slug(long) + assert(result.length <= 50, s"length ${result.length} > 50") + + // --------------------------------------------------------------------------- + // slug: deterministic fallback contains 'flow-' + // --------------------------------------------------------------------------- + + test("slug fallback for empty input starts with 'flow-'"): + val result = BranchNamingStrategy.slug("") + assert(result.startsWith("flow-"), s"expected 'flow-' prefix: $result") + + test("slug fallback for all-punctuation starts with 'flow-'"): + val result = BranchNamingStrategy.slug("💥✨") + assert(result.startsWith("flow-"), s"expected 'flow-' prefix: $result") + + // --------------------------------------------------------------------------- + // issue strategy + // --------------------------------------------------------------------------- + + test("issue strategy resolves to fix/issue-<number>"): + val handle = IssueHandle("acme", "repo", 42) + val strategy = BranchNamingStrategy.issue(handle) + val result = strategy.resolve("ignored prompt", ThrowingAgent) + assertEquals(result, "fix/issue-42") + + test("issue strategy with custom prefix slugs the prefix"): + val handle = IssueHandle("acme", "repo", 7) + val strategy = BranchNamingStrategy.issue(handle, prefix = "feature") + val result = strategy.resolve("ignored", ThrowingAgent) + assertEquals(result, "feature/issue-7") + + test("issue strategy prefix is slugged"): + val handle = IssueHandle("acme", "repo", 3) + val strategy = BranchNamingStrategy.issue(handle, prefix = "Hot Fix!") + val result = strategy.resolve("ignored", ThrowingAgent) + assertEquals(result, "hot-fix/issue-3") + + // --------------------------------------------------------------------------- + // fromText strategy + // --------------------------------------------------------------------------- + + test("fromText strategy slugs the text"): + val strategy = BranchNamingStrategy.fromText("Add a Multiply Function!") + val result = strategy.resolve("ignored prompt", ThrowingAgent) + assertEquals(result, "add-a-multiply-function") + + test("fromText strategy ignores userPrompt and agent"): + val strategy = BranchNamingStrategy.fromText("my-feature") + val result = strategy.resolve("should be ignored", ThrowingAgent) + assertEquals(result, "my-feature") + + // --------------------------------------------------------------------------- + // shortenPrompt strategy + // --------------------------------------------------------------------------- + + test("shortenPrompt slugs the agent reply"): + val agent = stubbedAgent("add multiply function") + val result = BranchNamingStrategy.shortenPrompt.resolve( + "Add a multiply function to the calc", + agent + ) + assertEquals(result, "add-multiply-function") + + test( + "shortenPrompt: agent returns phrase with extra whitespace, still slugged" + ): + val agent = stubbedAgent(" fix login bug ") + val result = + BranchNamingStrategy.shortenPrompt.resolve("Fix the login bug", agent) + assertEquals(result, "fix-login-bug") + + test("shortenPrompt: agent throws -> falls back to slug(userPrompt)"): + val result = BranchNamingStrategy.shortenPrompt.resolve( + "add multiply function", + throwingAutonomousAgent + ) + assertEquals(result, "add-multiply-function") + + test( + "shortenPrompt: agent returns blank string -> falls back to slug(userPrompt)" + ): + val agent = stubbedAgent(" ") + val result = + BranchNamingStrategy.shortenPrompt.resolve("fix the login bug", agent) + assertEquals(result, "fix-the-login-bug") + + test("shortenPrompt: agent returns multi-line reply, uses only first line"): + val agent = stubbedAgent("fix login bug\nsome extra explanation") + val result = + BranchNamingStrategy.shortenPrompt.resolve("Fix login bug", agent) + assertEquals(result, "fix-login-bug") + + test("shortenPrompt: a markdown-fenced reply is unwrapped (no literal ```)"): + // The cheap model sometimes wraps its one-line reply in a code fence; + // cheapOneShot must skip the fence lines, not return a literal "```". + val agent = stubbedAgent("```\nfix login bug\n```") + val result = + BranchNamingStrategy.shortenPrompt.resolve("Fix login bug", agent) + assertEquals(result, "fix-login-bug") + + test( + "producer == validator: slug output always passes RecoveryCheck.isSafeBranchRef" + ): + // Pins that the producer (slug) and the untrusted-header validator agree by + // construction — they now share one predicate (BranchNamingStrategy.isSlugSegment). + val inputs = List( + "Add a Multiply Function!", + " -rf dangerous ", + "💥✨ only emoji", + "", + "!!!", + "café résumé", + "a" * 200, + "..", + "-leading-dash", + "Mixed/CASE/Segments" + ) + for in <- inputs do + val s = BranchNamingStrategy.slug(in) + assert( + BranchNamingStrategy.isSlugSegment(s), + s"slug('$in') = '$s' must be a valid slug segment" + ) + assert( + orca.progress.RecoveryCheck.isSafeBranchRef(s), + s"slug('$in') = '$s' must satisfy isSafeBranchRef" + ) diff --git a/flow/src/test/scala/orca/CapabilitiesTest.scala b/flow/src/test/scala/orca/CapabilitiesTest.scala new file mode 100644 index 00000000..37bd3fc6 --- /dev/null +++ b/flow/src/test/scala/orca/CapabilitiesTest.scala @@ -0,0 +1,23 @@ +package orca + +import orca.events.EventDispatcher + +/** Tests for the FlowControl capability type (ADR 0018 §2.2). + * + * Verifies that FlowControl <: FlowContext (subtyping satisfaction). + */ +class CapabilitiesTest extends munit.FunSuite: + + private def stubCtrl: FlowControl = + new TestFlowContext(new EventDispatcher(Nil)) with FlowControl: + def progressStore: orca.progress.ProgressStore = + throw new NotImplementedError + def nextOccurrence(stageName: String): Int = throw new NotImplementedError + def nextSessionOccurrence(): Int = throw new NotImplementedError + + test("FlowControl satisfies a using FlowContext requirement"): + def needsCtx(using FlowContext): Boolean = true + given FlowControl = stubCtrl + assert(needsCtx) + +end CapabilitiesTest diff --git a/flow/src/test/scala/orca/CommitMessageTest.scala b/flow/src/test/scala/orca/CommitMessageTest.scala new file mode 100644 index 00000000..92e8de3b --- /dev/null +++ b/flow/src/test/scala/orca/CommitMessageTest.scala @@ -0,0 +1,208 @@ +package orca + +import orca.events.OrcaEvent +import orca.agents.{ + Announce, + AutonomousTextCall, + BackendTag, + JsonData, + AgentCall, + AgentConfig, + Agent, + SessionId, + ToolSet +} +import orca.progress.ProgressStore +import orca.testkit.GitRepo +import orca.tools.{GitTool, OsGitTool} + +import java.util.concurrent.ConcurrentHashMap +import java.util.concurrent.atomic.AtomicInteger + +/** Tests for the agent-generated commit-message path in `recordAndCommit`. + * + * Strategy: build a `TestFlowControlWithAgent` that wires a real temp repo and + * a stubbed LLM, then assert the message in `git log` after a stage runs. + */ +class CommitMessageTest extends munit.FunSuite: + + // -------------------------------------------------------------------------- + // Stubs + // -------------------------------------------------------------------------- + + /** Agent stub whose `autonomous.run` returns a fixed reply. Models both the + * cheap (via `cheap`) and the full tool — the commit-message path calls + * `fc.cheapOneShot`, which runs the lead's `cheap`, so `cheap` must also + * return this stub. + */ + private def stubbedAgent( + reply: String + ): Agent[BackendTag.ClaudeCode.type] = + new Agent[BackendTag.ClaudeCode.type]: + val name: String = "stubbed" + override def cheap: Agent[BackendTag.ClaudeCode.type] = this + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + new AutonomousTextCall[BackendTag.ClaudeCode.type]: + def run( + prompt: String, + session: SessionId[BackendTag.ClaudeCode.type], + config: AgentConfig, + emitPrompt: Boolean + )(using + orca.InStage + ): (SessionId[BackendTag.ClaudeCode.type], String) = + (session, reply) + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = + this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? + + /** LLM stub that throws on `autonomous.run`. */ + private val throwingAgent: Agent[BackendTag.ClaudeCode.type] = + new Agent[BackendTag.ClaudeCode.type]: + val name: String = "throwing" + override def cheap: Agent[BackendTag.ClaudeCode.type] = this + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + new AutonomousTextCall[BackendTag.ClaudeCode.type]: + def run( + prompt: String, + session: SessionId[BackendTag.ClaudeCode.type], + config: AgentConfig, + emitPrompt: Boolean + )(using + orca.InStage + ): (SessionId[BackendTag.ClaudeCode.type], String) = + throw new RuntimeException("LLM unavailable") + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = + this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? + + // -------------------------------------------------------------------------- + // Test helper + // -------------------------------------------------------------------------- + + /** A `FlowControl` backed by a real temp git repo and the given LLM stub. */ + private class FlowControlWithAgent( + val agentStub: Agent[BackendTag.ClaudeCode.type], + val git: GitTool, + val progressStore: ProgressStore, + val userPrompt: String = "p" + ) extends FlowControl: + import orca.agents.{ + ClaudeAgent, + CodexAgent, + GeminiAgent, + OpencodeAgent, + PiAgent + } + private def stub(n: String) = + throw new NotImplementedError(s"$n not wired") + type LeadB = BackendTag.ClaudeCode.type + // The leading agent IS the test's stub; the commit path's + // `fc.agent.cheapOneShot` runs the stub's canned reply. + def agent: Agent[LeadB] = agentStub + lazy val claude: ClaudeAgent = stub("claude") + lazy val codex: CodexAgent = stub("codex") + lazy val opencode: OpencodeAgent = stub("opencode") + lazy val pi: PiAgent = stub("pi") + lazy val gemini: GeminiAgent = stub("gemini") + lazy val gh: orca.tools.GitHubTool = stub("gh") + lazy val fs: orca.tools.FsTool = stub("fs") + def emit(event: OrcaEvent): Unit = () + private val occ = new ConcurrentHashMap[String, AtomicInteger] + def nextOccurrence(name: String): Int = + occ.computeIfAbsent(name, _ => new AtomicInteger(0)).getAndIncrement() + private val sessOcc = new AtomicInteger(0) + def nextSessionOccurrence(): Int = sessOcc.getAndIncrement() + + private def withCtx( + agentStub: Agent[BackendTag.ClaudeCode.type] + )(body: (FlowControl, os.Path) => Unit): Unit = + val dir = GitRepo.seeded() + val git = new OsGitTool(dir) + val store = ProgressStore.default(dir, "p") + given InStage = InStage.unsafe + store.writeHeader( + orca.progress.ProgressHeader("main", "feat/test", "deadbeef") + ) + body(new FlowControlWithAgent(agentStub, git, store), dir) + + private def lastCommitMessage(dir: os.Path): String = + os.proc("git", "log", "-1", "--pretty=%s").call(cwd = dir).out.text().trim + + // -------------------------------------------------------------------------- + // Tests + // -------------------------------------------------------------------------- + + test( + "stage with no commitMessage and non-empty diff uses agent.cheap message" + ): + withCtx(stubbedAgent("Add feature file")): (ctx, dir) => + given FlowControl = ctx + val _ = stage("write file"): + // Modify the tracked seed file (not a new untracked file) so + // `git diff HEAD` captures the change. + os.write.over(dir / "seed.txt", "modified by stage") + "done" + assertEquals(lastCommitMessage(dir), "Add feature file") + + test("stage with no commitMessage but empty diff falls back to stage:<name>"): + // An empty working-tree diff (no code changes, only the progress file + // force-added) triggers the `s"stage: $name"` fallback. + withCtx(stubbedAgent("should not appear")): (ctx, dir) => + given FlowControl = ctx + // Run a stage that produces no code changes — only the progress file changes. + val _ = stage("no-op"): + "done" + // The commit message must be the fallback, not the LLM reply, because the + // diff was empty (no code files modified in the body). + assertEquals(lastCommitMessage(dir), "stage: no-op") + + test( + "stage with no commitMessage and throwing agent falls back to stage:<name>" + ): + withCtx(throwingAgent): (ctx, dir) => + given FlowControl = ctx + val _ = stage("write file"): + os.write.over(dir / "seed.txt", "modified by stage") + "done" + assertEquals(lastCommitMessage(dir), "stage: write file") + + test("stage with explicit commitMessage uses it verbatim (no agent call)"): + // The explicit message path must not touch the LLM — use throwingAgent to + // prove it. + withCtx(throwingAgent): (ctx, dir) => + given FlowControl = ctx + val _ = stage[String]( + "write file", + commitMessage = Some(_ => "explicit: my message") + ): + os.write.over(dir / "seed.txt", "modified by stage") + "done" + assertEquals(lastCommitMessage(dir), "explicit: my message") + + test( + "stage with no commitMessage and blank agent reply falls back to stage:<name>" + ): + withCtx(stubbedAgent(" ")): (ctx, dir) => + given FlowControl = ctx + val _ = stage("write file"): + os.write.over(dir / "seed.txt", "modified by stage") + "done" + assertEquals(lastCommitMessage(dir), "stage: write file") + + test("stage with no commitMessage uses first line of multi-line agent reply"): + withCtx(stubbedAgent("Add feature\n\nSome explanation here.")): + (ctx, dir) => + given FlowControl = ctx + val _ = stage("write file"): + os.write.over(dir / "seed.txt", "modified by stage") + "done" + assertEquals(lastCommitMessage(dir), "Add feature") diff --git a/flow/src/test/scala/orca/CostTrackerTest.scala b/flow/src/test/scala/orca/CostTrackerTest.scala index 987bb409..f8d1af9c 100644 --- a/flow/src/test/scala/orca/CostTrackerTest.scala +++ b/flow/src/test/scala/orca/CostTrackerTest.scala @@ -8,7 +8,7 @@ import orca.events.{ PriceList, Usage } -import orca.llm.Model +import orca.agents.Model import java.time.LocalDate @@ -46,7 +46,7 @@ class CostTrackerTest extends munit.FunSuite: tracker.onEvent(tokens("performance", Some("haiku"), Usage(30L, 20L, None))) assertEquals(tracker.total, Usage(130L, 70L, None)) - test("perAgent groups by LlmTool name"): + test("perAgent groups by Agent name"): val tracker = new CostTracker tracker.onEvent(tokens("claude", Some("opus"), Usage(10L, 5L, None))) tracker.onEvent(tokens("performance", Some("opus"), Usage(20L, 15L, None))) diff --git a/flow/src/test/scala/orca/FlowTest.scala b/flow/src/test/scala/orca/FlowTest.scala index 70df52b7..1dd8a307 100644 --- a/flow/src/test/scala/orca/FlowTest.scala +++ b/flow/src/test/scala/orca/FlowTest.scala @@ -12,17 +12,22 @@ class FlowTest extends munit.FunSuite: val _ = seen.updateAndGet(event :: _) def events: List[OrcaEvent] = seen.get().reverse - private def fixture: (RecordingListener, FlowContext) = + private def fixture: (RecordingListener, FlowControl) = val listener = new RecordingListener - (listener, new TestFlowContext(new EventDispatcher(List(listener)))) + val (control, _) = + TestFlowControl.create(new EventDispatcher(List(listener))) + (listener, control) test("stage emits StageStarted then StageCompleted around the body"): val (listener, ctx) = fixture - given FlowContext = ctx - val result = stage("plan") { 7 } + given FlowControl = ctx + val result = stage("plan")(7) assertEquals(result, 7) assertEquals( - listener.events, + listener.events.collect { + case e: OrcaEvent.StageStarted => e + case e: OrcaEvent.StageCompleted => e + }, List( OrcaEvent.StageStarted("plan"), OrcaEvent.StageCompleted("plan") @@ -31,9 +36,9 @@ class FlowTest extends munit.FunSuite: test("stage emits Error and re-raises when the body throws"): val (listener, ctx) = fixture - given FlowContext = ctx + given FlowControl = ctx val _ = intercept[RuntimeException]: - stage("risky") { throw new RuntimeException("kaboom") } + stage[String]("risky")(throw new RuntimeException("kaboom")) assert( listener.events.exists { case OrcaEvent.Error(msg) => @@ -49,9 +54,9 @@ class FlowTest extends munit.FunSuite: test("stage does not double-emit Error when the body calls fail"): val (listener, ctx) = fixture - given FlowContext = ctx + given FlowControl = ctx val _ = intercept[OrcaFlowException]: - stage("plan") { orca.fail("already emitted")(using ctx) } + stage[String]("plan")(orca.fail("already emitted")(using ctx)) val errors = listener.events.collect { case e: OrcaEvent.Error => e } assertEquals(errors, List(OrcaEvent.Error("already emitted"))) @@ -66,11 +71,17 @@ class FlowTest extends munit.FunSuite: // stage catch must surface them or the user sees `exit 1` with no // diagnostic. val (listener, ctx) = fixture - given FlowContext = ctx + given FlowControl = ctx val _ = intercept[OrcaFlowException]: - stage("tool-call") { throw new OrcaFlowException("git push failed") } + stage[String]("tool-call")(throw new OrcaFlowException("git push failed")) val errors = listener.events.collect { case e: OrcaEvent.Error => e } assertEquals( errors, List(OrcaEvent.Error("Stage 'tool-call' failed: git push failed")) ) + + test("display emits a Step without a stage or commit"): + val (listener, ctx) = fixture + given FlowControl = ctx + display("just a note") + assertEquals(listener.events, List(OrcaEvent.Step("just a note"))) diff --git a/flow/src/test/scala/orca/RunSeededTest.scala b/flow/src/test/scala/orca/RunSeededTest.scala new file mode 100644 index 00000000..e9924465 --- /dev/null +++ b/flow/src/test/scala/orca/RunSeededTest.scala @@ -0,0 +1,409 @@ +package orca + +import munit.FunSuite +import orca.backend.{Conversation, Interaction, AgentBackend, AgentResult} +import orca.events.OrcaListener +import orca.agents.{ + Announce, + AutonomousTextCall, + BackendTag, + JsonData, + AgentCall, + AgentConfig, + Agent, + Prompts, + SessionId, + ToolSet, + BaseAgent +} +import orca.progress.{ProgressHeader, ProgressStore, StageEntry, SessionRecord} + +/** Tests for `agent.runSeeded` (ADR 0018 §2.6, task D-seed). + * + * Each test scenario uses a [[StubAgentForSeeded]] whose `sessionExists` and + * `autonomous.run` behaviours are injected at construction time, and whose + * `capturedPrompt` lets tests assert what the prompt looked like after + * preamble/seed composition. + */ +class RunSeededTest extends FunSuite: + + // `runSeeded` is now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe + + /** A fixed session id used across all tests; avoids UUID randomness in + * assertions and lets `makeControl` pre-populate the log without forward + * references. + */ + private val testSessionId = "test-session-uuid-1234" + private val testSession: SessionId[BackendTag.ClaudeCode.type] = + SessionId[BackendTag.ClaudeCode.type](testSessionId) + + /** Controllable Agent stub for seeded-run tests. + * + * @param existsResult + * The value `sessionExists` returns — set `true` to exercise the "live + * session" branch, `false` to exercise the re-seed path. + * @param runResult + * The text `autonomous.run` echoes back. + */ + private class StubAgentForSeeded( + existsResult: Boolean, + runResult: String = "ok", + learnedWireId: Option[String] = None + ) extends Agent[BackendTag.ClaudeCode.type]: + val name: String = "stub-seeded" + + private var _capturedPrompt: Option[String] = None + + /** The prompt the stub's `autonomous.run` actually received. */ + def capturedPrompt: Option[String] = _capturedPrompt + + override def sessionExists( + session: SessionId[BackendTag.ClaudeCode.type] + ): Boolean = existsResult + + /** Reports a learned wire id (server-id backends) so the persist path can + * be exercised; `None` mirrors pi (no durable resume). + */ + override def resumeWireId( + client: SessionId[BackendTag.ClaudeCode.type] + ): Option[SessionId[BackendTag.ClaudeCode.type]] = + learnedWireId.map(SessionId[BackendTag.ClaudeCode.type](_)) + + val autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + new AutonomousTextCall[BackendTag.ClaudeCode.type]: + def run( + prompt: String, + session: SessionId[BackendTag.ClaudeCode.type], + config: AgentConfig, + emitPrompt: Boolean + )(using orca.InStage): (SessionId[BackendTag.ClaudeCode.type], String) = + _capturedPrompt = Some(prompt) + (session, runResult) + + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + ??? + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + + /** A minimal `AgentBackend` stub whose `sessionExists` returns a fixed value. + * All other methods throw — they must never be called in these tests. + */ + private class StubBackend(existsResult: Boolean) + extends AgentBackend[BackendTag.ClaudeCode.type]: + def runAutonomous( + prompt: String, + session: SessionId[BackendTag.ClaudeCode.type], + config: AgentConfig, + workDir: os.Path, + events: OrcaListener, + outputSchema: Option[String] + ): AgentResult[BackendTag.ClaudeCode.type] = ??? + def runInteractive( + prompt: String, + session: SessionId[BackendTag.ClaudeCode.type], + displayPrompt: String, + config: AgentConfig, + workDir: os.Path, + outputSchema: Option[String] + )(using ox.Ox): Conversation[BackendTag.ClaudeCode.type] = ??? + override def sessionExists( + session: SessionId[BackendTag.ClaudeCode.type] + ): Boolean = existsResult + + /** A minimal `BaseAgent`-derived tool backed by [[StubBackend]]. Exercises + * the real `BaseAgent.sessionExists → backend.sessionExists` delegation path + * in production wiring. + */ + private class StubBasedTool(backend: StubBackend) + extends BaseAgent[BackendTag.ClaudeCode.type, Agent[ + BackendTag.ClaudeCode.type + ]]( + backend, + AgentConfig.default, + NoOpPrompts, + os.temp.dir(), + OrcaListener.noop, + NoOpInteraction + ): + val name: String = "stub-based" + protected def copyTool( + config: AgentConfig, + name: String + ): Agent[BackendTag.ClaudeCode.type] = this + override def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = + this + override def withSystemPrompt( + p: String + ): Agent[BackendTag.ClaudeCode.type] = this + override def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + override def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = + this + override def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + ??? + + /** No-op [[Prompts]] for use in [[StubBasedTool]]; never called in these + * tests because we only exercise `sessionExists`, not the run path. + */ + private object NoOpPrompts extends Prompts: + def autonomous( + input: String, + outputSchema: String, + config: AgentConfig + ): String = ??? + def interactive( + input: String, + outputSchema: String, + config: AgentConfig + ): String = ??? + def retry(failedResponse: String, parseError: String): String = ??? + + /** No-op [[Interaction]] for use in [[StubBasedTool]]. */ + private object NoOpInteraction extends Interaction: + def listeners: List[OrcaListener] = Nil + def drive[B <: BackendTag]( + conversation: Conversation[B] + ): AgentResult[B] = ??? + + // ── test helpers ────────────────────────────────────────────────────────── + + /** Build a `TestFlowControl` over a temp dir with a header already written. + * Optionally writes session records and/or completed stage entries so tests + * can exercise the "progress preamble" and "recorded seed" paths. + */ + private def makeControl( + sessions: List[SessionRecord] = Nil, + completedStages: List[String] = Nil + ): TestFlowControl = + val dir = os.temp.dir() + val store = ProgressStore.default(dir, "p") + given InStage = InStage.unsafe + store.writeHeader(ProgressHeader("main", "feat/test", "deadbeef")) + for record <- sessions do store.upsertSession(record) + for stageName <- completedStages do + store.appendEntry( + StageEntry(id = s"$stageName#0", name = stageName, resultJson = "null") + ) + val git = new orca.tools.OsGitTool(dir) + new TestFlowControl(new orca.events.EventDispatcher(Nil), git, store, "p") + + // ── tests ───────────────────────────────────────────────────────────────── + + test("live session: prompt forwarded verbatim, no preamble, no seed"): + val seed = "You are a planning agent." + val fc = makeControl( + sessions = List(SessionRecord(index = 0, id = testSessionId, seed = seed)) + ) + val agent = new StubAgentForSeeded(existsResult = true) + val originalPrompt = "implement feature X" + val _ = agent.runSeeded(originalPrompt, testSession)(using fc) + assertEquals( + agent.capturedPrompt, + Some(originalPrompt), + "live session must pass prompt verbatim" + ) + + test( + "fresh session (not exists, no completed stages): seed + prompt, no preamble" + ): + val seed = "You are a planning agent." + val fc = makeControl( + sessions = List(SessionRecord(index = 0, id = testSessionId, seed = seed)) + ) + val agent = new StubAgentForSeeded(existsResult = false) + val originalPrompt = "implement feature X" + val _ = agent.runSeeded(originalPrompt, testSession)(using fc) + val prompt = agent.capturedPrompt.getOrElse(fail("no prompt captured")) + assert(prompt.contains(seed), s"prompt must contain seed; got: $prompt") + assert( + prompt.contains(originalPrompt), + s"prompt must contain original prompt; got: $prompt" + ) + assert( + !prompt.contains("Progress so far"), + s"no preamble expected on first run; got: $prompt" + ) + + test( + "lost session on resume (not exists, completed stages): preamble + seed + prompt" + ): + val seed = "You are a planning agent." + val fc = makeControl( + sessions = + List(SessionRecord(index = 0, id = testSessionId, seed = seed)), + completedStages = List("triage", "implement") + ) + val agent = new StubAgentForSeeded(existsResult = false) + val originalPrompt = "continue the work" + val _ = agent.runSeeded(originalPrompt, testSession)(using fc) + val prompt = agent.capturedPrompt.getOrElse(fail("no prompt captured")) + assert( + prompt.contains("Progress so far"), + s"expected preamble on resume; got: $prompt" + ) + assert( + prompt.contains("triage"), + s"preamble must name completed stage 'triage'; got: $prompt" + ) + assert( + prompt.contains("implement"), + s"preamble must name completed stage 'implement'; got: $prompt" + ) + // Neutral wording: the preamble is injected both on a true resume and on the + // first task after an earlier stage in the SAME run, so it must not claim + // the run was "interrupted". + assert( + !prompt.toLowerCase.contains("interrupted"), + s"preamble wording must stay neutral (no 'interrupted'); got: $prompt" + ) + assert(prompt.contains(seed), s"prompt must contain seed; got: $prompt") + assert( + prompt.contains(originalPrompt), + s"prompt must contain original prompt; got: $prompt" + ) + // Contract: preamble precedes seed precedes prompt + val preambleIdx = prompt.indexOf("Progress so far") + val seedIdx = prompt.indexOf(seed) + val promptIdx = prompt.indexOf(originalPrompt) + assert( + preambleIdx < seedIdx, + s"preamble must appear before seed; indices preamble=$preambleIdx seed=$seedIdx" + ) + assert( + seedIdx < promptIdx, + s"seed must appear before prompt; indices seed=$seedIdx prompt=$promptIdx" + ) + + test( + "no recorded seed for session: captured prompt equals bare original prompt" + ): + // Session id not in the log -> seed treated as absent; no preamble + // (no completed stages) -> prompt must be forwarded verbatim with no + // leading `---` separator or seed blob. + val fc = makeControl(sessions = Nil) + val agent = new StubAgentForSeeded(existsResult = false) + val originalPrompt = "do something" + val _ = agent.runSeeded(originalPrompt, testSession)(using fc) + assertEquals( + agent.capturedPrompt, + Some(originalPrompt), + "no seed + no preamble must produce bare original prompt, not '---\\n\\n' + prompt" + ) + + test( + "no seed but completed stages: captured prompt has preamble and prompt, no seed blob, no stray separator" + ): + // Session exists in log with empty seed; there are completed stages so + // a preamble is generated. The prompt must contain the preamble and the + // original prompt but MUST NOT contain a seed blob (it's empty) and MUST + // NOT start with `---` (the separator only appears between context and prompt). + val fc = makeControl( + sessions = List(SessionRecord(index = 0, id = testSessionId, seed = "")), + completedStages = List("triage") + ) + val agent = new StubAgentForSeeded(existsResult = false) + val originalPrompt = "continue" + val _ = agent.runSeeded(originalPrompt, testSession)(using fc) + val prompt = agent.capturedPrompt.getOrElse(fail("no prompt captured")) + assert( + prompt.contains("Progress so far"), + s"expected preamble when stages completed; got: $prompt" + ) + assert( + prompt.contains(originalPrompt), + s"prompt must contain original prompt; got: $prompt" + ) + assert( + !prompt.startsWith("---"), + s"prompt must not start with bare separator; got: $prompt" + ) + + test("runSeeded returns the session id and output from autonomous.run"): + val seed = "seed text" + val fc = makeControl( + sessions = List(SessionRecord(index = 0, id = testSessionId, seed = seed)) + ) + val agent = + new StubAgentForSeeded(existsResult = false, runResult = "agent output") + val (returnedSession, output) = + agent.runSeeded("prompt", testSession)(using fc) + assertEquals(returnedSession, testSession) + assertEquals(output, "agent output") + + test( + "runSeeded persists a newly-learned wire id into the SessionRecord" + ): + val fc = makeControl( + sessions = + List(SessionRecord(index = 0, id = testSessionId, seed = "seed")) + ) + val agent = new StubAgentForSeeded( + existsResult = false, + learnedWireId = Some("server-thread-xyz") + ) + val _ = agent.runSeeded("prompt", testSession)(using fc) + val record = + fc.progressStore.load().get.sessions.find(_.id == testSessionId).get + assertEquals(record.resumeWireId, Some("server-thread-xyz")) + + test( + "runSeeded leaves resumeWireId None when the backend reports no wire id" + ): + // pi: ephemeral sessions, resumeWireId returns None. + val fc = makeControl( + sessions = + List(SessionRecord(index = 0, id = testSessionId, seed = "seed")) + ) + val agent = + new StubAgentForSeeded(existsResult = false, learnedWireId = None) + val _ = agent.runSeeded("prompt", testSession)(using fc) + val record = + fc.progressStore.load().get.sessions.find(_.id == testSessionId).get + assertEquals(record.resumeWireId, None) + + test( + "runSeeded does NOT clobber a previously-persisted resumeWireId when the backend reports None" + ): + // The guard in persistResumeWireId calls agent.resumeWireId(session).foreach { … } + // so a None result short-circuits and the record's resumeWireId is left intact. + // Pre-seed the log with a SessionRecord whose resumeWireId is already + // Some("server-1"), then run with a stub whose resumeWireId returns None, and + // confirm the stored resumeWireId is still Some("server-1") (not cleared). + val fc = makeControl( + sessions = List( + SessionRecord( + index = 0, + id = testSessionId, + seed = "seed", + resumeWireId = Some("server-1") + ) + ) + ) + val agent = + new StubAgentForSeeded(existsResult = false, learnedWireId = None) + val _ = agent.runSeeded("prompt", testSession)(using fc) + val record = + fc.progressStore.load().get.sessions.find(_.id == testSessionId).get + assertEquals( + record.resumeWireId, + Some("server-1"), + "a previously-persisted resumeWireId must NOT be clobbered when the backend reports None" + ) + + test( + "Agent.sessionExists: BaseAgent delegates to backend (returns false)" + ): + // Real delegation: StubBasedTool → BaseAgent.sessionExists → StubBackend.sessionExists + val tool = new StubBasedTool(new StubBackend(existsResult = false)) + assert(!tool.sessionExists(testSession)) + + test( + "Agent.sessionExists: BaseAgent delegates to backend (returns true)" + ): + // Real delegation: StubBasedTool → BaseAgent.sessionExists → StubBackend.sessionExists + val tool = new StubBasedTool(new StubBackend(existsResult = true)) + assert(tool.sessionExists(testSession)) diff --git a/flow/src/test/scala/orca/SessionTest.scala b/flow/src/test/scala/orca/SessionTest.scala new file mode 100644 index 00000000..b8045c17 --- /dev/null +++ b/flow/src/test/scala/orca/SessionTest.scala @@ -0,0 +1,129 @@ +package orca + +import munit.FunSuite +import orca.events.{EventDispatcher, OrcaEvent, OrcaListener} +import orca.agents.{ + Announce, + AutonomousTextCall, + BackendTag, + JsonData, + AgentCall, + AgentConfig, + Agent, + SessionId, + ToolSet +} +import orca.progress.{ProgressHeader, ProgressStore} +import orca.tools.OsGitTool + +/** Tests for `agent.session(seed)` get-or-create (ADR 0018 §2.6). */ +class SessionTest extends FunSuite: + + /** Minimal Agent stub — `session(seed)` is pure and never calls the backend, + * so no methods need real implementations. + */ + private class StubAgent extends Agent[BackendTag.ClaudeCode.type]: + val name: String = "stub-agent" + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + + private def freshStore(prompt: String = "p"): (ProgressStore, os.Path) = + val dir = os.temp.dir() + val store = ProgressStore.default(dir, prompt) + given InStage = InStage.unsafe + store.writeHeader( + ProgressHeader("main", "feat/test", "deadbeef") + ) + (store, dir) + + private def makeControl( + store: ProgressStore, + dir: os.Path, + listeners: List[OrcaListener] = Nil + ): TestFlowControl = + val git = new OsGitTool(dir) + new TestFlowControl(new EventDispatcher(listeners), git, store, "p") + + /** Captures emitted `Step` messages so a test can assert on warnings. */ + private class RecordingListener extends OrcaListener: + private val buf = scala.collection.mutable.ListBuffer.empty[String] + def steps: List[String] = buf.toList + def onEvent(event: OrcaEvent): Unit = event match + case OrcaEvent.Step(msg) => buf += msg + case _ => () + + test("first agent.session call mints a SessionId and records it at index 0"): + val (store, dir) = freshStore() + val fc = makeControl(store, dir) + val agent = new StubAgent + val id = agent.session("plan brief")(using fc) + val log = store.load().get + assertEquals(log.sessions.size, 1) + assertEquals(log.sessions.head.index, 0) + assertEquals(log.sessions.head.seed, "plan brief") + assertEquals(log.sessions.head.id, id.value) + + test("second agent.session call mints a separate id at index 1"): + val (store, dir) = freshStore() + val fc = makeControl(store, dir) + val agent = new StubAgent + val id0 = agent.session("seed zero")(using fc) + val id1 = agent.session("seed one")(using fc) + assert(id0.value != id1.value, "distinct sessions must have different ids") + val sessions = store.load().get.sessions + assertEquals(sessions.size, 2) + assertEquals(sessions(0).index, 0) + assertEquals(sessions(1).index, 1) + + test( + "resume: a fresh FlowControl over the same store returns the recorded id" + ): + val (store, dir) = freshStore() + val fc1 = makeControl(store, dir) + val agent = new StubAgent + val originalId = agent.session("plan brief")(using fc1) + + // Simulate a second run: new FlowControl, same underlying store. + val fc2 = makeControl(store, dir) + val resumedId = agent.session("plan brief")(using fc2) + + assertEquals(resumedId, originalId) + // Must not mint a second record — still exactly one session. + assertEquals(store.load().get.sessions.size, 1) + + test("resume with a matching seed emits no divergence warning"): + val (store, dir) = freshStore() + val agent = new StubAgent + val _ = agent.session("plan brief")(using makeControl(store, dir)) + val recorder = new RecordingListener + val _ = + agent.session("plan brief")(using makeControl(store, dir, List(recorder))) + assert( + !recorder.steps.exists(_.contains("warning")), + s"no warning expected; got: ${recorder.steps}" + ) + + test("resume with a divergent seed at the same index warns loudly"): + // The positional key (index 0) matches but the seed differs — the most + // likely symptom of a shifted `session(...)` call sequence. + val (store, dir) = freshStore() + val agent = new StubAgent + val originalId = + agent.session("original seed")(using makeControl(store, dir)) + val recorder = new RecordingListener + val resumedId = + agent.session("different seed")(using + makeControl(store, dir, List(recorder)) + ) + // Still returns the recorded id (re-seed is the safe fallback)... + assertEquals(resumedId, originalId) + // ...but the divergence is surfaced. + assert( + recorder.steps.exists(s => s.contains("warning") && s.contains("#0")), + s"expected a divergence warning; got: ${recorder.steps}" + ) diff --git a/flow/src/test/scala/orca/StageRuntimeTest.scala b/flow/src/test/scala/orca/StageRuntimeTest.scala new file mode 100644 index 00000000..723612e7 --- /dev/null +++ b/flow/src/test/scala/orca/StageRuntimeTest.scala @@ -0,0 +1,148 @@ +package orca + +import orca.events.{EventDispatcher, OrcaEvent, OrcaListener} +import orca.progress.StageEntry + +import java.util.concurrent.atomic.{AtomicInteger, AtomicReference} + +/** The stage runtime's resume + commit guarantees (ADR 0018 §2.1/§2.4). */ +class StageRuntimeTest extends munit.FunSuite: + + private class RecordingListener extends OrcaListener: + private val seen: AtomicReference[List[OrcaEvent]] = AtomicReference(Nil) + def onEvent(event: OrcaEvent): Unit = + val _ = seen.updateAndGet(event :: _) + def events: List[OrcaEvent] = seen.get().reverse + + test("a completed stage commits both code changes and the progress log"): + val (ctx, dir) = TestFlowControl.create(new EventDispatcher(Nil)) + given FlowControl = ctx + val before = commitCount(dir) + val _ = stage("write file"): + os.write(dir / "out.txt", "hello") + "done" + assertEquals(commitCount(dir), before + 1) + // The progress file is tracked even though nothing gitignores it here. + assert( + tracked(dir).contains(ctx.progressStore.path.relativeTo(dir).toString), + "progress log must be committed" + ) + assert(tracked(dir).contains("out.txt"), "code change must be committed") + // And the result is recorded for resume. + val entry = + ctx.progressStore.load().get.entries.find(_.id == "write file#0") + assertEquals(entry.map(_.resultJson), Some("\"done\"")) + + test("re-running replays the stored result without running the body again"): + val listener = new RecordingListener + val (ctx, dir) = TestFlowControl.create(new EventDispatcher(List(listener))) + val runs = new AtomicInteger(0) + + def runOnce()(using FlowControl): String = + stage("compute"): + val _ = runs.incrementAndGet() + os.write(dir / "marker.txt", "x") + "value-42" + + val first = runOnce()(using ctx) + // A second control over the SAME repo + store: a fresh process re-run. + val (ctx2, _) = reopen(dir, listener) + val second = runOnce()(using ctx2) + + assertEquals(first, "value-42") + assertEquals(second, "value-42") + assertEquals(runs.get(), 1, "body must run exactly once across both runs") + assert( + listener.events.contains( + OrcaEvent.Step("Resuming 'compute' from recorded result") + ), + "second run must report a resume" + ) + + test("a crash in a later stage leaves earlier stages committed and recorded"): + val (ctx, dir) = TestFlowControl.create(new EventDispatcher(Nil)) + given FlowControl = ctx + val _ = stage("stage one"): + os.write(dir / "one.txt", "1") + "one-result" + val countAfterOne = commitCount(dir) + + val _ = intercept[RuntimeException]: + stage[String]("stage two"): + os.write(dir / "two.txt", "2") + throw new RuntimeException("boom") + + // Stage one's commit + record survive the crash in stage two. + assertEquals(commitCount(dir), countAfterOne) + val ids = ctx.progressStore.load().get.entries.map(_.id) + assert(ids.contains("stage one#0"), "stage one must remain recorded") + assert( + !ids.contains("stage two#0"), + "the crashed stage must not be recorded" + ) + + test("a nested stage commits and records both the outer and inner stages"): + val (ctx, dir) = TestFlowControl.create(new EventDispatcher(Nil)) + given FlowControl = ctx + val before = commitCount(dir) + val _ = stage("outer"): + os.write(dir / "outer.txt", "o") + val _ = stage("inner"): + os.write(dir / "inner.txt", "i") + "inner-result" + "outer-result" + // Two stages → two commits (one per stage, inner committing before outer). + assertEquals(commitCount(dir), before + 2) + val ids = ctx.progressStore.load().get.entries.map(_.id) + assert(ids.contains("inner#0"), s"inner must be recorded; got $ids") + assert(ids.contains("outer#0"), s"outer must be recorded; got $ids") + + test("an undecodable stored entry re-runs the stage"): + val (ctx, dir) = TestFlowControl.create(new EventDispatcher(Nil)) + given FlowControl = ctx + val ran = new AtomicInteger(0) + // Seed an entry under id "typed#0" whose JSON cannot decode to Int. + locally: + given InStage = InStage.unsafe + ctx.progressStore.appendEntry( + StageEntry("typed#0", "typed", "\"not-an-int\"") + ) + val result = stage[Int]("typed"): + val _ = ran.incrementAndGet() + 99 + assertEquals(result, 99) + assertEquals(ran.get(), 1, "an undecodable entry must re-run the body") + + // --- helpers --- + + private def reopen( + dir: os.Path, + listener: OrcaListener + ): (TestFlowControl, os.Path) = + val git = new orca.tools.OsGitTool(dir) + val store = orca.progress.ProgressStore.default(dir, "p") + ( + new TestFlowControl( + new EventDispatcher(List(listener)), + git, + store, + "p" + ), + dir + ) + + private def commitCount(dir: os.Path): Int = + os.proc("git", "rev-list", "--count", "HEAD") + .call(cwd = dir) + .out + .text() + .trim + .toInt + + private def tracked(dir: os.Path): Set[String] = + os.proc("git", "ls-files") + .call(cwd = dir) + .out + .text() + .linesIterator + .toSet diff --git a/flow/src/test/scala/orca/TestFlowContext.scala b/flow/src/test/scala/orca/TestFlowContext.scala index c8a2e804..686a3415 100644 --- a/flow/src/test/scala/orca/TestFlowContext.scala +++ b/flow/src/test/scala/orca/TestFlowContext.scala @@ -1,10 +1,24 @@ package orca import orca.events.{EventDispatcher, OrcaEvent} -import orca.llm.{ClaudeTool, CodexTool, GeminiTool, OpencodeTool, PiTool} +import orca.agents.{ + Agent, + BackendTag, + ClaudeAgent, + CodexAgent, + GeminiAgent, + OpencodeAgent, + PiAgent +} +import orca.progress.{ProgressHeader, ProgressStore} +import orca.testkit.GitRepo import orca.tools.FsTool import orca.tools.GitTool import orca.tools.GitHubTool +import orca.tools.OsGitTool + +import java.util.concurrent.ConcurrentHashMap +import java.util.concurrent.atomic.AtomicInteger /** Minimal FlowContext stub for unit-testing stage/fail and other helpers that * only touch `emit` + `userPrompt`. Tool accessors are lazy so merely @@ -18,13 +32,66 @@ class TestFlowContext( private def stub(name: String) = throw new NotImplementedError(s"$name is not wired in TestFlowContext") - lazy val claude: ClaudeTool = stub("claude") - lazy val codex: CodexTool = stub("codex") - lazy val opencode: OpencodeTool = stub("opencode") - lazy val pi: PiTool = stub("pi") - lazy val gemini: GeminiTool = stub("gemini") + type LeadB = BackendTag.ClaudeCode.type + lazy val agent: Agent[LeadB] = stub("agent") + lazy val claude: ClaudeAgent = stub("claude") + lazy val codex: CodexAgent = stub("codex") + lazy val opencode: OpencodeAgent = stub("opencode") + lazy val pi: PiAgent = stub("pi") + lazy val gemini: GeminiAgent = stub("gemini") lazy val git: GitTool = stub("git") lazy val gh: GitHubTool = stub("gh") lazy val fs: FsTool = stub("fs") def emit(event: OrcaEvent): Unit = dispatcher.onEvent(event) + +/** A `FlowControl` backed by a real temp git repo and a real temp progress + * store, for exercising the `stage` runtime (commit + resume). Stubs the LLM + * tools (stages under test don't call them) but wires a real `OsGitTool` and a + * default `ProgressStore` over freshly-`git init`ed temp dirs. + */ +class TestFlowControl( + dispatcher: EventDispatcher, + val git: GitTool, + val progressStore: ProgressStore, + val userPrompt: String = "" +) extends FlowControl: + private def stub(name: String) = + throw new NotImplementedError(s"$name is not wired in TestFlowControl") + + type LeadB = BackendTag.ClaudeCode.type + lazy val agent: Agent[LeadB] = stub("agent") + lazy val claude: ClaudeAgent = stub("claude") + lazy val codex: CodexAgent = stub("codex") + lazy val opencode: OpencodeAgent = stub("opencode") + lazy val pi: PiAgent = stub("pi") + lazy val gemini: GeminiAgent = stub("gemini") + lazy val gh: GitHubTool = stub("gh") + lazy val fs: FsTool = stub("fs") + + def emit(event: OrcaEvent): Unit = dispatcher.onEvent(event) + + private val occurrences = new ConcurrentHashMap[String, AtomicInteger] + def nextOccurrence(stageName: String): Int = + occurrences + .computeIfAbsent(stageName, _ => new AtomicInteger(0)) + .getAndIncrement() + + private val sessionOccurrences = new AtomicInteger(0) + def nextSessionOccurrence(): Int = sessionOccurrences.getAndIncrement() + +object TestFlowControl: + /** Build a `TestFlowControl` over a fresh temp git repo (with one seed commit + * so HEAD exists) and a default progress store seeded with a header. Returns + * the control plus the repo dir for assertions on commits/files. + */ + def create( + dispatcher: EventDispatcher, + userPrompt: String = "p" + ): (TestFlowControl, os.Path) = + val dir = GitRepo.seeded() + val git = new OsGitTool(dir) + val store = ProgressStore.default(dir, userPrompt) + given InStage = InStage.unsafe + store.writeHeader(ProgressHeader("main", "feat/test", "deadbeef")) + (new TestFlowControl(dispatcher, git, store, userPrompt), dir) diff --git a/flow/src/test/scala/orca/plan/AssessThenPlanTest.scala b/flow/src/test/scala/orca/plan/AssessThenPlanTest.scala index 9da35c9d..43046518 100644 --- a/flow/src/test/scala/orca/plan/AssessThenPlanTest.scala +++ b/flow/src/test/scala/orca/plan/AssessThenPlanTest.scala @@ -1,14 +1,18 @@ package orca.plan import orca.events.EventDispatcher -import orca.llm.ToolSet +import orca.agents.ToolSet class AssessThenPlanTest extends munit.FunSuite: + // Planning helpers are now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe + private val samplePlan = Plan( epicId = "x", description = "d", - tasks = List(Task(Title("t1"), "body")) + tasks = List(Task(Title("t1"), "body")), + brief = "the brief" ) test("toVerdict maps verdict=proceed + plan to Verdict.Proceed"): @@ -70,7 +74,7 @@ class AssessThenPlanTest extends munit.FunSuite: ) val result = Plan.autonomous.assessThenPlan( "the report", - new CannedResultLlm(assessed) + new CannedResultAgent(assessed) ) // The verdict is carried alongside the session that produced it. assertEquals(result.sessionId.value, "stub-sid") @@ -81,7 +85,7 @@ class AssessThenPlanTest extends munit.FunSuite: test("Plan.autonomous planner runs NetworkOnly (reads + read-only network)"): given orca.FlowContext = new orca.TestFlowContext(new EventDispatcher(Nil)) - val stub = new CannedResultLlm( + val stub = new CannedResultAgent( AssessedPlan("proceed", Some(samplePlan), None, None) ) val _ = Plan.autonomous.assessThenPlan("the report", stub) @@ -95,6 +99,6 @@ class AssessThenPlanTest extends munit.FunSuite: val ex = intercept[orca.OrcaFlowException]: Plan.autonomous.assessThenPlan( "the report", - new CannedResultLlm(malformed) + new CannedResultAgent(malformed) ) assert(ex.getMessage.contains("no plan"), ex.getMessage) diff --git a/flow/src/test/scala/orca/plan/PersistentPlanTest.scala b/flow/src/test/scala/orca/plan/PersistentPlanTest.scala deleted file mode 100644 index f180bd45..00000000 --- a/flow/src/test/scala/orca/plan/PersistentPlanTest.scala +++ /dev/null @@ -1,318 +0,0 @@ -package orca.plan - -import orca.{FlowContext, TestFlowContext} -import orca.events.{EventDispatcher, OrcaEvent, OrcaListener} -import orca.tools.{GitTool, OsGitTool} - -import java.util.concurrent.atomic.AtomicReference - -class PersistentPlanTest extends munit.FunSuite: - - /** Init a fresh repo with one commit and wire a FlowContext whose `git` is a - * real `OsGitTool` rooted there. Returns the context + the dir + the - * recorded events. - */ - private def withRepoCtx( - body: (FlowContext, os.Path, AtomicReference[List[OrcaEvent]]) => Unit - ): Unit = - val dir = os.temp.dir() - val _ = os.proc("git", "init", "-b", "main").call(cwd = dir) - val _ = - os.proc("git", "config", "user.email", "test@example.com").call(cwd = dir) - val _ = os.proc("git", "config", "user.name", "Test").call(cwd = dir) - os.write(dir / "seed.txt", "seed") - val _ = os.proc("git", "add", "-A").call(cwd = dir) - val _ = os.proc("git", "commit", "-m", "seed").call(cwd = dir) - val seen = new AtomicReference[List[OrcaEvent]](Nil) - val listener: OrcaListener = (e: OrcaEvent) => - val _ = seen.updateAndGet(e :: _) - val dispatcher = new EventDispatcher(List(listener)) - val realGit = new OsGitTool(dir, listener) - val ctx = new TestFlowContext(dispatcher): - override lazy val git: GitTool = realGit - body(ctx, dir, seen) - - // --- defaultPath / hashUserPrompt --- - - test("defaultPath puts the hash into '.orca/plan-<hash>.md'"): - val expected = - os.Path("/tmp") / ".orca" / s"plan-${Plan.hashUserPrompt("hello")}.md" - assertEquals(Plan.defaultPath("hello", workDir = os.Path("/tmp")), expected) - - test("hashUserPrompt is stable and 12 hex chars wide"): - val a = Plan.hashUserPrompt("the same prompt") - val b = Plan.hashUserPrompt("the same prompt") - assertEquals(a, b) - assertEquals(a.length, 12) - assert(a.forall(c => "0123456789abcdef".contains(c)), a) - - test("hashUserPrompt differs for different prompts"): - assertNotEquals( - Plan.hashUserPrompt("alpha"), - Plan.hashUserPrompt("beta") - ) - - // --- recover --- - - test("recover returns None when no plan file exists"): - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - assertEquals(Plan.recover(dir / "missing.md"), None) - - test( - "recover restores an untracked plan file that ensureClean would stash away" - ): - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - val plan = Plan( - epicId = "feat-untracked", - description = "", - tasks = List(Task(Title("t1"), "body")) - ) - val planFile = dir / "untracked-plan.md" - // Write the plan file but never commit it — the crash-before-first- - // task-commit scenario the snapshot-restore guards against. - os.write(planFile, Plan.render(plan)) - - val recovered = Plan.recover(planFile).getOrElse(fail("expected a plan")) - assertEquals(recovered.epicId, "feat-untracked") - assert( - os.exists(planFile), - "plan file should have been restored after stash" - ) - assertEquals(os.read(planFile), Plan.render(plan)) - - test("recover parses the plan, stashes dirty changes, and switches branch"): - withRepoCtx: (ctx, dir, seen) => - given FlowContext = ctx - val plan = Plan( - epicId = "feat-x", - description = "", - tasks = List(Task(Title("t1"), "body")) - ) - val planFile = dir / "plan.md" - os.write(planFile, Plan.render(plan)) - // Stage `plan.md` into the initial commit so it's tracked, then make a - // dirty edit afterwards to exercise the stash path. - val _ = os.proc("git", "add", "-A").call(cwd = dir) - val _ = os.proc("git", "commit", "-m", "add plan").call(cwd = dir) - os.write.over(planFile, "dirty edit") - - val recovered = Plan.recover(planFile).getOrElse(fail("expected a plan")) - assertEquals(recovered.epicId, "feat-x") - assertEquals(ctx.git.currentBranch(), "feat-x") - // Stash was created — the working tree is back to the committed version. - assertEquals(os.read(planFile), Plan.render(plan)) - val steps = seen.get().reverse.collect { case OrcaEvent.Step(m) => m } - assert( - steps.exists(_.contains("Working tree wasn't clean")), - s"expected a stash Step; got: $steps" - ) - assert( - steps.exists(_.contains("Recovered plan")), - s"expected a recovery Step; got: $steps" - ) - - // --- recoverOrCreate --- - - test("recoverOrCreate returns the generated plan on the create path"): - // Pinning that the helper returns the plan from `generate` (no session - // id — session allocation is the caller's responsibility). - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - val plan = Plan( - epicId = "feat-r", - description = "", - tasks = List(Task(Title("t1"), "body")) - ) - val planFile = dir / "plan.md" - val returned = Plan.recoverOrCreate(planFile)(plan) - assertEquals(returned, plan) - assert(os.exists(planFile), "plan file should be persisted on create") - - test("recoverOrCreate on the recover path skips generate"): - // Pre-existing file → `generate` must not be evaluated. Pins the - // resume-cheaply contract. - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - val existing = Plan( - epicId = "feat-existing", - description = "", - tasks = List(Task(Title("t1"), "body")) - ) - val planFile = dir / "plan.md" - os.write(planFile, Plan.render(existing)) - val returned = Plan.recoverOrCreate(planFile): - fail("generate must not run when the plan file exists") - assertEquals(returned.epicId, "feat-existing") - - // --- implementTaskLoop --- - - test( - "implementTaskLoop on an all-complete plan skips the body and only does cleanup" - ): - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - val plan = Plan( - epicId = "feat-done", - description = "", - tasks = List(Task(Title("t1"), "body1", completed = true)) - ) - val planFile = dir / "plan.md" - os.write(planFile, Plan.render(plan)) - val _ = os.proc("git", "add", "-A").call(cwd = dir) - val _ = os.proc("git", "commit", "-m", "add done plan").call(cwd = dir) - - var bodyCalls = 0 - Plan.implementTaskLoop(planFile, plan): _ => - bodyCalls += 1 - - assertEquals( - bodyCalls, - 0, - "body should not run when every task is complete" - ) - assert(!os.exists(planFile), "plan file should be removed") - val commits = ctx.git.log(10).map(_.message) - assertEquals( - commits, - List("chore: remove plan.md", "add done plan", "seed"), - "only the cleanup commit should fire — no task commits" - ) - - test( - "implementTaskLoop runs body per task, persists completion, commits, removes file" - ): - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - val plan = Plan( - epicId = "feat-y", - description = "", - tasks = List( - Task(Title("t1"), "body1"), - Task(Title("t2"), "body2") - ) - ) - val planFile = dir / "plan.md" - os.write(planFile, Plan.render(plan)) - val _ = os.proc("git", "add", "-A").call(cwd = dir) - val _ = os.proc("git", "commit", "-m", "add plan").call(cwd = dir) - - val bodyRan = collection.mutable.ListBuffer[String]() - Plan.implementTaskLoop(planFile, plan): task => - bodyRan += task.title.value - // Simulate work the body would normally do — write a file so the - // task commit has something to record. - os.write(dir / s"${task.title.value}.txt", task.description) - - assertEquals(bodyRan.toList, List("t1", "t2")) - assert(!os.exists(planFile), "plan file should be removed at the end") - // Three commits added by implementTaskLoop: one per task + final cleanup. - val commits = ctx.git.log(10).map(_.message) - assertEquals( - commits.take(3), - List("chore: remove plan.md", "task: t2", "task: t1") - ) - // Pin the persistComplete contract: the `task: t1` commit (HEAD~2 from - // now, HEAD~1 from before the cleanup commit) must show t1 ticked and - // t2 still unchecked, and the `task: t2` commit must show both ticked. - val showT1 = - os.proc("git", "show", "HEAD~2:plan.md").call(cwd = dir).out.text() - val planAfterT1 = Plan.parse(showT1) - assertEquals(planAfterT1.tasks.map(_.completed), List(true, false)) - val showT2 = - os.proc("git", "show", "HEAD~1:plan.md").call(cwd = dir).out.text() - val planAfterT2 = Plan.parse(showT2) - assertEquals(planAfterT2.tasks.map(_.completed), List(true, true)) - - test( - "implementTaskLoop tolerates a no-op body (NothingToCommit is non-fatal)" - ): - // Regression: previously `commit().orThrow` would abort the loop on a - // body that produced no tracked change — even though the on-disk plan - // already had the tick written, so the next resume would skip the - // task. Now the loop advances cleanly past a no-op body. - withRepoCtx: (ctx, dir, seen) => - given FlowContext = ctx - val plan = Plan( - epicId = "feat-noop", - description = "", - tasks = List(Task(Title("t1"), "body1"), Task(Title("t2"), "body2")) - ) - - var bodyCalls = 0 - Plan.implementTaskLoop(plan): _ => - bodyCalls += 1 - // intentionally produce nothing the working tree would record - - assertEquals(bodyCalls, 2) - // Each no-op task emits a Step so the silent advance is observable. - val noOpSteps = seen.get.collect: - case OrcaEvent.Step(m) if m.contains("produced no tracked changes") => m - assertEquals(noOpSteps.size, 2) - - test( - "file-backed implementTaskLoop tolerates a no-op body with a gitignored plan file" - ): - // Regression: the .orca/ dir is conventionally gitignored, so a no-op - // body produces a working tree with no tracked changes — the per-task - // `commit().orThrow` would raise NothingToCommit even though - // `persistComplete` had already written the tick to disk. On the next - // resume the (still-on-disk, ticked) plan file would advance past a - // task whose body never actually committed. - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - // Mark `.orca/` as gitignored before the plan file is written, so the - // tick goes onto disk but git ignores it. Commit the .gitignore so it - // sticks for the rest of the test. - os.write(dir / ".gitignore", ".orca/\n") - val _ = os.proc("git", "add", "-A").call(cwd = dir) - val _ = os.proc("git", "commit", "-m", "ignore .orca").call(cwd = dir) - - val plan = Plan( - epicId = "feat-gitignored", - description = "", - tasks = List(Task(Title("t1"), "body1"), Task(Title("t2"), "body2")) - ) - val planFile = dir / ".orca" / "plan.md" - os.write(planFile, Plan.render(plan), createFolders = true) - - var bodyCalls = 0 - Plan.implementTaskLoop(planFile, plan): _ => - bodyCalls += 1 - // no tracked output - - // Both bodies ran — the loop didn't abort on NothingToCommit even - // though every per-task commit had nothing to record. The commit - // log being unchanged is the load-bearing assertion: it proves - // both that no `NothingToCommit` abort fired AND that no spurious - // per-task commit landed. - assertEquals(bodyCalls, 2) - val commits = ctx.git.log(10).map(_.message) - assertEquals(commits, List("ignore .orca", "seed")) - - test( - "in-memory implementTaskLoop runs body + commits per task, no file activity" - ): - withRepoCtx: (ctx, dir, _) => - given FlowContext = ctx - val plan = Plan( - epicId = "feat-mem", - description = "", - tasks = List( - Task(Title("t1"), "body1"), - Task(Title("t2"), "body2") - ) - ) - - val bodyRan = collection.mutable.ListBuffer[String]() - Plan.implementTaskLoop(plan): task => - bodyRan += task.title.value - os.write(dir / s"${task.title.value}.txt", task.description) - - assertEquals(bodyRan.toList, List("t1", "t2")) - // No plan file is created or removed by the in-memory variant. - assert(!os.exists(dir / ".orca")) - // Two per-task commits; no chore-remove since there's no file. - val commits = ctx.git.log(10).map(_.message) - assertEquals(commits.take(3), List("task: t2", "task: t1", "seed")) diff --git a/flow/src/test/scala/orca/plan/PlanGridTest.scala b/flow/src/test/scala/orca/plan/PlanGridTest.scala index aef98e6c..84101e85 100644 --- a/flow/src/test/scala/orca/plan/PlanGridTest.scala +++ b/flow/src/test/scala/orca/plan/PlanGridTest.scala @@ -1,7 +1,7 @@ package orca.plan import orca.events.EventDispatcher -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} /** Runtime wiring of the autonomous planning grid: each operation pairs its * result with the producing session, and `triage` converts the wire @@ -15,14 +15,19 @@ class PlanGridTest extends munit.FunSuite: private given orca.FlowContext = new orca.TestFlowContext(new EventDispatcher(Nil)) + // Planning helpers are now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe + private val samplePlan = Plan( epicId = "x", description = "d", - tasks = List(Task(Title("t1"), "body")) + tasks = List(Task(Title("t1"), "body")), + brief = "the brief" ) test("autonomous.from pairs the plan with the producing session"): - val result = Plan.autonomous.from("prompt", new CannedResultLlm(samplePlan)) + val result = + Plan.autonomous.from("prompt", new CannedResultAgent(samplePlan)) assertEquals(result.sessionId.value, "stub-sid") assertEquals(result.value, samplePlan) @@ -36,7 +41,7 @@ class PlanGridTest extends munit.FunSuite: branchName = "fix-foo", summary = "Foo overflows" ) - val result = Plan.autonomous.triage("report", new CannedResultLlm(wire)) + val result = Plan.autonomous.triage("report", new CannedResultAgent(wire)) assertEquals(result.sessionId.value, "stub-sid") assertEquals( result.value, @@ -47,29 +52,12 @@ class PlanGridTest extends munit.FunSuite: ) ) - // --- post-planning steps (reviewed / briefed) on the planning session --- + // --- post-planning step (reviewed) on the planning session --- private def sessioned[A](value: A): Sessioned[BackendTag.ClaudeCode.type, A] = Sessioned(SessionId[BackendTag.ClaudeCode.type]("planner-sid"), value) - test("reviewed on a bare plan returns the improved plan"): - val improved = samplePlan.copy(description = "tighter") - val result = sessioned(samplePlan).reviewed(new CannedResultLlm(improved)) + test("reviewed returns the improved plan, brief included"): + val improved = samplePlan.copy(description = "tighter", brief = "sharper") + val result = sessioned(samplePlan).reviewed(new CannedResultAgent(improved)) assertEquals(result.value, improved) - - test("briefed attaches the brief and threads the planning session"): - val result = sessioned(samplePlan).briefed(new CannedTextLlm("the brief")) - assertEquals(result.value, PlanWithBrief(samplePlan, "the brief")) - assertEquals(result.sessionId.value, "planner-sid") - - test("reviewed on a PlanWithBrief reviews plan and brief together"): - val improved = - PlanWithBrief(samplePlan.copy(description = "tighter"), "sharper brief") - val result = - sessioned(PlanWithBrief(samplePlan, "brief")) - .reviewed(new CannedResultLlm(improved)) - assertEquals(result.value, improved) - - test("briefed fails when the brief turn produces a blank brief"): - val _ = intercept[orca.OrcaFlowException]: - sessioned(samplePlan).briefed(new CannedTextLlm(" ")) diff --git a/flow/src/test/scala/orca/plan/PlanTest.scala b/flow/src/test/scala/orca/plan/PlanTest.scala index dc258af7..dcb1557b 100644 --- a/flow/src/test/scala/orca/plan/PlanTest.scala +++ b/flow/src/test/scala/orca/plan/PlanTest.scala @@ -1,8 +1,7 @@ package orca.plan import orca.plan.Title -import orca.llm.{JsonData} -import orca.events.{EventDispatcher, OrcaEvent, OrcaListener} +import orca.agents.JsonData import com.github.plokhotnyuk.jsoniter_scala.core.{ readFromString, @@ -11,29 +10,9 @@ import com.github.plokhotnyuk.jsoniter_scala.core.{ class PlanTest extends munit.FunSuite: - private val sample = - """# Plan: add-divide-method - | - |Extend Calculator with safe integer division. The current API - |covers add/subtract/multiply but not divide, and callers have - |started rolling their own with inconsistent zero-handling. - | - |## Task: add-divide - |Status: [ ] - | - |Add a `divide(int a, int b)` method to Calculator that returns - |`a / b` and throws `IllegalArgumentException` for `b == 0`. - | - |## Task: add-divide-test - |Status: [x] - | - |Add unit tests covering the happy path and the zero-divisor - |case. - |""".stripMargin + // --- JSON — the structured-output / stage-result path --- - // --- JSON / Announce — covers the in-memory `Plan.from` path --- - - test("Plan round-trips through JSON via the JsonData codec"): + test("Plan round-trips through JSON via the JsonData codec (brief included)"): val plan = Plan( epicId = "calculator-features", description = "Round out Calculator with the missing arithmetic ops.", @@ -47,7 +26,8 @@ class PlanTest extends munit.FunSuite: description = "Add a divide(int, int) method with a zero-divisor guard." ) - ) + ), + brief = "Calculator lives in core/Calculator.scala; follow its style." ) given codec : com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec[Plan] = @@ -63,9 +43,10 @@ class PlanTest extends munit.FunSuite: tasks = List( Task(Title("Add feature A"), "do A"), Task(Title("Add feature B"), "do B") - ) + ), + brief = "" ) - val msg = summon[orca.llm.Announce[Plan]] + val msg = summon[orca.agents.Announce[Plan]] .message(plan) .getOrElse(fail("expected a non-empty announce message")) assert(msg.startsWith("Planned 2 tasks on branch 'feat-pair'")) @@ -74,214 +55,67 @@ class PlanTest extends munit.FunSuite: test("Announce[Plan] returns None for an empty plan (no Step emitted)"): assertEquals( - summon[orca.llm.Announce[Plan]].message(Plan("empty", "", Nil)), + summon[orca.agents.Announce[Plan]].message(Plan("empty", "", Nil, "")), None ) - // --- Markdown parser / renderer — covers the persisted path --- - - test("parse extracts the branch name from the H1"): - assertEquals(Plan.parse(sample).epicId, "add-divide-method") - - test("parse extracts the epic description between the H1 and the first task"): - val description = Plan.parse(sample).description - assert(description.startsWith("Extend Calculator")) - assert(description.contains("inconsistent zero-handling")) - // The description must not bleed into the first task block. - assert(!description.contains("## Task")) - - test("parse yields an empty description when the file has no preamble"): - val noPreamble = - """# Plan: x - | - |## Task: t - |Status: [ ] - | - |body - |""".stripMargin - assertEquals(Plan.parse(noPreamble).description, "") - - test("parse splits the file into tasks and reads each status checkbox"): - val plan = Plan.parse(sample) - assertEquals(plan.tasks.size, 2) - assertEquals(plan.tasks.head.title, Title("add-divide")) - assertEquals(plan.tasks.head.completed, false) - assertEquals(plan.tasks(1).title, Title("add-divide-test")) - assertEquals(plan.tasks(1).completed, true) - - test("parse keeps the multi-line description body intact"): - val description = Plan.parse(sample).tasks.head.description - assert(description.startsWith("Add a `divide")) - assert(description.contains("IllegalArgumentException")) - - test("render + parse round-trips the plan"): - val original = Plan.parse(sample) - assertEquals(Plan.parse(Plan.render(original)), original) - - // --- PlanWithBrief: the trailing ## Brief section --- - - private val samplePlan = - Plan("epic", "desc", List(Task(Title("t"), "implement t"))) - - test("parse without a ## Brief section yields a bare Plan"): - assert(Plan.parse(sample).isInstanceOf[Plan]) - - test("parse with a ## Brief section yields a PlanWithBrief"): - Plan.parse(sample + "\n## Brief\n\nUse the existing Foo helper.\n") match - case PlanWithBrief(plan, brief) => - assertEquals(plan.tasks.size, 2) - assertEquals(brief, "Use the existing Foo helper.") - case other => fail(s"expected PlanWithBrief, got $other") - - test("render + parse round-trips a PlanWithBrief, brief preserved"): - val pwb = PlanWithBrief(samplePlan, "Build on bar/Baz.scala.") - assertEquals(Plan.parse(Plan.render(pwb)), pwb) - - test("a literal ## Brief line in the description does not swallow the tasks"): - val tricky = - """# Plan: x - | - |Context. - |## Brief - |more context - | - |## Task: t - |Status: [ ] - | - |body - |""".stripMargin - Plan.parse(tricky) match - case p: Plan => - assertEquals(p.tasks.size, 1) - assert(p.description.contains("## Brief")) - case other => fail(s"expected a bare Plan, got $other") + // --- Markdown renderer (cosmetic checklist; never parsed back, ADR 0018 §2.8) --- + + private val samplePlan = Plan( + epicId = "add-divide-method", + description = "Extend Calculator with safe integer division.", + tasks = List( + Task(Title("add-divide"), "Add a divide method.", completed = false), + Task(Title("add-divide-test"), "Add unit tests.", completed = true) + ), + brief = "Build on core/Calculator.scala." + ) + + test("render emits the H1, description, and a checkbox per task"): + val md = Plan.render(samplePlan) + assert(md.startsWith("# Plan: add-divide-method"), md) + assert(md.contains("Extend Calculator"), md) + assert(md.contains("## Task: add-divide\nStatus: [ ]"), md) + assert(md.contains("## Task: add-divide-test\nStatus: [x]"), md) + + test("render appends the brief as a trailing ## Brief section"): + assert( + Plan + .render(samplePlan) + .contains("## Brief\n\nBuild on core/Calculator.scala."), + Plan.render(samplePlan) + ) - test("a brief is kept verbatim even if it contains ## Task lines"): - // The brief is split off before task parsing, so its markdown can't be - // mistaken for plan tasks. - val brief = "## Task: not a real task\nsome notes" - Plan.parse(Plan.render(PlanWithBrief(samplePlan, brief))) match - case PlanWithBrief(plan, b) => - assertEquals(plan.tasks.size, 1) - assertEquals(b, brief) - case other => fail(s"expected PlanWithBrief, got $other") + test("render omits the ## Brief section when the brief is empty"): + assert(!Plan.render(samplePlan.copy(brief = "")).contains("## Brief")) - test("markComplete on a PlanWithBrief flips the task and keeps the brief"): - val updated = PlanWithBrief(samplePlan, "CONTEXT").markComplete(Title("t")) - assertEquals(updated.tasks.head.completed, true) - assertEquals(updated.brief, "CONTEXT") + test("taskPrompt prepends the brief when non-empty"): + val task = samplePlan.tasks.head + assertEquals( + samplePlan.copy(brief = "CONTEXT").taskPrompt(task), + "CONTEXT\n\n---\n\nAdd a divide method." + ) - test("taskPrompt prepends the brief only for a PlanWithBrief"): + test("taskPrompt returns description verbatim when brief is empty"): val task = samplePlan.tasks.head - assertEquals(samplePlan.taskPrompt(task), "implement t") assertEquals( - PlanWithBrief(samplePlan, "CONTEXT").taskPrompt(task), - "CONTEXT\n\n---\n\nimplement t" + samplePlan.copy(brief = "").taskPrompt(task), + "Add a divide method." ) test("markComplete flips one task's checkbox without touching others"): - val plan = Plan.parse(sample) - val updated = plan.markComplete(Title("add-divide")) + val updated = samplePlan.markComplete(Title("add-divide")) assertEquals(updated.tasks.head.completed, true) assertEquals(updated.tasks(1).completed, true) // markComplete on a title that doesn't exist is a no-op. - assertEquals(plan.markComplete(Title("ghost")), plan) + assertEquals(samplePlan.markComplete(Title("ghost")), samplePlan) test("firstIncomplete returns the first task with [ ] in declaration order"): - val plan = Plan.parse(sample) - assertEquals(plan.firstIncomplete.map(_.title), Some(Title("add-divide"))) - assertEquals(plan.markComplete(Title("add-divide")).firstIncomplete, None) - - test("parse throws on a missing # Plan header"): - intercept[PlanParseException]: - Plan.parse("## Task: orphan\nStatus: [ ]\n\nbody\n") - - test("parse throws on a task missing the Status line"): - val bad = - """# Plan: x - | - |## Task: t - | - |body - |""".stripMargin - intercept[PlanParseException](Plan.parse(bad)) - - test("parse throws on an unrecognised status checkbox"): - val bad = - """# Plan: x - | - |## Task: t - |Status: [?] - | - |body - |""".stripMargin - intercept[PlanParseException](Plan.parse(bad)) - - test("parse throws on a plan with no tasks"): - intercept[PlanParseException](Plan.parse("# Plan: empty\n")) - - test("parse normalises CRLF line endings and a leading BOM"): - val crlf = sample.replace("\n", "\r\n") - val plan = Plan.parse("" + crlf) - assertEquals(plan.epicId, "add-divide-method") - assertEquals(plan.tasks.size, 2) - - test("parse throws on a task with empty prompt"): - val bad = - """# Plan: x - | - |## Task: t - |Status: [ ] - |""".stripMargin - intercept[PlanParseException](Plan.parse(bad)) - - // --- autonomous.loadOrGenerate / persistComplete — resume path --- - - test( - "autonomous.loadOrGenerate parses and reuses an existing file (no LLM call)" - ): - val seen = - new java.util.concurrent.atomic.AtomicReference[List[ - orca.events.OrcaEvent - ]](Nil) - val listener = new orca.events.OrcaListener: - def onEvent(event: orca.events.OrcaEvent): Unit = - val _ = seen.updateAndGet(event :: _) - given orca.FlowContext = new orca.TestFlowContext( - new orca.events.EventDispatcher(List(listener)) - ) - val tmp = os.temp(suffix = ".md") - os.write.over(tmp, sample) - val llm = - new ExplodingLlm("loadOrGenerate must not call the LLM when file exists") - val plan = Plan.autonomous.loadOrGenerate(tmp, "ignored", llm) - assertEquals(plan.epicId, "add-divide-method") - assert( - seen.get().exists { - case orca.events.OrcaEvent.Step(msg) => - msg.contains("Reusing existing plan") - case _ => false - } + assertEquals( + samplePlan.firstIncomplete.map(_.title), + Some(Title("add-divide")) ) - - test("autonomous.loadOrGenerate writes a new file when none exists"): - given orca.FlowContext = - new orca.TestFlowContext(new orca.events.EventDispatcher(Nil)) - val target = os.temp.dir() / "dev.md" - val expected = Plan.parse(sample) - val plan = Plan.autonomous.loadOrGenerate( - target, - "Add a divide method", - new CannedResultLlm(expected) + assertEquals( + samplePlan.markComplete(Title("add-divide")).firstIncomplete, + None ) - assert(os.exists(target)) - assertEquals(plan, expected) - assertEquals(Plan.parse(os.read(target)), expected) - - test("persistComplete updates the on-disk plan"): - val tmp = os.temp(suffix = ".md") - os.write.over(tmp, sample) - Plan.persistComplete(tmp, Title("add-divide")) - val reread = Plan.parse(os.read(tmp)) - assertEquals(reread.tasks.head.completed, true) - assertEquals(reread.tasks(1).completed, true) diff --git a/flow/src/test/scala/orca/plan/StubAgents.scala b/flow/src/test/scala/orca/plan/StubAgents.scala new file mode 100644 index 00000000..74b7d741 --- /dev/null +++ b/flow/src/test/scala/orca/plan/StubAgents.scala @@ -0,0 +1,54 @@ +package orca.plan + +import orca.agents.{ + AgentInput, + Announce, + AutonomousAgentCall, + AutonomousTextCall, + BackendTag, + InteractiveAgentCall, + JsonData, + AgentCall, + AgentConfig, + Agent, + SessionId, + ToolSet +} + +/** Test double whose `resultAs[O].autonomous.run` returns a pre-built `value` + * (cast to `O`) paired with a fixed session id. Other call shapes throw so + * accidental use surfaces immediately. One stub serves every autonomous + * planning operation — pass a `Plan`, `AssessedPlan`, or `BugTriage`. + */ +private[plan] class CannedResultAgent[T](value: T) + extends Agent[BackendTag.ClaudeCode.type]: + val name: String = "stub" + + /** Records the most recent `withTools` tier so tests can assert which + * capability a helper selected (e.g. planners use `NetworkOnly`). + */ + var lastToolSet: Option[ToolSet] = None + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = + lastToolSet = Some(tools) + this + + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + new AgentCall[BackendTag.ClaudeCode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.ClaudeCode.type, O] = + new AutonomousAgentCall[BackendTag.ClaudeCode.type, O]: + def run[I: AgentInput]( + input: I, + session: SessionId[BackendTag.ClaudeCode.type], + config: AgentConfig, + emitPrompt: Boolean + )(using orca.InStage): (SessionId[BackendTag.ClaudeCode.type], O) = + ( + SessionId[BackendTag.ClaudeCode.type]("stub-sid"), + value.asInstanceOf[O] + ) + def interactive: InteractiveAgentCall[BackendTag.ClaudeCode.type, O] = ??? diff --git a/flow/src/test/scala/orca/plan/StubLlmTools.scala b/flow/src/test/scala/orca/plan/StubLlmTools.scala deleted file mode 100644 index 61da385e..00000000 --- a/flow/src/test/scala/orca/plan/StubLlmTools.scala +++ /dev/null @@ -1,89 +0,0 @@ -package orca.plan - -import orca.llm.{ - AgentInput, - Announce, - AutonomousLlmCall, - AutonomousTextCall, - BackendTag, - InteractiveLlmCall, - JsonData, - LlmCall, - LlmConfig, - LlmTool, - SessionId, - ToolSet -} - -/** Test double whose `resultAs[O].autonomous.run` returns a pre-built `value` - * (cast to `O`) paired with a fixed session id. Other call shapes throw so - * accidental use surfaces immediately. One stub serves every autonomous - * planning operation — pass a `Plan`, `AssessedPlan`, or `BugTriage`. - */ -private[plan] class CannedResultLlm[T](value: T) - extends LlmTool[BackendTag.ClaudeCode.type]: - val name: String = "stub" - - /** Records the most recent `withTools` tier so tests can assert which - * capability a helper selected (e.g. planners use `NetworkOnly`). - */ - var lastToolSet: Option[ToolSet] = None - def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = - lastToolSet = Some(tools) - this - - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.ClaudeCode.type, O] = - new LlmCall[BackendTag.ClaudeCode.type, O]: - val autonomous: AutonomousLlmCall[BackendTag.ClaudeCode.type, O] = - new AutonomousLlmCall[BackendTag.ClaudeCode.type, O]: - def run[I: AgentInput]( - input: I, - session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, - emitPrompt: Boolean - ): (SessionId[BackendTag.ClaudeCode.type], O) = - ( - SessionId[BackendTag.ClaudeCode.type]("stub-sid"), - value.asInstanceOf[O] - ) - def interactive: InteractiveLlmCall[BackendTag.ClaudeCode.type, O] = ??? - -/** Test double: `autonomous.run` returns a fixed string with the session it was - * handed. `resultAs` throws. - */ -private[plan] class CannedTextLlm(text: String) - extends LlmTool[BackendTag.ClaudeCode.type]: - val name: String = "stub-text" - def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = - new AutonomousTextCall[BackendTag.ClaudeCode.type]: - def run( - prompt: String, - session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, - emitPrompt: Boolean - ): (SessionId[BackendTag.ClaudeCode.type], String) = (session, text) - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.ClaudeCode.type, O] = - ??? - -/** Test double that throws on every method — used to assert that a code path - * doesn't call the LLM. - */ -private[plan] class ExplodingLlm(reason: String) - extends LlmTool[BackendTag.ClaudeCode.type]: - val name: String = "exploding" - def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = - throw new AssertionError(reason) - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.ClaudeCode.type, O] = - throw new AssertionError(reason) diff --git a/flow/src/test/scala/orca/progress/ProgressLogTest.scala b/flow/src/test/scala/orca/progress/ProgressLogTest.scala new file mode 100644 index 00000000..f37949c5 --- /dev/null +++ b/flow/src/test/scala/orca/progress/ProgressLogTest.scala @@ -0,0 +1,94 @@ +package orca.progress + +import com.github.plokhotnyuk.jsoniter_scala.core.{ + readFromString, + writeToString +} +import munit.FunSuite +import orca.agents.JsonData + +class ProgressLogTest extends FunSuite: + + private def roundTrip[A](value: A)(using jd: JsonData[A]): A = + readFromString[A](writeToString(value)(using jd.codec))(using jd.codec) + + test("ProgressLog round-trips through JsonData codec"): + val log = ProgressLog( + header = ProgressHeader( + startingBranch = "main", + branch = "feat/my-feature", + promptHash = "abc123def456" + ), + entries = List( + StageEntry( + id = "stage-1", + name = "Analyse", + resultJson = """{"ok":true}""" + ), + StageEntry( + id = "stage-2", + name = "Implement", + resultJson = """{"files":["a.scala"]}""" + ) + ) + ) + assertEquals(roundTrip(log), log) + + test("ProgressLog with sessions round-trips through JsonData codec"): + val log = ProgressLog( + header = ProgressHeader( + startingBranch = "main", + branch = "feat/sessions", + promptHash = "abc123def456" + ), + entries = List( + StageEntry( + id = "stage-1", + name = "Plan", + resultJson = """{"ok":true}""" + ) + ), + sessions = List( + SessionRecord(index = 0, id = "sess-uuid-1", seed = "plan brief"), + SessionRecord(index = 1, id = "sess-uuid-2", seed = "other seed") + ) + ) + assertEquals(roundTrip(log), log) + + test("SessionRecord round-trips with resumeWireId = Some(...)"): + val log = ProgressLog( + header = ProgressHeader("main", "feat/server", "abc123"), + entries = Nil, + sessions = List( + SessionRecord( + index = 0, + id = "client-uuid", + seed = "brief", + resumeWireId = Some("ses_server_123") + ) + ) + ) + assertEquals(roundTrip(log), log) + + test( + "SessionRecord JSON without a resumeWireId field decodes to None (back-compat)" + ): + // JSON produced before the resumeWireId field existed: the record has only + // index/id/seed. The lenient ProgressLog codec must default it to None. + val oldJson = + """{"header":{"startingBranch":"main","branch":"feat/old","promptHash":"abc"},""" + + """"entries":[],"sessions":[{"index":0,"id":"u","seed":"s"}]}""" + val codec = summon[JsonData[ProgressLog]].codec + val decoded = readFromString[ProgressLog](oldJson)(using codec) + assertEquals(decoded.sessions.head.resumeWireId, None) + + test( + "ProgressLog JSON without a sessions field decodes to empty sessions list (back-compat)" + ): + // JSON produced by the old format (before sessions field existed) + val oldJson = + """{"header":{"startingBranch":"main","branch":"feat/old","promptHash":"abc123"},"entries":[]}""" + val codec = summon[JsonData[ProgressLog]].codec + val decoded = readFromString[ProgressLog](oldJson)(using codec) + assertEquals(decoded.sessions, Nil) + assertEquals(decoded.header.branch, "feat/old") diff --git a/flow/src/test/scala/orca/progress/ProgressStoreTest.scala b/flow/src/test/scala/orca/progress/ProgressStoreTest.scala new file mode 100644 index 00000000..7b99929d --- /dev/null +++ b/flow/src/test/scala/orca/progress/ProgressStoreTest.scala @@ -0,0 +1,122 @@ +package orca.progress + +import munit.FunSuite +import orca.InStage + +class ProgressStoreTest extends FunSuite: + + // All mutating calls require an InStage token; mint one for the test suite. + given InStage = InStage.unsafe + + private val header = ProgressHeader( + startingBranch = "main", + branch = "feat/some-feature", + promptHash = "abc123def456" + ) + + test("writeHeader then load returns the header with empty entries"): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "my prompt") + store.writeHeader(header) + val loaded = store.load() + assertEquals(loaded, Some(ProgressLog(header, Nil))) + + test( + "appendEntry with same id upserts (last write wins), different id appends" + ): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "my prompt") + store.writeHeader(header) + + val a = + StageEntry(id = "stage-1", name = "First", resultJson = """{"v":1}""") + val aPrime = + StageEntry(id = "stage-1", name = "First", resultJson = """{"v":2}""") + val b = + StageEntry(id = "stage-2", name = "Second", resultJson = """{"v":3}""") + + store.appendEntry(a) + store.appendEntry(aPrime) // same id — should replace a + store.appendEntry(b) // different id — should append + + val loaded = store.load() + assertEquals(loaded.map(_.entries), Some(List(aPrime, b))) + + test("load returns None when no file exists"): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "my prompt") + assertEquals(store.load(), None: Option[ProgressLog]) + + test("load returns None for a corrupt file (no throw)"): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "my prompt") + // Manually write garbage to the expected path location + val path = workDir / ".orca" + os.makeDir.all(path) + // Write garbage to every possible progress file via direct os.write + // The store's path is deterministic for a given prompt, so we can + // reconstruct it by writing to any file there — but we need the exact path. + // Instead, call writeHeader so the file exists, then overwrite with garbage. + store.writeHeader(header) + // Overwrite with non-JSON content + val files = os.list(path).filter(_.last.startsWith("progress-")) + assert(files.nonEmpty, "expected at least one progress file") + os.write.over(files.head, "not json {{{") + assertEquals(store.load(), None: Option[ProgressLog]) + + test("default path is <workDir>/.orca/progress-<12hexchars>.json"): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "test prompt") + store.writeHeader(header) + val orcaDir = workDir / ".orca" + val files = os.list(orcaDir).filter(_.last.startsWith("progress-")) + assert(files.size == 1, s"expected exactly one progress file, got $files") + val filename = files.head.last + // filename must match progress-<12 hex chars>.json + assert( + filename.matches("progress-[0-9a-f]{12}\\.json"), + s"unexpected filename: $filename" + ) + + test("default path is deterministic for a given prompt"): + val workDir1 = os.temp.dir() + val workDir2 = os.temp.dir() + val store1 = ProgressStore.default(workDir1, "deterministic prompt") + val store2 = ProgressStore.default(workDir2, "deterministic prompt") + store1.writeHeader(header) + store2.writeHeader(header) + val name1 = os.list(workDir1 / ".orca").head.last + val name2 = os.list(workDir2 / ".orca").head.last + assertEquals(name1, name2) + + test("upsertSession writes a session record and load shows it"): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "my prompt") + store.writeHeader(header) + val record = + SessionRecord(index = 0, id = "session-uuid-1", seed = "plan brief") + store.upsertSession(record) + val loaded = store.load() + assertEquals(loaded.map(_.sessions), Some(List(record))) + + test("upsertSession with same index replaces the record (last wins)"): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "my prompt") + store.writeHeader(header) + val first = SessionRecord(index = 0, id = "first-uuid", seed = "old seed") + val second = SessionRecord(index = 0, id = "second-uuid", seed = "new seed") + store.upsertSession(first) + store.upsertSession(second) + val loaded = store.load() + assertEquals(loaded.map(_.sessions), Some(List(second))) + + test("upsertSession with different indices results in two records"): + val workDir = os.temp.dir() + val store = ProgressStore.default(workDir, "my prompt") + store.writeHeader(header) + val r0 = SessionRecord(index = 0, id = "uuid-0", seed = "seed zero") + val r1 = SessionRecord(index = 1, id = "uuid-1", seed = "seed one") + store.upsertSession(r0) + store.upsertSession(r1) + val loaded = store.load() + assertEquals(loaded.map(_.sessions), Some(List(r0, r1))) diff --git a/flow/src/test/scala/orca/progress/RecoveryCheckTest.scala b/flow/src/test/scala/orca/progress/RecoveryCheckTest.scala new file mode 100644 index 00000000..3a96c287 --- /dev/null +++ b/flow/src/test/scala/orca/progress/RecoveryCheckTest.scala @@ -0,0 +1,94 @@ +package orca.progress + +import munit.FunSuite + +class RecoveryCheckTest extends FunSuite: + + test("isSafeBranchRef accepts slug names and issue branches"): + assert(RecoveryCheck.isSafeBranchRef("add-foo")) + assert(RecoveryCheck.isSafeBranchRef("fix/issue-42")) + assert(RecoveryCheck.isSafeBranchRef("flow-1a2b3c4d")) + + test("isSafeBranchRef rejects empty, leading-dash, traversal, and spaces"): + assert(!RecoveryCheck.isSafeBranchRef("")) + assert(!RecoveryCheck.isSafeBranchRef("-x")) + assert(!RecoveryCheck.isSafeBranchRef("a/..")) + assert(!RecoveryCheck.isSafeBranchRef("a b")) + assert(!RecoveryCheck.isSafeBranchRef("Feat")) + assert(!RecoveryCheck.isSafeBranchRef("a/")) + + test("validateHeader rejects the main/master floor regardless of the set"): + val prompt = "do the thing" + for protectedName <- List("main", "master", "MAIN", "Master") do + val header = ProgressHeader( + startingBranch = "main", + branch = protectedName, + promptHash = ProgressStore.hashPrompt(prompt) + ) + assert( + RecoveryCheck.validateHeader(header, prompt, Set.empty).isLeft, + s"$protectedName must be rejected as a feature branch" + ) + + test("validateHeader rejects the repo's actual default branch"): + val prompt = "do the thing" + // A repo whose default is `trunk` (not main/master): a header naming it as + // a feature branch must be refused when `trunk` is in the protected set. + // `trunk` is lowercase + slug-valid, so it passes `isSafeBranchRef` and + // reaches the protected-branch check — proving that code path fires (rather + // than an incidental safe-ref rejection on a mixed-case name). + val header = ProgressHeader( + startingBranch = "trunk", + branch = "trunk", + promptHash = ProgressStore.hashPrompt(prompt) + ) + val rejected = RecoveryCheck.validateHeader(header, prompt, Set("trunk")) + assert( + rejected.left.exists(_.contains("protected")), + s"the default branch must be rejected as PROTECTED, got: $rejected" + ) + // Case-insensitive: a mixed-case entry in the protected set still matches. + assert( + RecoveryCheck + .validateHeader(header, prompt, Set("Trunk")) + .left + .exists(_.contains("protected")), + "protected-branch match must be case-insensitive" + ) + // ...but a normal feature branch still passes with the same set. + val ok = header.copy(branch = "feat/do-the-thing") + assertEquals( + RecoveryCheck.validateHeader(ok, prompt, Set("trunk")), + Right(()) + ) + + test("validateHeader allows a protected startingBranch"): + val prompt = "do the thing" + val header = ProgressHeader( + startingBranch = "main", + branch = "feat/do-the-thing", + promptHash = ProgressStore.hashPrompt(prompt) + ) + assertEquals( + RecoveryCheck.validateHeader(header, prompt, Set.empty), + Right(()) + ) + + test("validateHeader rejects a prompt-hash mismatch"): + val header = ProgressHeader( + startingBranch = "main", + branch = "feat/do-the-thing", + promptHash = ProgressStore.hashPrompt("a different prompt") + ) + assert( + RecoveryCheck.validateHeader(header, "do the thing", Set.empty).isLeft + ) + + test("validateHeader rejects an unsafe startingBranch"): + val prompt = "do the thing" + val header = ProgressHeader( + startingBranch = "-evil", + branch = "feat/do-the-thing", + promptHash = ProgressStore.hashPrompt(prompt) + ) + assert(RecoveryCheck.validateHeader(header, prompt, Set.empty).isLeft) diff --git a/flow/src/test/scala/orca/review/AllReviewersTest.scala b/flow/src/test/scala/orca/review/AllReviewersTest.scala index 241c8b4f..1051773d 100644 --- a/flow/src/test/scala/orca/review/AllReviewersTest.scala +++ b/flow/src/test/scala/orca/review/AllReviewersTest.scala @@ -1,19 +1,19 @@ package orca.review -import orca.llm.{ +import orca.agents.{ Announce, AutonomousTextCall, BackendTag, JsonData, - LlmCall, - LlmConfig, - LlmTool, + AgentCall, + AgentConfig, + Agent, ToolSet } class AllReviewersTest extends munit.FunSuite: - /** LlmTool that records every `withSystemPrompt` call into a shared buffer - * (so renamed copies still feed the same record) and otherwise behaves as a + /** Agent that records every `withSystemPrompt` call into a shared buffer (so + * renamed copies still feed the same record) and otherwise behaves as a * no-op stub. `withName` returns a fresh instance carrying the new name so * `allReviewers` can tag each reviewer. */ @@ -21,18 +21,18 @@ class AllReviewersTest extends munit.FunSuite: val name: String = "base", systemPromptsSeen: collection.mutable.ListBuffer[String] = collection.mutable.ListBuffer.empty - ) extends LlmTool[BackendTag.ClaudeCode.type]: + ) extends Agent[BackendTag.ClaudeCode.type]: def seen: List[String] = systemPromptsSeen.toList def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = val _ = systemPromptsSeen += p this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = new RecordingTool(n, systemPromptsSeen) - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this def resultAs[O: JsonData: Announce] - : LlmCall[BackendTag.ClaudeCode.type, O] = + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? test("allReviewers exposes the full canonical reviewer set"): diff --git a/flow/src/test/scala/orca/review/FixLoopTest.scala b/flow/src/test/scala/orca/review/FixLoopTest.scala index 6acc4ab8..e5bc1a2f 100644 --- a/flow/src/test/scala/orca/review/FixLoopTest.scala +++ b/flow/src/test/scala/orca/review/FixLoopTest.scala @@ -33,11 +33,6 @@ class FixLoopTest extends munit.FunSuite: .reverse .collect: case OrcaEvent.Step(msg) => msg - def stageNames: List[String] = seen - .get() - .reverse - .collect: - case OrcaEvent.StageStarted(name) => name /** Evaluator that returns a scripted sequence; throws when exhausted. */ private def scripted(results: List[ReviewResult]): () => ReviewResult = @@ -87,8 +82,11 @@ class FixLoopTest extends munit.FunSuite: result.issues, List(IgnoredIssue(Title("b"), "out of scope")) ) + // Iterations are progress markers (`display`) now, not committing stages — + // they run under the caller's task stage (ADR 0018 §2.2), so they surface + // as Step events rather than StageStarted. assertEquals( - rec.stageNames.filter(_.startsWith("Iteration ")), + rec.steps.filter(_.startsWith("Iteration ")), List("Iteration 1", "Iteration 2", "Iteration 3") ) diff --git a/flow/src/test/scala/orca/review/JsonSchemaGenTest.scala b/flow/src/test/scala/orca/review/JsonSchemaGenTest.scala index 4850971d..38a7debd 100644 --- a/flow/src/test/scala/orca/review/JsonSchemaGenTest.scala +++ b/flow/src/test/scala/orca/review/JsonSchemaGenTest.scala @@ -1,6 +1,6 @@ package orca.review -import orca.llm.given +import orca.agents.given import orca.util.JsonSchemaGen import com.networknt.schema.{InputFormat, JsonSchemaFactory, SpecVersion} diff --git a/flow/src/test/scala/orca/review/LintTest.scala b/flow/src/test/scala/orca/review/LintTest.scala index f4655295..604b1fec 100644 --- a/flow/src/test/scala/orca/review/LintTest.scala +++ b/flow/src/test/scala/orca/review/LintTest.scala @@ -2,17 +2,17 @@ package orca.review import orca.{FlowContext} import orca.plan.Title -import orca.llm.{ +import orca.agents.{ AgentInput, Announce, - AutonomousLlmCall, + AutonomousAgentCall, AutonomousTextCall, BackendTag, - InteractiveLlmCall, + InteractiveAgentCall, JsonData, - LlmCall, - LlmConfig, - LlmTool, + AgentCall, + AgentConfig, + Agent, SessionId, ToolSet } @@ -21,37 +21,41 @@ import orca.{TestFlowContext} class LintTest extends munit.FunSuite: + // `lint` is now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe + private def ctx: FlowContext = new TestFlowContext(new EventDispatcher(Nil)) - /** LlmTool that records the serialized prompt passed to + /** Agent that records the serialized prompt passed to * `resultAs.autonomous.run` and returns a canned ReviewResult. Method-scope * mutable var holds the captured string. */ - private class CapturingLlmTool(canned: ReviewResult) - extends LlmTool[BackendTag.ClaudeCode.type]: + private class CapturingAgent(canned: ReviewResult) + extends Agent[BackendTag.ClaudeCode.type]: var captured: String = "" // Contents of the `*.log` file the prompt references, read inside `run` // while it still exists — `lint` deletes it once `run` returns. var capturedFileContent: String = "" val name = "mock" def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this def resultAs[O: JsonData: Announce] - : LlmCall[BackendTag.ClaudeCode.type, O] = - new LlmCall[BackendTag.ClaudeCode.type, O]: - val autonomous: AutonomousLlmCall[BackendTag.ClaudeCode.type, O] = - new AutonomousLlmCall[BackendTag.ClaudeCode.type, O]: + : AgentCall[BackendTag.ClaudeCode.type, O] = + new AgentCall[BackendTag.ClaudeCode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.ClaudeCode.type, O] = + new AutonomousAgentCall[BackendTag.ClaudeCode.type, O]: def run[I]( i: I, session: SessionId[BackendTag.ClaudeCode.type], - c: LlmConfig, + c: AgentConfig, emitPrompt: Boolean )(using - a: AgentInput[I] + a: AgentInput[I], + _x: orca.InStage ): (SessionId[BackendTag.ClaudeCode.type], O) = captured = a.serialize(i) capturedFileContent = "`([^`]+\\.log)`".r @@ -62,7 +66,8 @@ class LintTest extends munit.FunSuite: SessionId[BackendTag.ClaudeCode.type]("lint-test"), canned.asInstanceOf[O] ) - def interactive: InteractiveLlmCall[BackendTag.ClaudeCode.type, O] = ??? + def interactive: InteractiveAgentCall[BackendTag.ClaudeCode.type, O] = + ??? test("lint writes output to a file the prompt points to, returns its result"): given FlowContext = ctx @@ -79,7 +84,7 @@ class LintTest extends munit.FunSuite: ) ) ) - val mock = new CapturingLlmTool(expected) + val mock = new CapturingAgent(expected) val result = lint("echo LINT-BODY-MARKER", mock) assertEquals(result, expected) // The output goes to a file (so unbounded lint output can't overflow the @@ -100,7 +105,7 @@ class LintTest extends munit.FunSuite: test("lint short-circuits to ReviewResult.empty when the command is silent"): given FlowContext = ctx - val mock = new CapturingLlmTool(ReviewResult.empty) + val mock = new CapturingAgent(ReviewResult.empty) val result = lint("true", mock) assertEquals(result, ReviewResult.empty) assertEquals( diff --git a/flow/src/test/scala/orca/review/ReviewAndFixTest.scala b/flow/src/test/scala/orca/review/ReviewAndFixTest.scala index 0296ed22..305dbac7 100644 --- a/flow/src/test/scala/orca/review/ReviewAndFixTest.scala +++ b/flow/src/test/scala/orca/review/ReviewAndFixTest.scala @@ -2,60 +2,61 @@ package orca.review import orca.{FlowContext} import orca.plan.Title -import orca.llm.{ +import orca.agents.{ AgentInput, Announce, - AutonomousLlmCall, + AutonomousAgentCall, AutonomousTextCall, BackendTag, - InteractiveLlmCall, + InteractiveAgentCall, JsonData, - LlmCall, - LlmConfig, - LlmTool, + AgentCall, + AgentConfig, + Agent, SessionId, ToolSet } import orca.events.{EventDispatcher, OrcaEvent, OrcaListener} import orca.{TestFlowContext} -/** Fake LlmCall whose `autonomous.run` drains a scripted sequence of outputs in - * order — cast through `Any` because the trait is generic over output type. +/** Fake AgentCall whose `autonomous.run` drains a scripted sequence of outputs + * in order — cast through `Any` because the trait is generic over output type. * The session id from the call site is echoed back so tests can verify the * loop threaded a consistent id; `seenSessions` records each call's session id * so tests can assert "fresh on first, same id thereafter." */ -class FakeLlmCall[O](outputs: Iterator[Any]) - extends LlmCall[BackendTag.ClaudeCode.type, O]: +class FakeAgentCall[O](outputs: Iterator[Any]) + extends AgentCall[BackendTag.ClaudeCode.type, O]: /** Session ids the LLM was called with, in invocation order. */ val seenSessions = new java.util.concurrent.atomic.AtomicReference[ List[SessionId[BackendTag.ClaudeCode.type]] ](Nil) - val autonomous: AutonomousLlmCall[BackendTag.ClaudeCode.type, O] = - new AutonomousLlmCall[BackendTag.ClaudeCode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.ClaudeCode.type, O] = + new AutonomousAgentCall[BackendTag.ClaudeCode.type, O]: def run[I: AgentInput]( input: I, session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, + config: AgentConfig, emitPrompt: Boolean - ): (SessionId[BackendTag.ClaudeCode.type], O) = + )(using orca.InStage): (SessionId[BackendTag.ClaudeCode.type], O) = val _ = seenSessions.updateAndGet(session :: _) (session, outputs.next().asInstanceOf[O]) - def interactive: InteractiveLlmCall[BackendTag.ClaudeCode.type, O] = ??? + def interactive: InteractiveAgentCall[BackendTag.ClaudeCode.type, O] = ??? -class FakeLlmTool( +class FakeAgent( override val name: String, outputs: List[Any] = Nil -) extends LlmTool[BackendTag.ClaudeCode.type]: +) extends Agent[BackendTag.ClaudeCode.type]: private val it = outputs.iterator - val fakeCall: FakeLlmCall[Any] = new FakeLlmCall[Any](it) + val fakeCall: FakeAgentCall[Any] = new FakeAgentCall[Any](it) def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.ClaudeCode.type, O] = - fakeCall.asInstanceOf[LlmCall[BackendTag.ClaudeCode.type, O]] + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + fakeCall.asInstanceOf[AgentCall[BackendTag.ClaudeCode.type, O]] /** Session ids this tool was called with, in invocation order. Tests assert * the loop threaded a stable id across iterations. @@ -63,13 +64,16 @@ class FakeLlmTool( def seenSessions: List[SessionId[BackendTag.ClaudeCode.type]] = fakeCall.seenSessions.get().reverse - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this class ReviewAndFixTest extends munit.FunSuite: + // `reviewAndFixLoop` is now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe + private def ctx: FlowContext = new TestFlowContext(new EventDispatcher(Nil)) @@ -86,11 +90,11 @@ class ReviewAndFixTest extends munit.FunSuite: test("returns empty IgnoredIssues when no reviewer reports issues"): given FlowContext = ctx - val silentReviewer = new FakeLlmTool( + val silentReviewer = new FakeAgent( name = "quiet", outputs = List(ReviewResult.empty) ) - val coder = new FakeLlmTool("coder") + val coder = new FakeAgent("coder") val result = reviewAndFixLoop( coder = coder, sessionId = SessionId[BackendTag.ClaudeCode.type]("s"), @@ -108,11 +112,11 @@ class ReviewAndFixTest extends munit.FunSuite: // a fix. With `fixed` empty the loop halts after one round. val noisyIssue = issue("flaky", confidence = 0.3) val realIssue = issue("real bug", confidence = 0.95) - val reviewer = new FakeLlmTool( + val reviewer = new FakeAgent( name = "loud", outputs = List(ReviewResult(List(noisyIssue, realIssue))) ) - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "coder", outputs = List(FixOutcome(Nil, List(IgnoredIssue(Title("real bug"), "accepted")))) @@ -135,15 +139,15 @@ class ReviewAndFixTest extends munit.FunSuite: given FlowContext = ctx val issueA = issue("A") val issueB = issue("B") - val reviewerA = new FakeLlmTool( + val reviewerA = new FakeAgent( name = "a", outputs = List(ReviewResult(List(issueA))) ) - val reviewerB = new FakeLlmTool( + val reviewerB = new FakeAgent( name = "b", outputs = List(ReviewResult(List(issueB))) ) - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "coder", outputs = List( FixOutcome( @@ -174,11 +178,11 @@ class ReviewAndFixTest extends munit.FunSuite: // across iterations. given FlowContext = ctx val stubborn = issue("never ends") - val reviewer = new FakeLlmTool( + val reviewer = new FakeAgent( name = "loud", outputs = List.fill(4)(ReviewResult(List(stubborn))) ) - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "fixer", outputs = List.fill(3)(FixOutcome(List(Title("never ends")), Nil)) ) @@ -205,34 +209,36 @@ class ReviewAndFixTest extends munit.FunSuite: test("initialDiff is embedded in the reviewer's first prompt"): given FlowContext = ctx var capturedFirst: Option[String] = None - val captureReviewer = new LlmTool[BackendTag.ClaudeCode.type]: + val captureReviewer = new Agent[BackendTag.ClaudeCode.type]: val name = "capturing" def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this def resultAs[O: JsonData: Announce] - : LlmCall[BackendTag.ClaudeCode.type, O] = - new LlmCall[BackendTag.ClaudeCode.type, O]: - val autonomous: AutonomousLlmCall[BackendTag.ClaudeCode.type, O] = - new AutonomousLlmCall[BackendTag.ClaudeCode.type, O]: + : AgentCall[BackendTag.ClaudeCode.type, O] = + new AgentCall[BackendTag.ClaudeCode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.ClaudeCode.type, O] = + new AutonomousAgentCall[BackendTag.ClaudeCode.type, O]: def run[I: AgentInput]( i: I, session: SessionId[BackendTag.ClaudeCode.type], - c: LlmConfig, + c: AgentConfig, emitPrompt: Boolean + )(using + orca.InStage ): (SessionId[BackendTag.ClaudeCode.type], O) = capturedFirst = Some(i.toString) ( SessionId[BackendTag.ClaudeCode.type]("s"), ReviewResult.empty.asInstanceOf[O] ) - def interactive: InteractiveLlmCall[BackendTag.ClaudeCode.type, O] = + def interactive: InteractiveAgentCall[BackendTag.ClaudeCode.type, O] = ??? - val coder = new FakeLlmTool("coder") + val coder = new FakeAgent("coder") val _ = reviewAndFixLoop( coder = coder, sessionId = SessionId[BackendTag.ClaudeCode.type]("s"), @@ -246,19 +252,19 @@ class ReviewAndFixTest extends munit.FunSuite: assert(sent.contains("--- a/Foo.scala"), s"diff missing from prompt: $sent") assert(sent.contains("do thing"), s"task missing from prompt: $sent") - test("ReviewerSelector.llmDriven asks the LLM once and caches"): + test("ReviewerSelector.agentDriven asks the LLM once and caches"): given FlowContext = ctx - val perf = new FakeLlmTool(name = "performance") - val style = new FakeLlmTool(name = "readability") - val coverage = new FakeLlmTool(name = "test-coverage") + val perf = new FakeAgent(name = "performance") + val style = new FakeAgent(name = "readability") + val coverage = new FakeAgent(name = "test-coverage") val all = List(perf, style, coverage) // Single reply — if the selector calls the LLM more than once the iterator // will throw NoSuchElement on the second call, failing the test. - val picker = new FakeLlmTool( + val picker = new FakeAgent( name = "picker", outputs = List(SelectedReviewers(List("performance", "test-coverage"))) ) - val select = ReviewerSelector.llmDriven(llm = picker) + val select = ReviewerSelector.agentDriven(agent = picker) val title = Title("optimize hot path") val files = List("src/Cache.scala") assertEquals( @@ -274,24 +280,24 @@ class ReviewAndFixTest extends munit.FunSuite: ) test( - "an llmDriven reviewerSelection narrows the active set via its picker LLM" + "an agentDriven reviewerSelection narrows the active set via its picker LLM" ): given FlowContext = ctx val issueX = issue("only-x", confidence = 0.9) - val reviewerX = new FakeLlmTool( + val reviewerX = new FakeAgent( name = "x", outputs = List(ReviewResult(List(issueX))) ) - val reviewerY = new FakeLlmTool( + val reviewerY = new FakeAgent( name = "y" // promptOutputs intentionally empty: if the picker mistakenly chose y, // the loop would hit an empty iterator and throw. ) - val picker = new FakeLlmTool( + val picker = new FakeAgent( name = "picker", outputs = List(SelectedReviewers(List("x"))) ) - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "coder", outputs = List(FixOutcome(Nil, List(IgnoredIssue(Title("only-x"), "accepted")))) @@ -300,7 +306,7 @@ class ReviewAndFixTest extends munit.FunSuite: coder = coder, sessionId = SessionId[BackendTag.ClaudeCode.type]("s"), reviewers = List(reviewerX, reviewerY), - reviewerSelection = ReviewerSelector.llmDriven(llm = picker), + reviewerSelection = ReviewerSelector.agentDriven(agent = picker), task = "picker-routing check", initialDiff = Some("") ) @@ -314,13 +320,13 @@ class ReviewAndFixTest extends munit.FunSuite: ): given FlowContext = ctx val issueX = issue("only-x", confidence = 0.9) - val reviewerX = new FakeLlmTool( + val reviewerX = new FakeAgent( name = "x", outputs = List(ReviewResult(List(issueX))) ) // The coder's promptOutputs is empty: if the loop wrongly invokes the // picker against `coder`, the empty iterator throws and the test fails. - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "coder", outputs = List(FixOutcome(Nil, List(IgnoredIssue(Title("only-x"), "accepted")))) @@ -362,24 +368,26 @@ class ReviewAndFixTest extends munit.FunSuite: class GatedReviewer( label: String, gate: java.util.concurrent.CountDownLatch - ) extends LlmTool[BackendTag.ClaudeCode.type]: + ) extends Agent[BackendTag.ClaudeCode.type]: val name = label def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this def resultAs[O: JsonData: Announce] - : LlmCall[BackendTag.ClaudeCode.type, O] = - new LlmCall[BackendTag.ClaudeCode.type, O]: - val autonomous: AutonomousLlmCall[BackendTag.ClaudeCode.type, O] = - new AutonomousLlmCall[BackendTag.ClaudeCode.type, O]: + : AgentCall[BackendTag.ClaudeCode.type, O] = + new AgentCall[BackendTag.ClaudeCode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.ClaudeCode.type, O] = + new AutonomousAgentCall[BackendTag.ClaudeCode.type, O]: def run[I: AgentInput]( i: I, session: SessionId[BackendTag.ClaudeCode.type], - c: LlmConfig, + c: AgentConfig, emitPrompt: Boolean + )(using + orca.InStage ): (SessionId[BackendTag.ClaudeCode.type], O) = val ok = gate.await(2, java.util.concurrent.TimeUnit.SECONDS) assert(ok, s"$label gate never opened") @@ -387,14 +395,14 @@ class ReviewAndFixTest extends munit.FunSuite: SessionId[BackendTag.ClaudeCode.type](s"sid-$label"), ReviewResult.empty.asInstanceOf[O] ) - def interactive: InteractiveLlmCall[BackendTag.ClaudeCode.type, O] = + def interactive: InteractiveAgentCall[BackendTag.ClaudeCode.type, O] = ??? val slow = new GatedReviewer("slow", gate1) val fast = new GatedReviewer("fast", gate2) val runner = new Thread(() => val _ = reviewAndFixLoop( - coder = new FakeLlmTool("coder"), + coder = new FakeAgent("coder"), sessionId = SessionId[BackendTag.ClaudeCode.type]("s"), reviewers = List(slow, fast), task = "ordering check", @@ -429,19 +437,19 @@ class ReviewAndFixTest extends munit.FunSuite: val timeoutMs = 2000L class RendezvousReviewer(label: String) - extends LlmTool[BackendTag.ClaudeCode.type]: + extends Agent[BackendTag.ClaudeCode.type]: val name = label def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this def resultAs[O: JsonData: Announce] - : LlmCall[BackendTag.ClaudeCode.type, O] = - new LlmCall[BackendTag.ClaudeCode.type, O]: - val autonomous: AutonomousLlmCall[BackendTag.ClaudeCode.type, O] = - new AutonomousLlmCall[BackendTag.ClaudeCode.type, O]: + : AgentCall[BackendTag.ClaudeCode.type, O] = + new AgentCall[BackendTag.ClaudeCode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.ClaudeCode.type, O] = + new AutonomousAgentCall[BackendTag.ClaudeCode.type, O]: private def rendezvousThen(): O = rendezvous.countDown() val ok = rendezvous.await( @@ -457,25 +465,27 @@ class ReviewAndFixTest extends munit.FunSuite: def run[I: AgentInput]( i: I, session: SessionId[BackendTag.ClaudeCode.type], - c: LlmConfig, + c: AgentConfig, emitPrompt: Boolean + )(using + orca.InStage ): (SessionId[BackendTag.ClaudeCode.type], O) = ( SessionId[BackendTag.ClaudeCode.type](s"sid-$label"), rendezvousThen() ) - def interactive: InteractiveLlmCall[BackendTag.ClaudeCode.type, O] = + def interactive: InteractiveAgentCall[BackendTag.ClaudeCode.type, O] = ??? val _ = reviewAndFixLoop( - coder = new FakeLlmTool("coder"), + coder = new FakeAgent("coder"), sessionId = SessionId[BackendTag.ClaudeCode.type]("s"), reviewers = List(new RendezvousReviewer("reviewer")), task = "concurrency check", // echo emits output so `lint` doesn't short-circuit on empty stdout // and actually calls the (rendezvousing) LLM summariser. lintCommand = Some("echo lint-output"), - lintLlm = Some(new RendezvousReviewer("lint")), + lintAgent = Some(new RendezvousReviewer("lint")), reviewerSelection = ReviewerSelector.allEveryRound, initialDiff = Some("") ) @@ -486,12 +496,12 @@ class ReviewAndFixTest extends munit.FunSuite: // then clean) mean it must run twice — once before reviewing the // implementation, once before re-reviewing the fix. val counter = os.temp.dir() / "fmt-count" - val reviewer = new FakeLlmTool( + val reviewer = new FakeAgent( name = "r", outputs = List(ReviewResult(List(issue("needs fixing"))), ReviewResult.empty) ) - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "coder", outputs = List(FixOutcome(List(Title("needs fixing")), Nil)) ) diff --git a/flow/src/test/scala/orca/review/ReviewFixFlowTest.scala b/flow/src/test/scala/orca/review/ReviewFixFlowTest.scala index aebbe18d..de658bbd 100644 --- a/flow/src/test/scala/orca/review/ReviewFixFlowTest.scala +++ b/flow/src/test/scala/orca/review/ReviewFixFlowTest.scala @@ -2,7 +2,7 @@ package orca.review import orca.{FlowContext} import orca.plan.Title -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} import orca.events.{EventDispatcher, OrcaEvent, OrcaListener} import orca.{TestFlowContext} @@ -15,6 +15,9 @@ import java.util.concurrent.atomic.AtomicReference */ class ReviewFixFlowTest extends munit.FunSuite: + // `reviewAndFixLoop` is now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe + private class RecordingListener extends OrcaListener: private val seen: AtomicReference[List[OrcaEvent]] = AtomicReference(Nil) def onEvent(event: OrcaEvent): Unit = @@ -32,16 +35,16 @@ class ReviewFixFlowTest extends munit.FunSuite: suggestion = None ) - test("reviewAndFixLoop wraps the loop in a `Review & fix` stage"): + test("reviewAndFixLoop marks the loop with a `Review & fix` progress line"): val listener = new RecordingListener given FlowContext = new TestFlowContext(new EventDispatcher(List(listener))) val real = issue("real problem", confidence = 0.9) - val reviewer = new FakeLlmTool( + val reviewer = new FakeAgent( name = "perf", outputs = List(ReviewResult(List(real))) ) - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "coder", outputs = List( FixOutcome(Nil, List(IgnoredIssue(Title("real problem"), "trade-off"))) @@ -57,19 +60,14 @@ class ReviewFixFlowTest extends munit.FunSuite: initialDiff = Some("") ) + // The loop now runs under the caller's task stage (ADR 0018 §2.2), so it + // emits a progress line rather than opening its own committing stage. val events = listener.events assert( events.exists { - case OrcaEvent.StageStarted("Review & fix") => true; case _ => false - }, - s"missing StageStarted(Review & fix); got: $events" - ) - assert( - events.exists { - case OrcaEvent.StageCompleted("Review & fix") => true; - case _ => false + case OrcaEvent.Step("Review & fix") => true; case _ => false }, - s"missing StageCompleted(Review & fix); got: $events" + s"missing Step(Review & fix); got: $events" ) test("max iterations path surfaces leftover issues with the cap reason"): @@ -81,11 +79,11 @@ class ReviewFixFlowTest extends munit.FunSuite: // still finds it. The cap is the only thing that can stop this. // Reviewer keeps reporting the same issue across all iterations. val stubborn = issue("never ends") - val reviewer = new FakeLlmTool( + val reviewer = new FakeAgent( name = "loud", outputs = List.fill(21)(ReviewResult(List(stubborn))) ) - val coder = new FakeLlmTool( + val coder = new FakeAgent( name = "fixer", outputs = List.fill(20)(FixOutcome(List(Title("never ends")), Nil)) ) diff --git a/flow/src/test/scala/orca/review/ReviewTypesTest.scala b/flow/src/test/scala/orca/review/ReviewTypesTest.scala index 57095c95..789cb717 100644 --- a/flow/src/test/scala/orca/review/ReviewTypesTest.scala +++ b/flow/src/test/scala/orca/review/ReviewTypesTest.scala @@ -1,7 +1,7 @@ package orca.review import orca.plan.Title -import orca.llm.given +import orca.agents.given import com.github.plokhotnyuk.jsoniter_scala.core.{ readFromString, writeToString diff --git a/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala b/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala index 37f32589..b24c71ea 100644 --- a/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala +++ b/flow/src/test/scala/orca/review/ReviewerSelectorTest.scala @@ -2,16 +2,16 @@ package orca.review import orca.{FlowContext, TestFlowContext} import orca.events.EventDispatcher -import orca.llm.{ +import orca.agents.{ AgentInput, Announce, - AutonomousLlmCall, + AutonomousAgentCall, AutonomousTextCall, BackendTag, JsonData, - LlmCall, - LlmConfig, - LlmTool, + AgentCall, + AgentConfig, + Agent, SessionId, ToolSet } @@ -20,28 +20,29 @@ import orca.plan.Title import java.util.concurrent.atomic.AtomicReference /** Captures every `ReviewerSelectionRequest` handed to the picker and replies - * with a scripted `SelectedReviewers`. Other `LlmTool` surface is unused. + * with a scripted `SelectedReviewers`. Other `Agent` surface is unused. */ private class RecordingPicker( response: SelectedReviewers, captured: AtomicReference[Option[ReviewerSelectionRequest]] -) extends LlmTool[BackendTag.ClaudeCode.type]: +) extends Agent[BackendTag.ClaudeCode.type]: val name: String = "picker" def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.ClaudeCode.type, O] = - new LlmCall[BackendTag.ClaudeCode.type, O]: - val autonomous: AutonomousLlmCall[BackendTag.ClaudeCode.type, O] = - new AutonomousLlmCall[BackendTag.ClaudeCode.type, O]: + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + new AgentCall[BackendTag.ClaudeCode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.ClaudeCode.type, O] = + new AutonomousAgentCall[BackendTag.ClaudeCode.type, O]: def run[I: AgentInput]( input: I, session: SessionId[BackendTag.ClaudeCode.type], - config: LlmConfig, + config: AgentConfig, emitPrompt: Boolean - ): (SessionId[BackendTag.ClaudeCode.type], O) = + )(using orca.InStage): (SessionId[BackendTag.ClaudeCode.type], O) = input match case r: ReviewerSelectionRequest => captured.set(Some(r)) @@ -51,26 +52,31 @@ private class RecordingPicker( response.asInstanceOf[O] ) def interactive - : orca.llm.InteractiveLlmCall[BackendTag.ClaudeCode.type, O] = ??? + : orca.agents.InteractiveAgentCall[BackendTag.ClaudeCode.type, O] = + ??? /** Inert reviewer tool — just carries the name the selector dispatches on. */ private class NamedTool(override val name: String) - extends LlmTool[BackendTag.ClaudeCode.type]: + extends Agent[BackendTag.ClaudeCode.type]: def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? - def withConfig(c: LlmConfig): LlmTool[BackendTag.ClaudeCode.type] = this - def withSystemPrompt(p: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withName(n: String): LlmTool[BackendTag.ClaudeCode.type] = this - def withTools(tools: ToolSet): LlmTool[BackendTag.ClaudeCode.type] = this - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.ClaudeCode.type, O] = + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(tools: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? class ReviewerSelectorTest extends munit.FunSuite: private given FlowContext = new TestFlowContext(new EventDispatcher(Nil)) - private val scalaFp: LlmTool[?] = new NamedTool("reviewer: scala-fp") - private val generic: LlmTool[?] = new NamedTool("reviewer: generic") - private val all: List[LlmTool[?]] = List(scalaFp, generic) + // `agentDriven` is now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe + + private val scalaFp: Agent[?] = new NamedTool("reviewer: scala-fp") + private val generic: Agent[?] = new NamedTool("reviewer: generic") + private val all: List[Agent[?]] = List(scalaFp, generic) private val filePatterns = Map("reviewer: scala-fp" -> """\.scala$""".r) @@ -81,8 +87,8 @@ class ReviewerSelectorTest extends munit.FunSuite: SelectedReviewers(List("scala-fp", "generic")), captured ) - val selector = ReviewerSelector.llmDriven( - llm = picker, + val selector = ReviewerSelector.agentDriven( + agent = picker, filePatterns = filePatterns ) val picked = selector(Nil, all, Title("any"), List("src/lib.rs")) @@ -103,7 +109,7 @@ class ReviewerSelectorTest extends munit.FunSuite: SelectedReviewers(List("generic", "reviewer: scala-fp")), captured ) - val selector = ReviewerSelector.llmDriven(llm = picker) + val selector = ReviewerSelector.agentDriven(agent = picker) val picked = selector(Nil, all, Title("any"), List("src/main/scala/Foo.scala")) assertEquals( @@ -116,8 +122,8 @@ class ReviewerSelectorTest extends munit.FunSuite: ): val captured = new AtomicReference[Option[ReviewerSelectionRequest]](None) val picker = new RecordingPicker(SelectedReviewers(Nil), captured) - val selector = ReviewerSelector.llmDriven( - llm = picker, + val selector = ReviewerSelector.agentDriven( + agent = picker, filePatterns = filePatterns ) // scala-fp is filtered out for a .rs change; generic is eligible. The @@ -131,8 +137,8 @@ class ReviewerSelectorTest extends munit.FunSuite: SelectedReviewers(List("reviewer: scala-fp", "reviewer: generic")), captured ) - val selector = ReviewerSelector.llmDriven( - llm = picker, + val selector = ReviewerSelector.agentDriven( + agent = picker, filePatterns = filePatterns ) val picked = selector( @@ -153,8 +159,8 @@ class ReviewerSelectorTest extends munit.FunSuite: captured ) val onlyScala = List(scalaFp) - val selector = ReviewerSelector.llmDriven( - llm = picker, + val selector = ReviewerSelector.agentDriven( + agent = picker, filePatterns = filePatterns ) val picked = selector(Nil, onlyScala, Title("any"), List("src/lib.rs")) diff --git a/gemini/src/main/scala/orca/tools/gemini/DefaultGeminiTool.scala b/gemini/src/main/scala/orca/tools/gemini/DefaultGeminiAgent.scala similarity index 54% rename from gemini/src/main/scala/orca/tools/gemini/DefaultGeminiTool.scala rename to gemini/src/main/scala/orca/tools/gemini/DefaultGeminiAgent.scala index 4dbb053a..bbd390ab 100644 --- a/gemini/src/main/scala/orca/tools/gemini/DefaultGeminiTool.scala +++ b/gemini/src/main/scala/orca/tools/gemini/DefaultGeminiAgent.scala @@ -1,23 +1,23 @@ package orca.tools.gemini -import orca.llm.{BackendTag, GeminiTool, LlmConfig, Model, Prompts} +import orca.agents.{BackendTag, GeminiAgent, AgentConfig, Model, Prompts} import orca.events.OrcaListener -import orca.backend.{Interaction, LlmBackend} -import orca.llm.BaseLlmTool +import orca.backend.{Interaction, AgentBackend} +import orca.agents.BaseAgent -/** Default [[GeminiTool]] implementation. Inherits the autonomous-text + - * `resultAs[O]` plumbing from [[BaseLlmTool]] and only adds the - * Gemini-specific `flash` model accessor. +/** Default [[GeminiAgent]] implementation. Inherits the autonomous-text + + * `resultAs[O]` plumbing from [[BaseAgent]] and only adds the Gemini-specific + * `flash` model accessor. */ -private[orca] class DefaultGeminiTool( - backend: LlmBackend[BackendTag.Gemini.type], - config: LlmConfig, +private[orca] class DefaultGeminiAgent( + backend: AgentBackend[BackendTag.Gemini.type], + config: AgentConfig, prompts: Prompts, workDir: os.Path, events: OrcaListener, interaction: Interaction, val name: String = "main" -) extends BaseLlmTool[BackendTag.Gemini.type, GeminiTool]( +) extends BaseAgent[BackendTag.Gemini.type, GeminiAgent]( backend, config, prompts, @@ -25,20 +25,20 @@ private[orca] class DefaultGeminiTool( events, interaction ) - with GeminiTool: + with GeminiAgent: /** Pin the cheap-and-fast model variant. The literal id matches what's * available in the installed `gemini` CLI; newer versions may rename, in - * which case callers override via `withConfig(LlmConfig(model = + * which case callers override via `withConfig(AgentConfig(model = * Some(Model("..."))))`. */ - def flash: GeminiTool = withModel(Model("gemini-2.5-flash")) + def flash: GeminiAgent = withModel(Model("gemini-2.5-flash")) protected def copyTool( - config: LlmConfig = config, + config: AgentConfig = config, name: String = name - ): GeminiTool = - new DefaultGeminiTool( + ): GeminiAgent = + new DefaultGeminiAgent( backend, config, prompts, @@ -48,7 +48,7 @@ private[orca] class DefaultGeminiTool( name ) -private[orca] object DefaultGeminiTool: +private[orca] object DefaultGeminiAgent: /** The strong default model. Bare `gemini` pins this (in the runtime wiring, * mirroring claude's Opus default for the long-lived implementer); `flash` diff --git a/gemini/src/main/scala/orca/tools/gemini/GeminiArgs.scala b/gemini/src/main/scala/orca/tools/gemini/GeminiArgs.scala index f4b88471..75daaa4d 100644 --- a/gemini/src/main/scala/orca/tools/gemini/GeminiArgs.scala +++ b/gemini/src/main/scala/orca/tools/gemini/GeminiArgs.scala @@ -1,9 +1,9 @@ package orca.tools.gemini import orca.backend.CliArgs -import orca.llm.{AutoApprove, BackendTag, LlmConfig, SessionId, ToolSet} +import orca.agents.{AutoApprove, BackendTag, AgentConfig, SessionId, ToolSet} -/** Maps `LlmConfig` fields to `gemini` headless CLI flags. `systemPrompt` is +/** Maps `AgentConfig` fields to `gemini` headless CLI flags. `systemPrompt` is * not handled here — gemini has no `--append-system-prompt` equivalent (it * picks up `GEMINI.md` files for static instructions), so the backend folds it * into the user prompt before this method runs. The `ask_user` MCP server is @@ -22,7 +22,7 @@ private[gemini] object GeminiArgs: * cwd is set on the OS process spawn (gemini headless has no `-C` flag), so * it isn't rendered here. */ - def headless(prompt: String, config: LlmConfig): Seq[String] = + def headless(prompt: String, config: AgentConfig): Seq[String] = Seq("gemini") ++ trustArgs ++ CliArgs.modelArgs(config) ++ @@ -35,7 +35,7 @@ private[gemini] object GeminiArgs: def resume( sessionId: SessionId[BackendTag.Gemini.type], prompt: String, - config: LlmConfig + config: AgentConfig ): Seq[String] = Seq("gemini") ++ trustArgs ++ @@ -63,7 +63,7 @@ private[gemini] object GeminiArgs: */ private val NetworkTools: Seq[String] = Seq("web_fetch") - /** Maps [[LlmConfig.tools]] to gemini's approval mode. The read-only tiers + /** Maps [[AgentConfig.tools]] to gemini's approval mode. The read-only tiers * use `--approval-mode plan` (no writes, no shelling out), matching claude's * `--permission-mode plan` and codex's `--sandbox read-only`. `Full` has no * per-tool CLI allowlist, and in headless mode `auto_edit` blocks on shell @@ -78,7 +78,7 @@ private[gemini] object GeminiArgs: * so the planner can fetch issue/PR/web content — no shell `gh`, so no * authed GitHub, but web reads work. */ - private def approvalArgs(config: LlmConfig): Seq[String] = + private def approvalArgs(config: AgentConfig): Seq[String] = config.tools match case ToolSet.ReadOnly => Seq("--approval-mode", "plan") case ToolSet.NetworkOnly => diff --git a/gemini/src/main/scala/orca/tools/gemini/GeminiBackend.scala b/gemini/src/main/scala/orca/tools/gemini/GeminiBackend.scala index 19953a38..42c96ee0 100644 --- a/gemini/src/main/scala/orca/tools/gemini/GeminiBackend.scala +++ b/gemini/src/main/scala/orca/tools/gemini/GeminiBackend.scala @@ -1,21 +1,22 @@ package orca.tools.gemini import orca.events.OrcaListener -import orca.llm.{BackendTag, LlmConfig, SessionId} -import orca.{AgentTurnFailed, OrcaFlowException} +import orca.agents.{BackendTag, AgentConfig, SessionId} +import orca.subprocess.CliResult import orca.backend.{ Conversation, Conversations, Dispatch, - LlmBackend, - LlmResult, + AgentBackend, + AgentResult, SessionMode, SessionRegistry, + SubprocessSpawn, SystemPromptComposer } import orca.backend.mcp.{AskUserMcpServer, AskUserSession} import orca.subprocess.CliRunner -import ox.Ox +import ox.{Ox, supervised} import ox.channels.BufferCapacity /** Gemini backend. Both autonomous and interactive paths drive `gemini -p @@ -41,8 +42,8 @@ import ox.channels.BufferCapacity * (the restore rides as an `extras` `AutoCloseable` on the * [[AskUserSession]]). Autonomous calls skip the bridge entirely. */ -private[orca] class GeminiBackend(cli: CliRunner)(using Ox, BufferCapacity) - extends LlmBackend[BackendTag.Gemini.type]: +private[orca] class GeminiBackend(cli: CliRunner)(using BufferCapacity) + extends AgentBackend[BackendTag.Gemini.type]: /** Maps the client-allocated session id to gemini's `init`-reported session * id. `gemini -p` mints its own id, so we keep this mapping to dispatch @@ -54,44 +55,42 @@ private[orca] class GeminiBackend(cli: CliRunner)(using Ox, BufferCapacity) def runAutonomous( prompt: String, session: SessionId[BackendTag.Gemini.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener = OrcaListener.noop, outputSchema: Option[String] = None - ): LlmResult[BackendTag.Gemini.type] = - val conv = openConversation( - prompt = prompt, - mode = SessionMode.Autonomous, - session = session, - config = config, - workDir = workDir, - // Forwarded so `conv.outputSchema` signals structured mode to the drain - // (suppressing the raw JSON payload from the user log). gemini has no - // `--output-schema` flag, so enforcement is prompt-only. - outputSchema = outputSchema - ) - try - val result = Conversations.drainAutonomous(conv, events) - sessions.commitSuccess(session, result.sessionId) + ): AgentResult[BackendTag.Gemini.type] = + // Self-scoped: the conversation forks its workers into this per-call Ox, the + // drain consumes them, and `cancel` (the `finally`) tears the subprocess + + // forks down before the scope joins. + supervised: + val conv = openConversation( + prompt = prompt, + mode = SessionMode.Autonomous, + session = session, + config = config, + workDir = workDir, + // Forwarded so `conv.outputSchema` signals structured mode to the drain + // (suppressing the raw JSON payload from the user log). gemini has no + // `--output-schema` flag, so enforcement is prompt-only. + outputSchema = outputSchema + ) // Hide the server-allocated id from the caller — they keep using the // client id they passed in. Future calls resolve via the registry. - result.copy(sessionId = session) - catch - // Preserve the non-retryable type: a turn that genuinely ran and failed - // must keep its `AgentTurnFailed` so the corrective-retry loop (which only - // retries parse failures) doesn't re-run it. - case e: AgentTurnFailed => throw e - case e: OrcaFlowException => - throw new OrcaFlowException(s"gemini CLI failed: ${e.getMessage}") + try + Conversations + .drainAndCommit("gemini", conv, session, sessions, events) + .copy(sessionId = session) + finally conv.cancel() def runInteractive( prompt: String, session: SessionId[BackendTag.Gemini.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Gemini.type] = + )(using Ox): Conversation[BackendTag.Gemini.type] = openConversation( prompt, mode = SessionMode.Interactive(displayPrompt), @@ -116,10 +115,10 @@ private[orca] class GeminiBackend(cli: CliRunner)(using Ox, BufferCapacity) prompt: String, mode: SessionMode, session: SessionId[BackendTag.Gemini.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Gemini.type] = + )(using Ox): Conversation[BackendTag.Gemini.type] = val (askUser, displayPrompt): (Option[AskUserSession], String) = mode match case SessionMode.Interactive(p) => @@ -127,7 +126,9 @@ private[orca] class GeminiBackend(cli: CliRunner)(using Ox, BufferCapacity) List(GeminiSettings.register(workDir, server.url)) (Some(askUserSession), p) case SessionMode.Autonomous => (None, "") - try + // On a spawn/build failure the ask_user bundle is closed, which also + // restores the settings.json via its `extras`, so nothing leaks. + SubprocessSpawn.open("gemini", askUser.toList) { // gemini has no `--append-system-prompt` flag (it picks up `GEMINI.md` // files for static instructions), so fold the composed system prompt into // the user prompt — same approach as codex. @@ -141,38 +142,55 @@ private[orca] class GeminiBackend(cli: CliRunner)(using Ox, BufferCapacity) GeminiArgs.resume(serverId, finalPrompt, config) case Dispatch.Fresh(_) => GeminiArgs.headless(finalPrompt, config) - val process = cli.spawnPiped(args, cwd = workDir, pipeStderr = true) - try - // Close stdin so the child stops waiting on EOF (gemini reads the - // prompt argv-side). - process.closeStdin() - new GeminiConversation( - process, - initialPrompt = displayPrompt, - outputSchema = outputSchema, - askUser = askUser - ) - catch - case e: Exception => - process.sendSigInt() - throw OrcaFlowException( - s"Failed to open gemini session: ${e.getMessage}" - ) - catch - case e: Throwable => - // Any failure between resource allocation and a fully-constructed - // GeminiConversation: tear down the MCP server (which also restores - // the settings.json via its extras) so nothing leaks. Once the - // conversation owns the resources they ride through `onFinalize`. - askUser.foreach(_.close()) - throw e + cli.spawnPiped(args, cwd = workDir, pipeStderr = true) + } { process => + // Close stdin so the child stops waiting on EOF (gemini reads the prompt + // argv-side). + process.closeStdin() + new GeminiConversation( + process, + initialPrompt = displayPrompt, + outputSchema = outputSchema, + askUser = askUser + ) + } /** Record the server session id so subsequent calls with the same client id * resume that session. Called by [[runAutonomous]] post-drain and by - * [[orca.llm.DefaultLlmCall]] post-`interaction.drive` on the interactive - * path; delegates to the registry's `commitSuccess`. + * [[orca.agents.DefaultAgentCall]] post-`interaction.drive` on the + * interactive path; delegates to the registry's `commitSuccess`. */ override def registerSession( client: SessionId[BackendTag.Gemini.type], server: SessionId[BackendTag.Gemini.type] ): Unit = sessions.commitSuccess(client, server) + + override def resumeWireId( + client: SessionId[BackendTag.Gemini.type] + ): Option[SessionId[BackendTag.Gemini.type]] = sessions.resumeWireId(client) + + /** Best-effort probe: resolves the SERVER id mapped to `client` (gemini mints + * its own session id; the caller's stable id never appears in + * `--list-sessions`), runs `gemini --list-sessions`, and checks whether the + * server id appears in the output (substring scan). Returns `false` — safe + * re-seed — when no server id is mapped (the map hasn't been rehydrated from + * the log, so there is no known live session), on non-zero exit, any + * exception, or if the id fails the [[orca.agents.isSafeSessionId]] guard + * (added for consistency; the substring scan is not injection-susceptible, + * but the guard keeps all probes uniform). + * + * The client→server map is persisted in the progress log and rehydrated on + * resume (D2), so on a resumed run this targets the right server id. + */ + override def sessionExists( + session: SessionId[BackendTag.Gemini.type] + ): Boolean = + probeServerSession(session, sessions): id => + val result = listSessionsOutput() + result.exitCode == 0 && result.stdout.linesIterator.exists(_.contains(id)) + + /** Overridable in tests via a stub `CliRunner`; default runs `gemini + * --list-sessions`. + */ + private[gemini] def listSessionsOutput(): CliResult = + cli.run(Seq("gemini", "--list-sessions")) diff --git a/gemini/src/main/scala/orca/tools/gemini/GeminiConversation.scala b/gemini/src/main/scala/orca/tools/gemini/GeminiConversation.scala index 01ca5957..b9858581 100644 --- a/gemini/src/main/scala/orca/tools/gemini/GeminiConversation.scala +++ b/gemini/src/main/scala/orca/tools/gemini/GeminiConversation.scala @@ -1,25 +1,26 @@ package orca.tools.gemini -import orca.llm.{BackendTag, Model, SessionId} +import orca.agents.{BackendTag, Model, SessionId} import orca.events.Usage import orca.{AgentTurnFailed, OrcaFlowException} import orca.backend.{ BufferedStderrDiagnostics, ConversationEvent, - LlmResult, - StreamConversation, + AgentResult, + ForkedConversation, StreamSource } import orca.backend.mcp.{AskUserMcpServer, AskUserSession} import orca.subprocess.PipedCliProcess import orca.tools.gemini.jsonl.InboundEvent +import ox.Ox import java.util.concurrent.atomic.AtomicReference /** Drives a `gemini -p <prompt> --output-format stream-json` session to - * completion. Boilerplate lives in [[StreamConversation]]; this class supplies - * the gemini-specific protocol translation: JSONL → [[InboundEvent]] → - * `ConversationEvent`s. + * completion. Boilerplate lives in [[orca.backend.ForkedConversation]]; this + * class supplies the gemini-specific protocol translation: JSONL → + * [[InboundEvent]] → `ConversationEvent`s. * * Notable parity gaps vs. claude (deliberate — see ADR 0015): * - gemini emits whole `message` chunks, not negotiated tool approvals; @@ -27,7 +28,7 @@ import java.util.concurrent.atomic.AtomicReference * emitted here. * - gemini headless is one-shot; multi-turn happens via `--resume` on a * fresh spawn, so `sendUserMessage` is a no-op. - * - **`LlmResult.output` is synthesised**: gemini has no single terminal + * - **`AgentResult.output` is synthesised**: gemini has no single terminal * message carrying the answer, so assistant-role `message` content is * accumulated as it streams and the result builder reads it at the * `result` event. The prompt template makes the closing content JSON in @@ -38,15 +39,14 @@ private[gemini] class GeminiConversation( initialPrompt: String = "", val outputSchema: Option[String] = None, override val askUser: Option[AskUserSession] = None -) extends StreamConversation[BackendTag.Gemini.type]( +)(using Ox) + extends ForkedConversation[BackendTag.Gemini.type]( source = StreamSource.fromProcess(process), backendName = "gemini", initialPrompt = initialPrompt ) with BufferedStderrDiagnostics[BackendTag.Gemini.type]: - import StreamConversation.Outcome - private val sessionIdRef = new AtomicReference[String]("") private val modelRef = new AtomicReference[Option[String]](None) @@ -64,20 +64,14 @@ private[gemini] class GeminiConversation( /** tool_use ids for `ask_user` MCP calls whose echo we drop — the host-side * bridge already surfaced the matching `UserQuestion`, so rendering the tool - * call + the answer-as-result on top would be noise. + * call + the answer-as-result on top would be noise. See + * [[orca.backend.AskUserEchoes]]. */ - private var suppressedToolIds: Set[String] = Set.empty - - // Subclass fields above are assigned; safe to spin up the reader. - start() + private val askUserEchoes = new orca.backend.AskUserEchoes - // --- Conversation surface --- - - /** gemini headless consumes its prompt argv-side and exits; injecting more - * user turns mid-session isn't supported (multi-turn is a fresh `--resume` - * spawn). The contract still requires a callable method — this is a no-op. - */ - def sendUserMessage(text: String): Unit = () + // No `start()`: the base spawns its reader / stderr / ask-user forks lazily + // on first touch of the conversation surface, after this subclass's fields + // are initialised. // --- Reader hooks --- @@ -133,27 +127,22 @@ private[gemini] class GeminiConversation( */ private def handleResult(usage: Usage, status: String): Unit = if status.nonEmpty && status != "success" then - val _ = outcomeRef.compareAndSet( - None, - Some( - Outcome.failed[BackendTag.Gemini.type]( - // Fold in the buffered stderr (the real reason — quota, auth, …) - // so the exception carries it even for a noop listener, matching - // the non-zero-exit and missing-result failure paths. - new AgentTurnFailed( - appendContext(s"gemini turn ended with status '$status'") - ) - ) + // Fold in the buffered stderr (the real reason — quota, auth, …) so the + // exception carries it even for a noop listener, matching the + // non-zero-exit and missing-result failure paths. + failWith( + new AgentTurnFailed( + appendContext(s"gemini turn ended with status '$status'") ) ) else - val result = LlmResult( + val result = AgentResult( sessionId = SessionId[BackendTag.Gemini.type](sessionIdRef.get()), output = answer.toString, usage = usage, model = modelRef.get().map(Model.apply) ) - val _ = outcomeRef.compareAndSet(None, Some(Outcome.Success(result))) + succeedWith(result) /** A `user`-role message is the prompt echo (the base already surfaced the * opening prompt as a `UserMessage`), so it's dropped from both the event @@ -167,8 +156,7 @@ private[gemini] class GeminiConversation( eventQueue.enqueue(ConversationEvent.AssistantTextDelta(content)) private def handleToolUse(name: String, id: String, params: String): Unit = - if GeminiConversation.isAskUserTool(name) then - suppressedToolIds = suppressedToolIds + id + if GeminiConversation.isAskUserTool(name) then askUserEchoes.suppress(id) else toolNames = toolNames + (id -> name) eventQueue.enqueue( @@ -180,8 +168,7 @@ private[gemini] class GeminiConversation( status: String, output: String ): Unit = - if suppressedToolIds.contains(id) then - suppressedToolIds = suppressedToolIds - id + if askUserEchoes.consume(id) then () else eventQueue.enqueue( ConversationEvent.ToolResult( diff --git a/gemini/src/test/scala/orca/tools/gemini/DefaultGeminiToolTest.scala b/gemini/src/test/scala/orca/tools/gemini/DefaultGeminiToolTest.scala index a83234bf..1e0bd63d 100644 --- a/gemini/src/test/scala/orca/tools/gemini/DefaultGeminiToolTest.scala +++ b/gemini/src/test/scala/orca/tools/gemini/DefaultGeminiToolTest.scala @@ -1,17 +1,20 @@ package orca.tools.gemini -import orca.backend.{Conversation, Interaction, LlmResult, SupervisedBackend} +import orca.backend.{Conversation, Interaction, AgentResult, SupervisedBackend} import orca.events.OrcaListener -import orca.llm.{BackendTag, DefaultPrompts, LlmConfig} +import orca.agents.{BackendTag, DefaultPrompts, AgentConfig} import orca.subprocess.{FakePipedCliProcess, SpawnStubCliRunner} -class DefaultGeminiToolTest extends munit.FunSuite: +class DefaultGeminiAgentTest extends munit.FunSuite: + + // LLM `run` is now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe private val stubInteraction: Interaction = new Interaction: val listeners: List[OrcaListener] = Nil def drive[B <: BackendTag]( conversation: Conversation[B] - ): LlmResult[B] = throw new UnsupportedOperationException("test stub") + ): AgentResult[B] = throw new UnsupportedOperationException("test stub") private def successfulProcess(): FakePipedCliProcess = val p = new FakePipedCliProcess() @@ -27,11 +30,11 @@ class DefaultGeminiToolTest extends munit.FunSuite: private def toolWith( runner: SpawnStubCliRunner, - config: LlmConfig - )(body: DefaultGeminiTool => Unit): Unit = + config: AgentConfig + )(body: DefaultGeminiAgent => Unit): Unit = SupervisedBackend.using(new GeminiBackend(runner)): backend => body( - new DefaultGeminiTool( + new DefaultGeminiAgent( backend = backend, config = config, prompts = DefaultPrompts, @@ -45,7 +48,7 @@ class DefaultGeminiToolTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(successfulProcess())) toolWith( runner, - LlmConfig.default.copy(model = Some(DefaultGeminiTool.Pro)) + AgentConfig.default.copy(model = Some(DefaultGeminiAgent.Pro)) ): tool => val _ = tool.autonomous.run("q") assert( @@ -57,7 +60,7 @@ class DefaultGeminiToolTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(successfulProcess())) toolWith( runner, - LlmConfig.default.copy(model = Some(DefaultGeminiTool.Pro)) + AgentConfig.default.copy(model = Some(DefaultGeminiAgent.Pro)) ): tool => val _ = tool.flash.autonomous.run("q") assert( @@ -65,8 +68,8 @@ class DefaultGeminiToolTest extends munit.FunSuite: s"expected the flash pin; got: ${runner.calls.head}" ) - test("withName preserves the GeminiTool type and renames"): + test("withName preserves the GeminiAgent type and renames"): val runner = new SpawnStubCliRunner(List(successfulProcess())) - toolWith(runner, LlmConfig.default): tool => + toolWith(runner, AgentConfig.default): tool => val renamed = tool.withName("planner") assertEquals(renamed.name, "planner") diff --git a/gemini/src/test/scala/orca/tools/gemini/GeminiArgsTest.scala b/gemini/src/test/scala/orca/tools/gemini/GeminiArgsTest.scala index 0e53257a..c25d643f 100644 --- a/gemini/src/test/scala/orca/tools/gemini/GeminiArgsTest.scala +++ b/gemini/src/test/scala/orca/tools/gemini/GeminiArgsTest.scala @@ -1,11 +1,18 @@ package orca.tools.gemini -import orca.llm.{AutoApprove, BackendTag, LlmConfig, Model, SessionId, ToolSet} +import orca.agents.{ + AutoApprove, + BackendTag, + AgentConfig, + Model, + SessionId, + ToolSet +} class GeminiArgsTest extends munit.FunSuite: test("headless emits gemini -p <prompt> --output-format stream-json"): - val args = GeminiArgs.headless("summarize", LlmConfig.default) + val args = GeminiArgs.headless("summarize", AgentConfig.default) assertEquals(args.head, "gemini") assert( args.containsSlice(Seq("--output-format", "stream-json")), @@ -18,20 +25,20 @@ class GeminiArgsTest extends munit.FunSuite: // silently overrides --approval-mode back to "default". orca always drives // a working dir the agent is meant to operate in, so trust is unconditional // — the analog of codex's --skip-git-repo-check. - val args = GeminiArgs.headless("x", LlmConfig.default) + val args = GeminiArgs.headless("x", AgentConfig.default) assert(args.contains("--skip-trust"), args.toString) - test("headless passes --model when LlmConfig.model is set"): + test("headless passes --model when AgentConfig.model is set"): val args = GeminiArgs.headless( "x", - LlmConfig.default.copy(model = Some(Model("gemini-2.5-flash"))) + AgentConfig.default.copy(model = Some(Model("gemini-2.5-flash"))) ) assert(args.containsSlice(Seq("--model", "gemini-2.5-flash"))) test("AutoApprove.All maps to --approval-mode yolo"): val args = GeminiArgs.headless( "x", - LlmConfig.default.copy(autoApprove = AutoApprove.All) + AgentConfig.default.copy(autoApprove = AutoApprove.All) ) assert(args.containsSlice(Seq("--approval-mode", "yolo")), args.toString) @@ -41,7 +48,7 @@ class GeminiArgsTest extends munit.FunSuite: // an approval prompt no one can answer. See ADR 0015. val args = GeminiArgs.headless( "x", - LlmConfig.default.copy(autoApprove = AutoApprove.Only(Set("Bash"))) + AgentConfig.default.copy(autoApprove = AutoApprove.Only(Set("Bash"))) ) assert(args.containsSlice(Seq("--approval-mode", "yolo")), args.toString) assert(!args.contains("plan")) @@ -51,7 +58,7 @@ class GeminiArgsTest extends munit.FunSuite: ): val args = GeminiArgs.headless( "x", - LlmConfig.default.copy( + AgentConfig.default.copy( tools = ToolSet.ReadOnly, autoApprove = AutoApprove.All ) @@ -63,7 +70,7 @@ class GeminiArgsTest extends munit.FunSuite: val args = GeminiArgs.headless( "x", - LlmConfig.default.copy(tools = ToolSet.NetworkOnly) + AgentConfig.default.copy(tools = ToolSet.NetworkOnly) ) assert(args.containsSlice(Seq("--approval-mode", "plan")), args.toString) assert( @@ -73,16 +80,16 @@ class GeminiArgsTest extends munit.FunSuite: test("resume builds gemini ... --resume <id> with the prompt"): val sid = SessionId[BackendTag.Gemini.type]("uuid-123") - val args = GeminiArgs.resume(sid, "next step", LlmConfig.default) + val args = GeminiArgs.resume(sid, "next step", AgentConfig.default) assertEquals(args.head, "gemini") assert(args.containsSlice(Seq("--resume", "uuid-123")), args.toString) assert(args.containsSlice(Seq("-p", "next step")), args.toString) - test("resume propagates --model when LlmConfig.model is set"): + test("resume propagates --model when AgentConfig.model is set"): val sid = SessionId[BackendTag.Gemini.type]("sid") val args = GeminiArgs.resume( sid, "x", - LlmConfig.default.copy(model = Some(Model("gemini-2.5-pro"))) + AgentConfig.default.copy(model = Some(Model("gemini-2.5-pro"))) ) assert(args.containsSlice(Seq("--model", "gemini-2.5-pro"))) diff --git a/gemini/src/test/scala/orca/tools/gemini/GeminiBackendTest.scala b/gemini/src/test/scala/orca/tools/gemini/GeminiBackendTest.scala index 8980b3cb..761037ae 100644 --- a/gemini/src/test/scala/orca/tools/gemini/GeminiBackendTest.scala +++ b/gemini/src/test/scala/orca/tools/gemini/GeminiBackendTest.scala @@ -1,9 +1,14 @@ package orca.tools.gemini import orca.backend.SupervisedBackend -import orca.llm.{BackendTag, LlmConfig, Model, SessionId} +import orca.agents.{BackendTag, AgentConfig, Model, SessionId} import orca.OrcaFlowException -import orca.subprocess.{FakePipedCliProcess, SpawnStubCliRunner} +import orca.subprocess.{ + CliResult, + FakePipedCliProcess, + SpawnStubCliRunner, + StubCliRunner +} class GeminiBackendTest extends munit.FunSuite: @@ -11,7 +16,7 @@ class GeminiBackendTest extends munit.FunSuite: SessionId[BackendTag.Gemini.type]("00000000-0000-0000-0000-000000000000") private def withBackend[T](runner: SpawnStubCliRunner)( - body: GeminiBackend => T + body: ox.Ox ?=> GeminiBackend => T ): T = SupervisedBackend.using(new GeminiBackend(runner))(body) private def successfulProcess( @@ -52,7 +57,12 @@ class GeminiBackendTest extends munit.FunSuite: ) withBackend(runner): backend => val result = - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) // The returned session id is the client id — the server's sess-42 is // mapped internally so subsequent calls resume it. assertEquals(result.sessionId, clientSid) @@ -67,7 +77,12 @@ class GeminiBackendTest extends munit.FunSuite: ) withBackend(runner): backend => val result = - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) assertEquals(result.model, Some(Model("gemini-2.5-pro"))) test("runAutonomous throws when gemini exits without a result event"): @@ -78,7 +93,12 @@ class GeminiBackendTest extends munit.FunSuite: p.sendSigInt() withBackend(new SpawnStubCliRunner(List(p))): backend => intercept[OrcaFlowException]: - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) test("runAutonomous throws with the exit code when gemini exits non-zero"): val p = new FakePipedCliProcess(initiallyAlive = false): @@ -87,7 +107,12 @@ class GeminiBackendTest extends munit.FunSuite: p.closeStderr() withBackend(new SpawnStubCliRunner(List(p))): backend => val ex = intercept[OrcaFlowException]: - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) assert( ex.getMessage.contains("exited with code 7"), s"expected the exit code in the message; got: ${ex.getMessage}" @@ -101,7 +126,12 @@ class GeminiBackendTest extends munit.FunSuite: p.closeStderr() withBackend(new SpawnStubCliRunner(List(p))): backend => val ex = intercept[OrcaFlowException]: - backend.runAutonomous("q", clientSid, LlmConfig.default, os.temp.dir()) + backend.runAutonomous( + "q", + clientSid, + AgentConfig.default, + os.temp.dir() + ) assert( ex.getMessage.contains("resume failed: session not found"), s"expected stderr in the exception; got: ${ex.getMessage}" @@ -113,7 +143,7 @@ class GeminiBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "list files", clientSid, - LlmConfig.default.copy(systemPrompt = Some("be terse")), + AgentConfig.default.copy(systemPrompt = Some("be terse")), os.temp.dir() ) val finalPrompt = runner.calls.head.last @@ -130,9 +160,9 @@ class GeminiBackendTest extends munit.FunSuite: withBackend(runner): backend => val workDir = os.temp.dir() val _ = - backend.runAutonomous("first", clientSid, LlmConfig.default, workDir) + backend.runAutonomous("first", clientSid, AgentConfig.default, workDir) val _ = - backend.runAutonomous("again", clientSid, LlmConfig.default, workDir) + backend.runAutonomous("again", clientSid, AgentConfig.default, workDir) val firstArgs = runner.calls(0) val secondArgs = runner.calls(1) assert(!firstArgs.contains("--resume"), firstArgs.toString) @@ -150,7 +180,7 @@ class GeminiBackendTest extends munit.FunSuite: backend.runAutonomous( "after", clientSid, - LlmConfig.default, + AgentConfig.default, os.temp.dir() ) val args = runner.calls.head @@ -165,8 +195,8 @@ class GeminiBackendTest extends munit.FunSuite: val workDir = os.temp.dir() val sidA = SessionId[BackendTag.Gemini.type]("aaaaaaaa") val sidB = SessionId[BackendTag.Gemini.type]("bbbbbbbb") - val _ = backend.runAutonomous("for A", sidA, LlmConfig.default, workDir) - val _ = backend.runAutonomous("for B", sidB, LlmConfig.default, workDir) + val _ = backend.runAutonomous("for A", sidA, AgentConfig.default, workDir) + val _ = backend.runAutonomous("for B", sidB, AgentConfig.default, workDir) assert( !runner.calls(1).contains("--resume"), s"second call with a new client id must NOT resume; got: ${runner.calls(1)}" @@ -178,7 +208,8 @@ class GeminiBackendTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(successfulProcess())) withBackend(runner): backend => val workDir = os.temp.dir() - val _ = backend.runAutonomous("q", clientSid, LlmConfig.default, workDir) + val _ = + backend.runAutonomous("q", clientSid, AgentConfig.default, workDir) assert( !os.exists(workDir / ".gemini" / "settings.json"), "autonomous must not write a .gemini/settings.json" @@ -194,7 +225,7 @@ class GeminiBackendTest extends munit.FunSuite: "q", clientSid, displayPrompt = "q", - LlmConfig.default, + AgentConfig.default, workDir, outputSchema = None ) @@ -219,7 +250,7 @@ class GeminiBackendTest extends munit.FunSuite: "list files", clientSid, displayPrompt = "list files", - LlmConfig.default.copy(systemPrompt = Some("be terse")), + AgentConfig.default.copy(systemPrompt = Some("be terse")), os.temp.dir(), outputSchema = None ) @@ -227,3 +258,87 @@ class GeminiBackendTest extends munit.FunSuite: assert(finalPrompt.contains("be terse")) assert(finalPrompt.contains("ask_user")) assert(finalPrompt.contains("list files")) + + // sessionExists probes the SERVER id, not the client id: it resolves the + // client→server mapping first (gemini mints its own id), then scans + // `--list-sessions` for that server id. A `registerSession` seeds the map. + + private val clientForProbe = SessionId[BackendTag.Gemini.type]("client-uuid") + private val serverForProbe = SessionId[BackendTag.Gemini.type]("sess-abc-123") + + test( + "sessionExists probes the SERVER id: true when it appears in --list-sessions" + ): + // clientForProbe ("client-uuid") and serverForProbe ("sess-abc-123") are + // distinct. The stub stdout contains the server id but NOT the client id, so + // the returned `true` can only be explained by the code scanning for the + // server id. A bug that scanned for the client id would return `false` + // (client id absent from stdout) and the `assert` below would fail. + val listOutput = "sess-abc-123 2024-01-01T00:00:00" + // Sanity: ensure client and server ids are truly distinct in the output. + assert( + !listOutput.contains(clientForProbe.value), + "test invariant: --list-sessions stdout must NOT contain the client id" + ) + val stub = new StubCliRunner(CliResult(0, listOutput, "")) + SupervisedBackend.using(new GeminiBackend(stub)): backend => + backend.registerSession(clientForProbe, serverForProbe) + assert(backend.sessionExists(clientForProbe)) + // Verify the probe used the correct command. + val probeArgs = stub.calls.head.args + assertEquals( + probeArgs, + List("gemini", "--list-sessions"), + s"sessionExists must invoke exactly `gemini --list-sessions`; got: $probeArgs" + ) + + test( + "sessionExists returns false when there is no client→server mapping" + ): + // No registerSession: the client id maps to nothing, so the probe must not + // run (and must not pass the client id to --list-sessions). + val stub = + new StubCliRunner(CliResult(0, "client-uuid 2024-01-01T00:00:00", "")) + SupervisedBackend.using(new GeminiBackend(stub)): backend => + assert(!backend.sessionExists(clientForProbe)) + + test( + "sessionExists returns false when the server id is not in the output" + ): + val stub = + new StubCliRunner(CliResult(0, "sess-other 2024-01-01T00:00:00", "")) + SupervisedBackend.using(new GeminiBackend(stub)): backend => + backend.registerSession(clientForProbe, serverForProbe) + assert(!backend.sessionExists(clientForProbe)) + + test( + "sessionExists returns false when gemini --list-sessions exits non-zero" + ): + val stub = new StubCliRunner(CliResult(1, "sess-abc-123", "")) + SupervisedBackend.using(new GeminiBackend(stub)): backend => + backend.registerSession(clientForProbe, serverForProbe) + assert(!backend.sessionExists(clientForProbe)) + + test( + "sessionExists returns false when the cli runner throws (verifies NonFatal catch)" + ): + val stub = new StubCliRunner(): + override def run( + args: Seq[String], + stdin: String, + env: Map[String, String], + cwd: os.Path + ): CliResult = throw new RuntimeException("binary not found") + SupervisedBackend.using(new GeminiBackend(stub)): backend => + backend.registerSession(clientForProbe, serverForProbe) + assert(!backend.sessionExists(clientForProbe)) + + test( + "sessionExists returns false for a malicious server id containing path chars" + ): + val maliciousServer = + SessionId[BackendTag.Gemini.type]("../../etc/passwd") + val stub = new StubCliRunner(CliResult(0, "../../etc/passwd", "")) + SupervisedBackend.using(new GeminiBackend(stub)): backend => + backend.registerSession(clientForProbe, maliciousServer) + assert(!backend.sessionExists(clientForProbe)) diff --git a/gemini/src/test/scala/orca/tools/gemini/GeminiCanAskUserTest.scala b/gemini/src/test/scala/orca/tools/gemini/GeminiCanAskUserTest.scala index 27e8e908..980c38a2 100644 --- a/gemini/src/test/scala/orca/tools/gemini/GeminiCanAskUserTest.scala +++ b/gemini/src/test/scala/orca/tools/gemini/GeminiCanAskUserTest.scala @@ -1,6 +1,6 @@ package orca.tools.gemini -import orca.llm.{BackendTag, CanAskUser} +import orca.agents.{BackendTag, CanAskUser} class GeminiCanAskUserTest extends munit.FunSuite: diff --git a/gemini/src/test/scala/orca/tools/gemini/GeminiConversationTest.scala b/gemini/src/test/scala/orca/tools/gemini/GeminiConversationTest.scala index cc8f936d..124c7e4a 100644 --- a/gemini/src/test/scala/orca/tools/gemini/GeminiConversationTest.scala +++ b/gemini/src/test/scala/orca/tools/gemini/GeminiConversationTest.scala @@ -1,20 +1,31 @@ package orca.tools.gemini -import orca.llm.SessionId +import orca.agents.SessionId import orca.events.Usage import orca.{OrcaFlowException, OrcaInteractiveCancelled} import orca.backend.ConversationEvent import orca.subprocess.FakePipedCliProcess +import ox.{Ox, supervised} class GeminiConversationTest extends munit.FunSuite: + /** `GeminiConversation` forks its reader/stderr/ask-user workers into the + * caller's per-turn Ox, so construction needs a `using Ox`. Run each test + * body in a fresh supervised scope that provides it. Tests managing their + * own scope (the ask-user ones) stay on plain `test`. + */ + private def convTest(name: String)(body: Ox ?=> Unit): Unit = + test(name)(supervised(body)) + /** Minimal terminating tail: a `result` event ends the turn and lets * `awaitResult` succeed. */ private def result(input: Long = 0L, output: Long = 0L): String = s"""{"type":"result","status":"success","stats":{"input_tokens":$input,"output_tokens":$output}}""" - test("assistant message accumulates into output; init sets session + model"): + convTest( + "assistant message accumulates into output; init sets session + model" + ): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -42,7 +53,7 @@ class GeminiConversationTest extends munit.FunSuite: assertEquals(r.usage, Usage(10L, 3L, None)) assertEquals(r.model.map(_.name), Some("gemini-2.5-pro")) - test("a user-role message is ignored (prompt echo, not agent output)"): + convTest("a user-role message is ignored (prompt echo, not agent output)"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -65,7 +76,7 @@ class GeminiConversationTest extends munit.FunSuite: val Right(r) = conv.awaitResult(): @unchecked assertEquals(r.output, "done") - test("multiple assistant chunks concatenate into the output"): + convTest("multiple assistant chunks concatenate into the output"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -84,7 +95,7 @@ class GeminiConversationTest extends munit.FunSuite: val Right(r) = conv.awaitResult(): @unchecked assertEquals(r.output, "foobar") - test("initialPrompt becomes a UserMessage event before agent output"): + convTest("initialPrompt becomes a UserMessage event before agent output"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process, initialPrompt = "do the thing") @@ -100,7 +111,7 @@ class GeminiConversationTest extends munit.FunSuite: assertEquals(events.head, ConversationEvent.UserMessage("do the thing")) val _ = conv.awaitResult() - test("tool_use + tool_result become AssistantToolCall + ToolResult"): + convTest("tool_use + tool_result become AssistantToolCall + ToolResult"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -133,7 +144,7 @@ class GeminiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("a tool whose name merely contains 'ask_user' is NOT suppressed"): + convTest("a tool whose name merely contains 'ask_user' is NOT suppressed"): // Suppression matches gemini's exact MCP qualification (orca__ask_user), // not any name containing the slug — an unrelated tool must still surface. val process = new FakePipedCliProcess() @@ -157,7 +168,7 @@ class GeminiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("tool_result with a non-success status yields ok=false"): + convTest("tool_result with a non-success status yields ok=false"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -179,7 +190,7 @@ class GeminiConversationTest extends munit.FunSuite: assertEquals(tr.ok, false) val _ = conv.awaitResult() - test("benign gemini stderr chatter is filtered (no Error events)"): + convTest("benign gemini stderr chatter is filtered (no Error events)"): // Observed on every successful headless run (gemini 0.45.2): a 256-color // warning, YOLO-mode notices, a cwd-reset line, and IDE-companion probe // chatter — all informational. @@ -214,7 +225,7 @@ class GeminiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("a real stderr line still surfaces as ConversationEvent.Error"): + convTest("a real stderr line still surfaces as ConversationEvent.Error"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -234,7 +245,7 @@ class GeminiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("error event surfaces as ConversationEvent.Error"): + convTest("error event surfaces as ConversationEvent.Error"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -254,7 +265,7 @@ class GeminiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("malformed JSONL surfaces as Error and the loop continues"): + convTest("malformed JSONL surfaces as Error and the loop continues"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -278,7 +289,7 @@ class GeminiConversationTest extends munit.FunSuite: val Right(r) = conv.awaitResult(): @unchecked assertEquals(r.output, "ok") - test("clean exit without a result event surfaces as OrcaFlowException"): + convTest("clean exit without a result event surfaces as OrcaFlowException"): val process = new FakePipedCliProcess(initiallyAlive = false) val conv = new GeminiConversation(process) @@ -293,7 +304,7 @@ class GeminiConversationTest extends munit.FunSuite: s"expected the missing-result message; got: ${ex.getMessage}" ) - test("a result event with a non-success status fails the turn"): + convTest("a result event with a non-success status fails the turn"): // gemini's `result` carries a status; a failed turn that still exits 0 // must not be reported as success. "success" is the documented good // token (headless stream-json) — anything else non-empty is a failure. @@ -317,7 +328,7 @@ class GeminiConversationTest extends munit.FunSuite: s"expected the failing status in the message; got: ${ex.getMessage}" ) - test("a result event with no status is treated as success"): + convTest("a result event with no status is treated as success"): // The status field is optional; an absent/empty status is success, not a // failed turn. val process = new FakePipedCliProcess() @@ -337,7 +348,7 @@ class GeminiConversationTest extends munit.FunSuite: val Right(r) = conv.awaitResult(): @unchecked assertEquals(r.output, "done") - test("interleaved tool calls are each keyed back to their own name"): + convTest("interleaved tool calls are each keyed back to their own name"): // Two tool calls complete out of order (B before A); each tool_result, // which carries only the id, must resolve to the right tool_name. val process = new FakePipedCliProcess() @@ -375,7 +386,9 @@ class GeminiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("cancel surfaces as Left(OrcaInteractiveCancelled) from awaitResult"): + convTest( + "cancel surfaces as Left(OrcaInteractiveCancelled) from awaitResult" + ): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) conv.cancel() @@ -385,22 +398,7 @@ class GeminiConversationTest extends munit.FunSuite: fail(s"expected Left(OrcaInteractiveCancelled), got: $other") assertEquals(process.sigIntCount, 1) - test("sendUserMessage is a documented no-op (no stdin write)"): - val process = new FakePipedCliProcess() - val conv = new GeminiConversation(process) - conv.sendUserMessage("ignored") - process.enqueueStdout("""{"type":"init","session_id":"s"}""") - process.enqueueStdout( - """{"type":"message","role":"assistant","content":"ok"}""" - ) - process.enqueueStdout(result()) - process.closeStdout() - process.closeStderr() - val _ = conv.events.toList - val _ = conv.awaitResult() - assertEquals(process.writes, Nil) - - test("unknown top-level events are ignored without surfacing"): + convTest("unknown top-level events are ignored without surfacing"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) @@ -423,11 +421,12 @@ class GeminiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("canAskUser is false when no bridge is provided"): + convTest("canAskUser is false when no bridge is provided"): val process = new FakePipedCliProcess() val conv = new GeminiConversation(process) assertEquals(conv.canAskUser, false) process.closeStdout() + process.closeStderr() val _ = conv.events.toList test("ask_user tool_use/tool_result are suppressed (no echo)"): diff --git a/gemini/src/test/scala/orca/tools/gemini/GeminiIntegrationTest.scala b/gemini/src/test/scala/orca/tools/gemini/GeminiIntegrationTest.scala index e319d1c4..b4f74306 100644 --- a/gemini/src/test/scala/orca/tools/gemini/GeminiIntegrationTest.scala +++ b/gemini/src/test/scala/orca/tools/gemini/GeminiIntegrationTest.scala @@ -1,6 +1,6 @@ package orca.tools.gemini -import orca.llm.{AutoApprove, BackendTag, LlmConfig, Model, SessionId} +import orca.agents.{AutoApprove, BackendTag, AgentConfig, Model, SessionId} import orca.backend.SupervisedBackend import orca.subprocess.OsProcCliRunner @@ -23,15 +23,15 @@ class GeminiIntegrationTest extends munit.FunSuite: import scala.concurrent.duration.DurationInt 3.minutes - private def withBackend(body: GeminiBackend => Unit): Unit = + private def withBackend(body: ox.Ox ?=> GeminiBackend => Unit): Unit = SupervisedBackend.using(new GeminiBackend(OsProcCliRunner))(body) // A cheap, widely-available model; override via ORCA_GEMINI_MODEL. private val model: Model = Model(sys.env.getOrElse("ORCA_GEMINI_MODEL", "gemini-2.5-flash")) - private val unsandboxed: LlmConfig = - LlmConfig.default.copy(autoApprove = AutoApprove.All, model = Some(model)) + private val unsandboxed: AgentConfig = + AgentConfig.default.copy(autoApprove = AutoApprove.All, model = Some(model)) private def fresh = SessionId.fresh[BackendTag.Gemini.type] diff --git a/opencode/src/main/scala/orca/tools/opencode/DefaultOpencodeAgent.scala b/opencode/src/main/scala/orca/tools/opencode/DefaultOpencodeAgent.scala new file mode 100644 index 00000000..8a8dbf73 --- /dev/null +++ b/opencode/src/main/scala/orca/tools/opencode/DefaultOpencodeAgent.scala @@ -0,0 +1,76 @@ +package orca.tools.opencode + +import orca.backend.{Interaction, AgentBackend} +import orca.events.OrcaListener +import orca.agents.{ + BackendTag, + BaseAgent, + AgentConfig, + Model, + OpencodeAgent, + Prompts +} + +/** Default [[OpencodeAgent]]. Inherits the autonomous-text + `resultAs[O]` + * plumbing from [[BaseAgent]] and adds OpenCode's provider-prefixed model + * accessors. The pinned ids are convenience defaults — any id from `opencode + * models` is valid; bump them as the catalog moves. + */ +private[orca] class DefaultOpencodeAgent( + backend: AgentBackend[BackendTag.Opencode.type], + config: AgentConfig, + prompts: Prompts, + workDir: os.Path, + events: OrcaListener, + interaction: Interaction, + val name: String = "main" +) extends BaseAgent[BackendTag.Opencode.type, OpencodeAgent]( + backend, + config, + prompts, + workDir, + events, + interaction + ) + with OpencodeAgent: + + def anthropicOpus: OpencodeAgent = withModel("anthropic", "claude-opus-4-8") + def anthropicSonnet: OpencodeAgent = + withModel("anthropic", "claude-sonnet-4-6") + def anthropicHaiku: OpencodeAgent = withModel("anthropic", "claude-haiku-4-5") + def openaiGpt5: OpencodeAgent = withModel("openai", "gpt-5.4") + def openaiGpt5Codex: OpencodeAgent = withModel("openai", "gpt-5.3-codex") + def openaiGpt5Mini: OpencodeAgent = withModel("openai", "gpt-5-mini") + + // Cheap is provider-matched so incidental work doesn't pull in a second + // provider's auth: an openai-led tool's cheap is an openai model, otherwise + // anthropic haiku (also the default when no model is pinned). Reads the + // provider prefix directly (not OpencodeModel.split, which throws on a bare + // id) so resolving the cheap model can never break a flow. + override protected def defaultCheap: OpencodeAgent = + config.model.map(m => Model.name(m).takeWhile(_ != '/')) match + case Some("openai") => openaiGpt5Mini + case _ => anthropicHaiku + + // Two-arg form validates and joins via OpencodeModel (one place); the + // accessors above share it. `withModel(String)` takes an already-joined id. + override def withModel(provider: String, modelId: String): OpencodeAgent = + super[BaseAgent].withModel(OpencodeModel(provider, modelId)) + + // `super` disambiguates from BaseAgent's protected `withModel(Model)`. + def withModel(providerModel: String): OpencodeAgent = + super[BaseAgent].withModel(Model(providerModel)) + + protected def copyTool( + config: AgentConfig = config, + name: String = name + ): OpencodeAgent = + new DefaultOpencodeAgent( + backend, + config, + prompts, + workDir, + events, + interaction, + name + ) diff --git a/opencode/src/main/scala/orca/tools/opencode/DefaultOpencodeTool.scala b/opencode/src/main/scala/orca/tools/opencode/DefaultOpencodeTool.scala deleted file mode 100644 index bf5f43c4..00000000 --- a/opencode/src/main/scala/orca/tools/opencode/DefaultOpencodeTool.scala +++ /dev/null @@ -1,66 +0,0 @@ -package orca.tools.opencode - -import orca.backend.{Interaction, LlmBackend} -import orca.events.OrcaListener -import orca.llm.{ - BackendTag, - BaseLlmTool, - LlmConfig, - Model, - OpencodeTool, - Prompts -} - -/** Default [[OpencodeTool]]. Inherits the autonomous-text + `resultAs[O]` - * plumbing from [[BaseLlmTool]] and adds OpenCode's provider-prefixed model - * accessors. The pinned ids are convenience defaults — any id from `opencode - * models` is valid; bump them as the catalog moves. - */ -private[orca] class DefaultOpencodeTool( - backend: LlmBackend[BackendTag.Opencode.type], - config: LlmConfig, - prompts: Prompts, - workDir: os.Path, - events: OrcaListener, - interaction: Interaction, - val name: String = "main" -) extends BaseLlmTool[BackendTag.Opencode.type, OpencodeTool]( - backend, - config, - prompts, - workDir, - events, - interaction - ) - with OpencodeTool: - - def anthropicOpus: OpencodeTool = withModel("anthropic", "claude-opus-4-8") - def anthropicSonnet: OpencodeTool = - withModel("anthropic", "claude-sonnet-4-6") - def anthropicHaiku: OpencodeTool = withModel("anthropic", "claude-haiku-4-5") - def openaiGpt5: OpencodeTool = withModel("openai", "gpt-5.4") - def openaiGpt5Codex: OpencodeTool = withModel("openai", "gpt-5.3-codex") - def openaiGpt5Mini: OpencodeTool = withModel("openai", "gpt-5-mini") - - // Two-arg form validates and joins via OpencodeModel (one place); the - // accessors above share it. `withModel(String)` takes an already-joined id. - override def withModel(provider: String, modelId: String): OpencodeTool = - super[BaseLlmTool].withModel(OpencodeModel(provider, modelId)) - - // `super` disambiguates from BaseLlmTool's protected `withModel(Model)`. - def withModel(providerModel: String): OpencodeTool = - super[BaseLlmTool].withModel(Model(providerModel)) - - protected def copyTool( - config: LlmConfig = config, - name: String = name - ): OpencodeTool = - new DefaultOpencodeTool( - backend, - config, - prompts, - workDir, - events, - interaction, - name - ) diff --git a/opencode/src/main/scala/orca/tools/opencode/JavaNetOpencodeHttp.scala b/opencode/src/main/scala/orca/tools/opencode/JavaNetOpencodeHttp.scala index 3797de2c..6bd7ab4f 100644 --- a/opencode/src/main/scala/orca/tools/opencode/JavaNetOpencodeHttp.scala +++ b/opencode/src/main/scala/orca/tools/opencode/JavaNetOpencodeHttp.scala @@ -56,6 +56,12 @@ private[opencode] class JavaNetOpencodeHttp(baseUrl: String, password: String) s"opencode POST $path failed: ${resp.statusCode()} ${resp.body()}" ) + override def getStatus(path: String): Int = + try + val req = request(path).GET().build() + client.send(req, HttpResponse.BodyHandlers.discarding()).statusCode() + catch case NonFatal(_) => 0 + def events(): StreamSource = val req = request("/event").GET().build() // `ofInputStream` returns once headers arrive; we read lines off the raw body diff --git a/opencode/src/main/scala/orca/tools/opencode/OpencodeArgs.scala b/opencode/src/main/scala/orca/tools/opencode/OpencodeArgs.scala index 4bc8394c..a2836e60 100644 --- a/opencode/src/main/scala/orca/tools/opencode/OpencodeArgs.scala +++ b/opencode/src/main/scala/orca/tools/opencode/OpencodeArgs.scala @@ -1,7 +1,7 @@ package orca.tools.opencode import orca.backend.{SessionMode, SystemPromptComposer} -import orca.llm.{LlmConfig, Model, ToolSet} +import orca.agents.{AgentConfig, Model, ToolSet} import orca.tools.opencode.OpencodeApi.{ MessageBody, MessagePart, @@ -10,7 +10,7 @@ import orca.tools.opencode.OpencodeApi.{ } import orca.util.RawJson -/** Maps an [[orca.llm.LlmConfig]] onto OpenCode's wire shapes: the `serve` +/** Maps an [[orca.agents.AgentConfig]] onto OpenCode's wire shapes: the `serve` * launch argv and the per-turn message body (ADR 0014). * * Unlike the subprocess backends, almost everything travels in the request @@ -42,7 +42,7 @@ private[opencode] object OpencodeArgs: * answer. */ def message( - config: LlmConfig, + config: AgentConfig, prompt: String, outputSchema: Option[String], mode: SessionMode @@ -72,7 +72,7 @@ private[opencode] object OpencodeArgs: * pre-fetch issue/PR context instead. */ private def toolFlags( - config: LlmConfig, + config: AgentConfig, mode: SessionMode ): Option[Map[String, Boolean]] = val writeGate = diff --git a/opencode/src/main/scala/orca/tools/opencode/OpencodeBackend.scala b/opencode/src/main/scala/orca/tools/opencode/OpencodeBackend.scala index 186bd1f3..7c57c988 100644 --- a/opencode/src/main/scala/orca/tools/opencode/OpencodeBackend.scala +++ b/opencode/src/main/scala/orca/tools/opencode/OpencodeBackend.scala @@ -8,19 +8,20 @@ import orca.backend.{ Conversation, Conversations, Dispatch, - LlmBackend, - LlmResult, + AgentBackend, + AgentResult, SessionMode, SessionRegistry, StreamSource } import orca.events.OrcaListener -import orca.llm.{BackendTag, LlmConfig, SessionId} +import orca.agents.{BackendTag, AgentConfig, SessionId} import orca.subprocess.CliRunner import orca.tools.opencode.OpencodeApi.{SessionCreateBody, SessionCreated} -import ox.Ox +import ox.{Ox, supervised} import java.util.concurrent.atomic.AtomicReference +import scala.util.control.NonFatal /** OpenCode backend (ADR 0014). Drives a shared `opencode serve` over HTTP+SSE. * @@ -44,12 +45,28 @@ private[orca] object OpencodeBackend: cli: CliRunner, launcher: OpencodeLauncher = OpencodeLauncher.default )(using Ox): OpencodeBackend = - new OpencodeBackend(workDir => - new OpencodeServer(cli, workDir, launcher).http + // Retain the server (created on first use) so its drain forks can be torn + // down at flow teardown via `shutdown` — see OpencodeServer's scaladoc. + val serverRef = new AtomicReference[OpencodeServer]() + new OpencodeBackend( + httpFor = workDir => { + val server = new OpencodeServer(cli, workDir, launcher) + serverRef.set(server) + server.http + }, + onShutdown = () => Option(serverRef.get()).foreach(_.shutdown()) ) -private[orca] class OpencodeBackend(httpFor: os.Path => OpencodeHttp)(using Ox) - extends LlmBackend[BackendTag.Opencode.type]: +private[orca] class OpencodeBackend( + httpFor: os.Path => OpencodeHttp, + onShutdown: () => Unit = () => () +) extends AgentBackend[BackendTag.Opencode.type]: + + /** Tear down the shared `opencode serve` process and its drain forks. A no-op + * if the server was never started (opencode wired but unused). Called by the + * runner in the flow body's `finally`, before the flow scope joins forks. + */ + def shutdown(): Unit = onShutdown() private val sessions = new SessionRegistry.ClientToServer[BackendTag.Opencode.type] @@ -68,14 +85,19 @@ private[orca] class OpencodeBackend(httpFor: os.Path => OpencodeHttp)(using Ox) def runAutonomous( prompt: String, session: SessionId[BackendTag.Opencode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener = OrcaListener.noop, outputSchema: Option[String] = None - ): LlmResult[BackendTag.Opencode.type] = + ): AgentResult[BackendTag.Opencode.type] = val http = server(workDir) - val source = http.events() - try + // Self-scoped: the conversation forks its reader into this per-turn Ox, the + // drain consumes it, and `cancel` (the `finally`) POSTs `/abort`, interrupts + // the SSE source, and finalizes — tearing the stream + forks down before the + // scope joins. `drainAutonomous` doesn't tear down, so the `finally` is + // load-bearing (and harmless on the happy path). + supervised: + val source = http.events() val conv = openConversation( http, source, @@ -85,20 +107,20 @@ private[orca] class OpencodeBackend(httpFor: os.Path => OpencodeHttp)(using Ox) outputSchema, SessionMode.Autonomous ) - val result = Conversations.drainAutonomous(conv, events) - sessions.commitSuccess(session, result.sessionId) - result.copy(sessionId = session) // keep the caller's id as the handle - finally - source.interrupt() // release the SSE connection at turn end (idempotent) + try + val result = Conversations.drainAutonomous(conv, events) + sessions.commitSuccess(session, result.sessionId) + result.copy(sessionId = session) // keep the caller's id as the handle + finally conv.cancel() def runInteractive( prompt: String, session: SessionId[BackendTag.Opencode.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Opencode.type] = + )(using Ox): Conversation[BackendTag.Opencode.type] = val http = server(workDir) // The returned conversation owns its stream: it interrupts on the terminal // event or `cancel`, so no scope-level backstop is needed here. @@ -117,6 +139,39 @@ private[orca] class OpencodeBackend(httpFor: os.Path => OpencodeHttp)(using Ox) serverSession: SessionId[BackendTag.Opencode.type] ): Unit = sessions.commitSuccess(client, serverSession) + override def resumeWireId( + client: SessionId[BackendTag.Opencode.type] + ): Option[SessionId[BackendTag.Opencode.type]] = sessions.resumeWireId(client) + + /** Probe `http` for the given session id via `GET /session/<id>` → status + * 200. Callable directly in tests without going through the lazy-init guard. + * Returns `false` on any transport error. The + * [[orca.agents.isSafeSessionId]] guard must have passed before this method + * is called — it is not re-checked here. + */ + private[opencode] def probeSession(id: String, http: OpencodeHttp): Boolean = + try http.getStatus(s"/session/$id") == 200 + catch case NonFatal(_) => false + + /** Best-effort probe: resolves the SERVER id (`ses_…`) mapped to `client` + * (opencode mints server-side ids; the caller's stable id never matches one) + * and checks `GET /session/<serverId>` → 200. Returns `false` — safe re-seed + * — when no server id is mapped (the map hasn't been rehydrated from the + * log, so there is no known live session), the opencode server has not been + * started yet, the request fails for any reason, or the id fails the + * [[orca.agents.isSafeSessionId]] guard (blocks URL injection such as `a/b` + * routing to a different endpoint). + * + * The client→server map is persisted in the progress log and rehydrated on + * resume (D2), so on a resumed run this targets the right server id. + */ + override def sessionExists( + session: SessionId[BackendTag.Opencode.type] + ): Boolean = + probeServerSession(session, sessions): id => + if firstWorkDir.get() == null then false + else probeSession(id, sharedServer) + /** The server `ses_…` to drive: a fresh `POST /session`, or the one a prior * turn registered for this caller id. */ @@ -138,11 +193,11 @@ private[orca] class OpencodeBackend(httpFor: os.Path => OpencodeHttp)(using Ox) http: OpencodeHttp, source: StreamSource, serverSession: String, - config: LlmConfig, + config: AgentConfig, prompt: String, outputSchema: Option[String], mode: SessionMode - ): OpencodeConversation = + )(using Ox): OpencodeConversation = val displayPrompt = mode match case SessionMode.Interactive(p) => p case SessionMode.Autonomous => "" diff --git a/opencode/src/main/scala/orca/tools/opencode/OpencodeConversation.scala b/opencode/src/main/scala/orca/tools/opencode/OpencodeConversation.scala index 237eadf6..2280e1a0 100644 --- a/opencode/src/main/scala/orca/tools/opencode/OpencodeConversation.scala +++ b/opencode/src/main/scala/orca/tools/opencode/OpencodeConversation.scala @@ -5,12 +5,13 @@ import orca.AgentTurnFailed import orca.backend.{ ApprovalDecision, ConversationEvent, - LlmResult, - StreamConversation, + AgentResult, + ForkedConversation, StreamSource } import orca.events.Usage -import orca.llm.{BackendTag, Model, SessionId} +import orca.agents.{BackendTag, Model, SessionId} +import ox.Ox import orca.tools.opencode.OpencodeApi.{ AssistantInfo, PermissionReply, @@ -26,9 +27,9 @@ import scala.util.control.NonFatal * 0014). * * The reader-loop / event-queue / outcome lifecycle lives in - * [[StreamConversation]]; this class supplies the OpenCode-specific + * [[ForkedConversation]]; this class supplies the OpenCode-specific * translation: SSE frame → [[OpencodeEvent]] → `ConversationEvent`, deriving - * the [[LlmResult]] from the assistant `message.updated` at `session.idle`. + * the [[AgentResult]] from the assistant `message.updated` at `session.idle`. * The SSE stream stays open after a turn, so reaching the terminal interrupts * `source` to make the reader observe EOF. * @@ -43,28 +44,24 @@ private[opencode] class OpencodeConversation( val outputSchema: Option[String], canAsk: Boolean, initialPrompt: String = "" -) extends StreamConversation[BackendTag.Opencode.type]( +)(using Ox) + extends ForkedConversation[BackendTag.Opencode.type]( source, "opencode", initialPrompt, nativeAskUser = canAsk ): - /** Follow-up turns are issued as separate `runInteractive` calls (a fresh - * `prompt_async`); mid-turn injection isn't wired. A turn paused on - * `ask_user` resumes via the question reply, not this. - */ - def sendUserMessage(text: String): Unit = () - /** Best-effort `POST /session/{id}/abort` before closing the stream, so a * cancelled turn stops running (and writing) on the shared server instead of - * continuing headless after the user has moved on. + * continuing headless after the user has moved on. `super.cancel` (the base + * [[ForkedConversation.cancel]]) is idempotent, so a second call is a no-op + * past the best-effort abort. */ override def cancel(): Unit = - if !cancelled.get() then - try - val _ = http.postJson(s"/session/$session/abort", "{}") - catch case NonFatal(_) => () + try + val _ = http.postJson(s"/session/$session/abort", "{}") + catch case NonFatal(_) => () super.cancel() /** Turn state, accumulated as the reader thread processes frames. @@ -75,6 +72,13 @@ private[opencode] class OpencodeConversation( */ private var turnState: TurnState = TurnState() + /** Flips once [[finishTurn]]/[[failTurn]] settle the outcome, so frames the + * SSE stream emits after the terminal event are ignored. Written and read + * only on the reader thread (inside [[handleLine]]), so a plain `var` is + * safe — see [[turnState]]. + */ + private var settled: Boolean = false + private case class TurnState( text: Vector[String] = Vector.empty, info: Option[AssistantInfo] = None, @@ -85,7 +89,7 @@ private[opencode] class OpencodeConversation( sseData(rawLine).foreach: json => val event = OpencodeEvent.parse(json) // Drop other sessions' frames; once the turn has settled, ignore the rest. - if forThisSession(event) && outcomeRef.get().isEmpty then translate(event) + if forThisSession(event) && !settled then translate(event) /** The JSON payload of one SSE line, or `None` for blank / comment / framing * lines (`event:`, `id:`, heartbeat `:`). @@ -144,18 +148,20 @@ private[opencode] class OpencodeConversation( failTurn("session went idle without an assistant message") else eventQueue.enqueue(ConversationEvent.AssistantTurnEnd) + settled = true succeedWith(buildResult()) private def failTurn(message: String): Unit = + settled = true failWith(AgentTurnFailed(message)) /** In structured mode the validated object is the result; otherwise the * accrued assistant text. Usage and model come from the captured `info`. */ - private def buildResult(): LlmResult[BackendTag.Opencode.type] = + private def buildResult(): AgentResult[BackendTag.Opencode.type] = val info = turnState.info val structured = info.flatMap(_.structured).map(_.value) - LlmResult( + AgentResult( sessionId = SessionId[BackendTag.Opencode.type](session), output = structured.getOrElse(turnState.text.mkString), usage = usageOf(info), @@ -194,5 +200,3 @@ private[opencode] class OpencodeConversation( s"/permission/${req.id}/reply", writeToString(PermissionReplyBody(verdict)) ) - - start() diff --git a/opencode/src/main/scala/orca/tools/opencode/OpencodeEvent.scala b/opencode/src/main/scala/orca/tools/opencode/OpencodeEvent.scala index dd7b8de9..43238f86 100644 --- a/opencode/src/main/scala/orca/tools/opencode/OpencodeEvent.scala +++ b/opencode/src/main/scala/orca/tools/opencode/OpencodeEvent.scala @@ -36,7 +36,8 @@ private[opencode] enum OpencodeEvent: output: String ) - /** The assistant message metadata updated — the source of the `LlmResult`. */ + /** The assistant message metadata updated — the source of the `AgentResult`. + */ case MessageUpdated(session: String, info: AssistantInfo) case QuestionAsked(request: QuestionRequest) case PermissionAsked(request: PermissionRequest) diff --git a/opencode/src/main/scala/orca/tools/opencode/OpencodeHttp.scala b/opencode/src/main/scala/orca/tools/opencode/OpencodeHttp.scala index afab391b..2d6f3d27 100644 --- a/opencode/src/main/scala/orca/tools/opencode/OpencodeHttp.scala +++ b/opencode/src/main/scala/orca/tools/opencode/OpencodeHttp.scala @@ -24,6 +24,12 @@ private[opencode] trait OpencodeHttp: */ def events(): StreamSource + /** GET `path` (relative to the server base) and return the HTTP status code. + * Returns 0 when the server is unreachable or the request fails for any + * reason — callers treat any non-200 as "session not found". + */ + def getStatus(path: String): Int = 0 + /** Release transport resources (the HTTP client). Default no-op for stubs; * the real client shuts down its connection pool and selector thread. Called * once at scope teardown. diff --git a/opencode/src/main/scala/orca/tools/opencode/OpencodeModel.scala b/opencode/src/main/scala/orca/tools/opencode/OpencodeModel.scala index b3cd02e0..b4b05839 100644 --- a/opencode/src/main/scala/orca/tools/opencode/OpencodeModel.scala +++ b/opencode/src/main/scala/orca/tools/opencode/OpencodeModel.scala @@ -1,7 +1,7 @@ package orca.tools.opencode import orca.OrcaFlowException -import orca.llm.Model +import orca.agents.Model /** Construction and parsing of OpenCode's `provider/model` identifiers. * diff --git a/opencode/src/main/scala/orca/tools/opencode/OpencodeServer.scala b/opencode/src/main/scala/orca/tools/opencode/OpencodeServer.scala index bd4197b4..82f725c3 100644 --- a/opencode/src/main/scala/orca/tools/opencode/OpencodeServer.scala +++ b/opencode/src/main/scala/orca/tools/opencode/OpencodeServer.scala @@ -1,26 +1,35 @@ package orca.tools.opencode import orca.OrcaFlowException -import orca.subprocess.CliRunner -import ox.{releaseAfterScope, Ox} +import orca.subprocess.{CliRunner, PipedCliProcess} +import ox.{fork, forkDiscard, Ox} import org.slf4j.LoggerFactory import java.util.UUID import java.util.concurrent.ConcurrentLinkedDeque +import java.util.concurrent.atomic.{AtomicBoolean, AtomicReference} +import scala.util.control.NonFatal /** Lifecycle owner for a shared `opencode serve` process (ADR 0014): it spawns - * the server, reads its base URL, and tears the process down at scope end. The - * HTTP/SSE client to talk to it is exposed via [[http]] — this class *owns* a - * client rather than *being* one, keeping process lifecycle separate from the - * request surface ([[OpencodeHttp]]). + * the server, reads its base URL, and tears the process down via [[shutdown]]. + * The HTTP/SSE client to talk to it is exposed via [[http]] — this class + * *owns* a client rather than *being* one, keeping process lifecycle separate + * from the request surface ([[OpencodeHttp]]). * * The process is spawned the first time [[http]] is forced (so a backend wired - * but never used starts nothing), and both the process and client are torn - * down when the enclosing Ox scope ends. A random `OPENCODE_SERVER_PASSWORD` - * keeps the bound localhost port closed to other processes; `--pure` is *not* - * passed (`OpencodeArgs.serve`) so the server inherits the user's configured - * providers. + * but never used starts nothing). Its stdout/stderr are drained by Ox forks + * bound to the enclosing (flow) scope; [[shutdown]] destroys the process — + * which unblocks those forks' non-interruptible reads — so the scope can then + * join them cleanly. `shutdown` MUST be called from the flow body's `finally` + * (before the scope joins its forks): Ox runs `releaseAfterScope` finalizers + * *after* the join, so a `releaseAfterScope`-based kill would deadlock on a + * fork blocked in `readLine`. The runner wires this via + * `DefaultFlowContext.close`. + * + * A random `OPENCODE_SERVER_PASSWORD` keeps the bound localhost port closed to + * other processes; `--pure` is *not* passed (`OpencodeArgs.serve`) so the + * server inherits the user's configured providers. */ private[opencode] class OpencodeServer( cli: CliRunner, @@ -29,21 +38,47 @@ private[opencode] class OpencodeServer( httpFor: (String, String) => OpencodeHttp = JavaNetOpencodeHttp.start )(using Ox): - // Server lifecycle goes to the per-run trace (/tmp/orca-*.log). The raw - // `spawn:` line (orca.proc) shows `--port 0`, so log the resolved URL — and a - // teardown line, since a long-lived shared server starting/stopping silently - // is otherwise invisible. private val log = LoggerFactory.getLogger(classOf[OpencodeServer]) - /** The HTTP/SSE client against this server. Forcing it spawns `opencode serve` - * exactly once: a `lazy val` gives one spawn under concurrent first use and - * does not cache a failed start (Scala re-runs the initializer if it threw). - * This is the load-bearing once-init — `OpencodeBackend`'s AtomicReference - * only guarantees a single server *instance*; this guarantees a single - * *spawn*. + // Set during start() so shutdown() can tear them down. The reader forks block + // on a non-interruptible read, so destroying the process is the only way to + // unblock them — see the class scaladoc. + private val processRef = new AtomicReference[PipedCliProcess]() + private val clientRef = new AtomicReference[OpencodeHttp]() + private val stopped = new AtomicBoolean(false) + + /** The HTTP/SSE client against this server. Forcing it spawns `opencode + * serve` exactly once: a `lazy val` gives one spawn under concurrent first + * use and does not cache a failed start (Scala re-runs the initializer if it + * threw). This is the load-bearing once-init — `OpencodeBackend`'s lazy + * `sharedServer` only guarantees a single server *instance*; this `lazy val` + * guarantees a single *spawn*. */ lazy val http: OpencodeHttp = start() + /** Tear down the server: tree-destroy the process (unblocking the drain + * forks' reads so the enclosing scope can join them) and close the HTTP + * client. Idempotent and a no-op if the server was never started. Must run + * in the flow body's `finally`, before the scope joins the drain forks (see + * class scaladoc). + * + * In the runner's normal flow `http` is forced during the body and + * `shutdown` runs in the same scope's `finally` afterwards, so the process + * is already recorded here. Should a background fork ever force `http` + * concurrently with this call and lose the `processRef` write/read race, + * `start` re-checks `stopped` after spawning and tree-destroys the process + * itself — so the kill is never silently missed. + */ + def shutdown(): Unit = + if stopped.compareAndSet(false, true) then + val proc = Option(processRef.get()) + if proc.isDefined then log.debug("opencode server stopping") + // Tree-destroy (not SIGINT, not PID-only) so EVERY pipe holder dies and + // the drains' native reads hit EOF before the enclosing scope joins them — + // a launch wrapper (ollama) forks the real serve, which inherits the pipes. + proc.foreach(_.destroyForciblyTree()) + Option(clientRef.get()).foreach(_.close()) + private def start(): OpencodeHttp = val password = UUID.randomUUID.toString // Pipe stderr (don't inherit it): a failed launch — e.g. `ollama launch` @@ -55,27 +90,22 @@ private[opencode] class OpencodeServer( cwd = workDir, pipeStderr = true ) + processRef.set(process) process.closeStdin() - // Tree, not single-PID: a launch wrapper (e.g. `ollama launch opencode`) - // may fork the real serve process, which a PID-only SIGINT would orphan. - releaseAfterScope(process.sendSigIntTree()) - // Drain stderr in a daemon (a chatty launcher mustn't fill the pipe and - // stall startup), tracing each line and keeping a bounded tail to report if - // the server never binds. + // Drain stderr in a fork (a chatty launcher mustn't fill the pipe and stall + // startup), tracing each line and keeping a bounded tail to report if the + // server never binds. A joinable `fork` (not `forkDiscard`) so the bind- + // failure path can wait for the tail; the body swallows NonFatal so a stray + // read error never tears down the flow scope. val errTail = new ConcurrentLinkedDeque[String]() - val errDrain = new Thread( - () => { - process.stderrLines.foreach { line => + val errFork = fork: + try + process.stderrLines.foreach: line => log.debug("opencode serve stderr: {}", line) errTail.addLast(line) while errTail.size > OpencodeServer.MaxErrTailLines do val _ = errTail.poll() - } - }, - "opencode-stderr-drain" - ) - errDrain.setDaemon(true) - errDrain.start() + catch case NonFatal(e) => log.debug("opencode stderr drain ended", e) // serve prints "listening on …" within ~1s of binding; a serve that exits // without it surfaces as EOF here. val out = process.stdoutLines @@ -84,9 +114,11 @@ private[opencode] class OpencodeServer( .nextOption() .getOrElse: // stdout closed with no listening line — the launcher/serve exited. - // Surface its stderr (e.g. ollama's "model not found; run 'ollama pull - // …'") rather than a bare "no URL" message. - errDrain.join(OpencodeServer.StderrFlushMillis) + // Tree-destroy first so the stderr fork's read EOFs (even if a wrapper + // forked a pipe-holding child) and the join below can't hang, then + // surface its stderr (e.g. ollama's "model not found"). + process.destroyForciblyTree() + errFork.join() val tail = String.join("\n", errTail) throw OrcaFlowException( "opencode serve did not start" + @@ -96,19 +128,17 @@ private[opencode] class OpencodeServer( log.debug("opencode server started, listening on {}", baseUrl) // Keep draining stdout — resuming the *same* lazy iterator past the bind // line — so the server's log output can't back-fill the pipe and stall it. - // A daemon thread, not an Ox fork: the drain blocks in a native `readLine` - // that thread interruption can't cancel, so an Ox fork would hang scope - // teardown forever waiting to join it (the SIGINT that would unblock it runs - // only *after* the join). The daemon thread ends when teardown SIGINTs the - // process (stdout EOF), and never blocks JVM exit regardless. - val drain = new Thread(() => out.foreach(_ => ()), "opencode-stdout-drain") - drain.setDaemon(true) - drain.start() + // A `forkDiscard` in the flow scope; `shutdown`'s destroy unblocks its read + // before the scope joins it (the read is native and interrupt-immune). + forkDiscard: + try out.foreach(_ => ()) + catch case NonFatal(e) => log.debug("opencode stdout drain ended", e) + // Close the shutdown-before-processRef window structurally: if `shutdown` + // latched `stopped` before `processRef` was set, it destroyed nothing — do + // it here so the drains we just forked don't outlive that shutdown. + if stopped.get() then process.destroyForciblyTree() val client = httpFor(baseUrl, password) - releaseAfterScope(client.close()) // runs before the SIGINT (LIFO) - // Registered last → runs first at teardown, announcing the stop before the - // client close + SIGINT above. - releaseAfterScope(log.debug("opencode server at {} stopping", baseUrl)) + clientRef.set(client) client private[opencode] object OpencodeServer: @@ -117,9 +147,6 @@ private[opencode] object OpencodeServer: /** Cap on stderr lines kept for a start-failure message. */ private val MaxErrTailLines = 50 - /** Grace for the stderr drain to finish after stdout EOF, before reporting. */ - private val StderrFlushMillis = 2000L - /** The base URL from a serve startup line (`opencode server listening on * http://127.0.0.1:4096`), or `None`. */ diff --git a/opencode/src/test/scala/orca/tools/opencode/DefaultOpencodeToolTest.scala b/opencode/src/test/scala/orca/tools/opencode/DefaultOpencodeToolTest.scala index 46aeaf42..7dee8b5b 100644 --- a/opencode/src/test/scala/orca/tools/opencode/DefaultOpencodeToolTest.scala +++ b/opencode/src/test/scala/orca/tools/opencode/DefaultOpencodeToolTest.scala @@ -1,51 +1,54 @@ package orca.tools.opencode -import orca.backend.{Conversation, Interaction, LlmBackend, LlmResult} +import orca.backend.{Conversation, Interaction, AgentBackend, AgentResult} import orca.events.{OrcaListener, Usage} -import orca.llm.{ +import orca.agents.{ BackendTag, DefaultPrompts, - LlmConfig, - OpencodeTool, + AgentConfig, + OpencodeAgent, SessionId, ToolSet } -class DefaultOpencodeToolTest extends munit.FunSuite: +class DefaultOpencodeAgentTest extends munit.FunSuite: + + // LLM `run` is now gated on `InStage`; mint the token for the suite. + private given orca.InStage = orca.InStage.unsafe /** Captures the config the tool resolves for an autonomous call. */ - private class RecordingBackend extends LlmBackend[BackendTag.Opencode.type]: - var lastConfig: Option[LlmConfig] = None + private class RecordingBackend extends AgentBackend[BackendTag.Opencode.type]: + var lastConfig: Option[AgentConfig] = None def runAutonomous( prompt: String, session: SessionId[BackendTag.Opencode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener, outputSchema: Option[String] - ): LlmResult[BackendTag.Opencode.type] = + ): AgentResult[BackendTag.Opencode.type] = lastConfig = Some(config) - LlmResult(session, "ok", Usage(0L, 0L, None)) + AgentResult(session, "ok", Usage(0L, 0L, None)) def runInteractive( prompt: String, session: SessionId[BackendTag.Opencode.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Opencode.type] = + )(using ox.Ox): Conversation[BackendTag.Opencode.type] = throw new UnsupportedOperationException private val noInteraction: Interaction = new Interaction: def listeners: List[OrcaListener] = Nil def drive[B <: BackendTag]( conversation: Conversation[B] - ): LlmResult[B] = throw new UnsupportedOperationException + ): AgentResult[B] = throw new UnsupportedOperationException - private def toolWith(backend: RecordingBackend): OpencodeTool = - new DefaultOpencodeTool( + private def toolWith(backend: RecordingBackend): OpencodeAgent = + new DefaultOpencodeAgent( backend, - LlmConfig.default, + AgentConfig.default, DefaultPrompts, os.temp.dir(), OrcaListener.noop, @@ -54,7 +57,7 @@ class DefaultOpencodeToolTest extends munit.FunSuite: /** Run an autonomous call and return the model id the backend saw. */ private def modelOf( - tool: OpencodeTool, + tool: OpencodeAgent, backend: RecordingBackend ): Option[String] = val _ = tool.autonomous.run("x") diff --git a/opencode/src/test/scala/orca/tools/opencode/OpencodeArgsTest.scala b/opencode/src/test/scala/orca/tools/opencode/OpencodeArgsTest.scala index 583a24c3..03c53f3a 100644 --- a/opencode/src/test/scala/orca/tools/opencode/OpencodeArgsTest.scala +++ b/opencode/src/test/scala/orca/tools/opencode/OpencodeArgsTest.scala @@ -1,7 +1,7 @@ package orca.tools.opencode import orca.backend.{SessionMode, SystemPromptComposer} -import orca.llm.{AutoApprove, LlmConfig, Model, ToolSet} +import orca.agents.{AutoApprove, AgentConfig, Model, ToolSet} class OpencodeArgsTest extends munit.FunSuite: @@ -22,13 +22,25 @@ class OpencodeArgsTest extends munit.FunSuite: test("serve prefixes a custom launcher before the serve args"): assertEquals( OpencodeArgs.serve(OpencodeLauncher.ollama("qwen3-coder")), - Seq("ollama", "launch", "opencode", "--model", "qwen3-coder", "--", - "serve", "--port", "0", "--log-level", "WARN") + Seq( + "ollama", + "launch", + "opencode", + "--model", + "qwen3-coder", + "--", + "serve", + "--port", + "0", + "--log-level", + "WARN" + ) ) test("message splits the model into provider/id"): val body = OpencodeArgs.message( - LlmConfig.default.copy(model = Some(Model("anthropic/claude-opus-4-8"))), + AgentConfig.default + .copy(model = Some(Model("anthropic/claude-opus-4-8"))), "hi", outputSchema = None, interactive @@ -41,7 +53,7 @@ class OpencodeArgsTest extends munit.FunSuite: test("message splits a multi-slash (self-hosted) model on the first / only"): val body = OpencodeArgs.message( - LlmConfig.default + AgentConfig.default .copy(model = Some(Model("lmstudio/google/gemma-3n-e4b"))), "hi", None, @@ -54,17 +66,18 @@ class OpencodeArgsTest extends munit.FunSuite: test("message omits the model when config has none (server default)"): assertEquals( - OpencodeArgs.message(LlmConfig.default, "hi", None, interactive).model, + OpencodeArgs.message(AgentConfig.default, "hi", None, interactive).model, None ) test("message carries the composed system prompt (RuntimeOwnsGit rule)"): - val body = OpencodeArgs.message(LlmConfig.default, "hi", None, interactive) + val body = + OpencodeArgs.message(AgentConfig.default, "hi", None, interactive) assertEquals(body.system, Some(SystemPromptComposer.RuntimeOwnsGit)) test("structured turn sets format=json_schema with the schema verbatim"): val body = OpencodeArgs.message( - LlmConfig.default, + AgentConfig.default, "hi", outputSchema = Some("""{"type":"object"}"""), interactive @@ -73,16 +86,17 @@ class OpencodeArgsTest extends munit.FunSuite: assertEquals(body.format.map(_.schema.value), Some("""{"type":"object"}""")) test("autonomous turn disables the question tool"): - val body = OpencodeArgs.message(LlmConfig.default, "hi", None, autonomous) + val body = OpencodeArgs.message(AgentConfig.default, "hi", None, autonomous) assertEquals(body.tools.flatMap(_.get("question")), Some(false)) test("interactive turn leaves the question tool enabled (no tools gate)"): - val body = OpencodeArgs.message(LlmConfig.default, "hi", None, interactive) + val body = + OpencodeArgs.message(AgentConfig.default, "hi", None, interactive) assertEquals(body.tools, None) test("read-only turn disables the write tools (write/edit/bash/patch)"): val cfg = - LlmConfig.default.copy( + AgentConfig.default.copy( tools = ToolSet.ReadOnly, autoApprove = AutoApprove.All ) @@ -99,7 +113,7 @@ class OpencodeArgsTest extends munit.FunSuite: test("NetworkOnly keeps bash disabled (no writable-shell network)"): // opencode has no scoped network: NetworkOnly gates the same write tools as // ReadOnly, so the planner can't shell out to `gh`. - val cfg = LlmConfig.default.copy(tools = ToolSet.NetworkOnly) + val cfg = AgentConfig.default.copy(tools = ToolSet.NetworkOnly) val tools = OpencodeArgs .message(cfg, "hi", None, interactive) @@ -109,7 +123,7 @@ class OpencodeArgsTest extends munit.FunSuite: assertEquals(tools.get("edit"), Some(false)) test("read-only autonomous turn gates both write tools and question"): - val cfg = LlmConfig.default.copy(tools = ToolSet.ReadOnly) + val cfg = AgentConfig.default.copy(tools = ToolSet.ReadOnly) val tools = OpencodeArgs .message(cfg, "hi", None, autonomous) diff --git a/opencode/src/test/scala/orca/tools/opencode/OpencodeBackendTest.scala b/opencode/src/test/scala/orca/tools/opencode/OpencodeBackendTest.scala index 21ae6880..9c601488 100644 --- a/opencode/src/test/scala/orca/tools/opencode/OpencodeBackendTest.scala +++ b/opencode/src/test/scala/orca/tools/opencode/OpencodeBackendTest.scala @@ -1,15 +1,19 @@ package orca.tools.opencode import orca.backend.StreamSource -import orca.llm.{BackendTag, LlmConfig, Model, SessionId} +import orca.agents.{BackendTag, AgentConfig, Model, SessionId} import ox.supervised class OpencodeBackendTest extends munit.FunSuite: /** Serves a canned turn over SSE and records POSTs. `events` hands back the - * same canned stream each call. + * same canned stream each call. `statusFor` controls what `getStatus` + * returns for each path prefix: defaults to 404 for unknown paths. */ - private class FakeHttp(sse: List[String]) extends OpencodeHttp: + private class FakeHttp( + sse: List[String], + statusFor: String => Int = _ => 404 + ) extends OpencodeHttp: var posts: List[(String, String)] = Nil def postJson(path: String, body: String): String = posts = posts :+ (path -> body) @@ -19,6 +23,7 @@ class OpencodeBackendTest extends munit.FunSuite: def errorLines: Iterator[String] = Iterator.empty def interrupt(): Unit = () def tryExitCode: Option[Int] = Some(0) + override def getStatus(path: String): Int = statusFor(path) private def data(json: String): String = s"data: $json" @@ -56,15 +61,22 @@ class OpencodeBackendTest extends munit.FunSuite: val backend = new OpencodeBackend(_ => http) val client = fresh val result = - backend.runAutonomous("hi", client, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("hi", client, AgentConfig.default, os.temp.dir()) assertEquals(result.output, "done") assertEquals(result.model, Some(Model("gpt-4o-mini"))) // The caller's id stays the handle; the server id is hidden. assertEquals(result.sessionId, client) + // The turn finalizes through `conv.cancel()` (the self-scoped per-turn + // `finally`), whose best-effort `POST /abort` trails the turn — a no-op on + // the already-idle session. assertEquals( http.posts.map(_._1), - List("/session", "/session/ses_server1/prompt_async") + List( + "/session", + "/session/ses_server1/prompt_async", + "/session/ses_server1/abort" + ) ) // The backend forwards the prompt into the prompt_async body. val (_, body) = http.posts.find(_._1.endsWith("/prompt_async")).get @@ -78,9 +90,9 @@ class OpencodeBackendTest extends munit.FunSuite: val backend = new OpencodeBackend(_ => http) val client = fresh val _ = - backend.runAutonomous("one", client, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("one", client, AgentConfig.default, os.temp.dir()) val _ = - backend.runAutonomous("two", client, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("two", client, AgentConfig.default, os.temp.dir()) assertEquals(http.posts.count(_._1 == "/session"), 1) assertEquals(http.posts.count(_._1.endsWith("/prompt_async")), 2) @@ -94,7 +106,7 @@ class OpencodeBackendTest extends munit.FunSuite: SessionId[BackendTag.Opencode.type]("ses_X") ) val _ = - backend.runAutonomous("hi", client, LlmConfig.default, os.temp.dir()) + backend.runAutonomous("hi", client, AgentConfig.default, os.temp.dir()) assertEquals( http.posts.count(_._1 == "/session"), 0 @@ -119,7 +131,7 @@ class OpencodeBackendTest extends munit.FunSuite: "q", fresh, "display", - LlmConfig.default, + AgentConfig.default, os.temp.dir(), outputSchema = Some("""{"type":"object"}""") ) @@ -130,3 +142,108 @@ class OpencodeBackendTest extends munit.FunSuite: ) // schema threaded through conv.events.foreach(_ => ()) assertEquals(conv.awaitResult().toOption.get.output, "hi") + + test("sessionExists returns false when the server has not been started yet"): + supervised: + val http = new FakeHttp(Nil, _ => 200) + val backend = new OpencodeBackend(_ => http) + // No runAutonomous call — no client→server mapping AND firstWorkDir null. + assert(!backend.sessionExists(fresh)) + + test( + "sessionExists returns false when there is no client→server mapping" + ): + supervised: + // Server started (would answer 200), but the probed client id was never + // mapped to a server id, so the probe must not run on the client id. + val existingId = "ses_server1" + val http = new FakeHttp( + turn(existingId, "stop", Nil), + path => if path == s"/session/$existingId" then 200 else 404 + ) + val backend = new OpencodeBackend(_ => http) + val _ = + backend.runAutonomous("hi", fresh, AgentConfig.default, os.temp.dir()) + // A different, unmapped client id resolves to no server id → false. + assert(!backend.sessionExists(fresh)) + + test("probeSession returns true when getStatus is 200"): + supervised: + val http = new FakeHttp( + Nil, + path => if path == "/session/ses_abc" then 200 else 404 + ) + val backend = new OpencodeBackend(_ => http) + assert(backend.probeSession("ses_abc", http)) + + test("probeSession returns false when getStatus is 404"): + supervised: + val http = new FakeHttp(Nil, _ => 404) + val backend = new OpencodeBackend(_ => http) + assert(!backend.probeSession("ses_missing", http)) + + test( + "sessionExists probes the SERVER id: true after a turn maps client→server" + ): + supervised: + val existingId = "ses_server1" + val http = new FakeHttp( + turn(existingId, "stop", Nil), + path => if path == s"/session/$existingId" then 200 else 404 + ) + val backend = new OpencodeBackend(_ => http) + val client = fresh + // A real turn maps client → ses_server1 in the registry. + val _ = + backend.runAutonomous("hi", client, AgentConfig.default, os.temp.dir()) + // Probing the CLIENT id resolves to the server id, which the server has. + assert(backend.sessionExists(client)) + + test( + "sessionExists returns false when the mapped server id is unknown to the server" + ): + supervised: + val http = new FakeHttp(turn("ses_server1", "stop", Nil), _ => 404) + val backend = new OpencodeBackend(_ => http) + val client = fresh + val _ = + backend.runAutonomous("hi", client, AgentConfig.default, os.temp.dir()) + // client → ses_server1 is mapped, but the server now 404s for it. + assert(!backend.sessionExists(client)) + + test( + "probeSession returns false when getStatus throws (verifies NonFatal catch)" + ): + supervised: + val http = new FakeHttp(Nil): + override def getStatus(path: String): Int = + throw new java.io.IOException("connection refused") + val backend = new OpencodeBackend(_ => http) + assert(!backend.probeSession("ses_abc", http)) + + test( + "sessionExists returns false for a malicious mapped server id (slashes)" + ): + supervised: + val http = new FakeHttp(Nil, _ => 200) // would return 200 if called + val backend = new OpencodeBackend(_ => http) + val client = fresh + // Even if the registry maps to a malicious server id, the guard blocks it. + backend.registerSession( + client, + SessionId[BackendTag.Opencode.type]("a/b") + ) + assert(!backend.sessionExists(client)) + + test( + "sessionExists returns false for a malicious mapped server id (query/fragment chars)" + ): + supervised: + val http = new FakeHttp(Nil, _ => 200) + val backend = new OpencodeBackend(_ => http) + val client = fresh + backend.registerSession( + client, + SessionId[BackendTag.Opencode.type]("x?y#z") + ) + assert(!backend.sessionExists(client)) diff --git a/opencode/src/test/scala/orca/tools/opencode/OpencodeConversationTest.scala b/opencode/src/test/scala/orca/tools/opencode/OpencodeConversationTest.scala index 493df2ba..0c97fdd6 100644 --- a/opencode/src/test/scala/orca/tools/opencode/OpencodeConversationTest.scala +++ b/opencode/src/test/scala/orca/tools/opencode/OpencodeConversationTest.scala @@ -2,9 +2,18 @@ package orca.tools.opencode import orca.AgentTurnFailed import orca.backend.{ApprovalDecision, ConversationEvent, StreamSource} +import ox.{Ox, supervised} class OpencodeConversationTest extends munit.FunSuite: + /** `OpencodeConversation` forks its reader into the caller's per-turn Ox, so + * construction needs a `using Ox`. Run each test body in a fresh supervised + * scope that provides it — keeping build + consume in one scope so the + * reader fork isn't cancelled before the events are drained. + */ + private def convTest(name: String)(body: Ox ?=> Any): Unit = + test(name)(supervised(body)) + /** Records reply POSTs; never serves the event stream (the source is injected * directly). */ @@ -32,7 +41,7 @@ class OpencodeConversationTest extends munit.FunSuite: lines: List[String], session: String = "ses_A", schema: Option[String] = None - ): (OpencodeConversation, RecordingHttp) = + )(using Ox): (OpencodeConversation, RecordingHttp) = val http = new RecordingHttp val conv = new OpencodeConversation( source(lines), @@ -43,7 +52,9 @@ class OpencodeConversationTest extends munit.FunSuite: ) (conv, http) - test("free-form turn: text deltas, then result from accrued text + tokens"): + convTest( + "free-form turn: text deltas, then result from accrued text + tokens" + ): val (conv, _) = conversation( List( data( @@ -76,7 +87,7 @@ class OpencodeConversationTest extends munit.FunSuite: assertEquals(result.usage.outputTokens, 2L) assertEquals(result.model.map(_.name), Some("gpt-4o-mini")) - test("structured turn: result is the validated object, not text"): + convTest("structured turn: result is the validated object, not text"): val (conv, _) = conversation( List( data( @@ -98,7 +109,7 @@ class OpencodeConversationTest extends munit.FunSuite: ) assertEquals(conv.awaitResult().toOption.get.output, """{"x":1}""") - test("a repeated tool part surfaces one AssistantToolCall"): + convTest("a repeated tool part surfaces one AssistantToolCall"): val running = data( """{"type":"message.part.updated","properties":{"part":{"type":"tool","tool":"bash","state":{"status":"running","input":{"command":"echo hi"}},"id":"prt_1","sessionID":"ses_A"}}}""" @@ -126,7 +137,7 @@ class OpencodeConversationTest extends munit.FunSuite: ) ) - test("events for other sessions are dropped"): + convTest("events for other sessions are dropped"): val (conv, _) = conversation( List( data( @@ -149,7 +160,7 @@ class OpencodeConversationTest extends munit.FunSuite: ) ) - test("blank, comment, and event: framing lines are skipped"): + convTest("blank, comment, and event: framing lines are skipped"): val (conv, _) = conversation( List( ":heartbeat", @@ -169,7 +180,7 @@ class OpencodeConversationTest extends munit.FunSuite: ) ) - test("free-form turn with no message.updated: text result, zero usage"): + convTest("free-form turn with no message.updated: text result, zero usage"): val (conv, _) = conversation( List( data( @@ -185,7 +196,7 @@ class OpencodeConversationTest extends munit.FunSuite: assertEquals(result.usage.outputTokens, 0L) assertEquals(result.model, None) - test("idle with no assistant message at all fails the turn"): + convTest("idle with no assistant message at all fails the turn"): val (conv, _) = conversation( List( data("""{"type":"session.idle","properties":{"sessionID":"ses_A"}}""") @@ -194,7 +205,7 @@ class OpencodeConversationTest extends munit.FunSuite: conv.events.foreach(_ => ()) intercept[AgentTurnFailed](conv.awaitResult()) - test("message.updated carrying info.error fails the turn"): + convTest("message.updated carrying info.error fails the turn"): val (conv, _) = conversation( List( data( @@ -206,7 +217,7 @@ class OpencodeConversationTest extends munit.FunSuite: conv.events.foreach(_ => ()) intercept[AgentTurnFailed](conv.awaitResult()) - test("session.error fails the turn"): + convTest("session.error fails the turn"): val (conv, _) = conversation( List( data( @@ -217,7 +228,7 @@ class OpencodeConversationTest extends munit.FunSuite: conv.events.foreach(_ => ()) intercept[AgentTurnFailed](conv.awaitResult()) - test("answering a question.asked POSTs the reply"): + convTest("answering a question.asked POSTs the reply"): val (conv, http) = conversation( List( data( @@ -238,7 +249,7 @@ class OpencodeConversationTest extends munit.FunSuite: private def permissionReplyPost( decision: ApprovalDecision - ): List[(String, String)] = + )(using Ox): List[(String, String)] = val (conv, http) = conversation( List( data( @@ -255,19 +266,19 @@ class OpencodeConversationTest extends munit.FunSuite: case _ => () http.posts - test("approving a permission.asked POSTs reply=once"): + convTest("approving a permission.asked POSTs reply=once"): assertEquals( permissionReplyPost(ApprovalDecision.Allow()), List("/permission/per_1/reply" -> """{"reply":"once"}""") ) - test("denying a permission.asked POSTs reply=reject"): + convTest("denying a permission.asked POSTs reply=reject"): assertEquals( permissionReplyPost(ApprovalDecision.Deny()), List("/permission/per_1/reply" -> """{"reply":"reject"}""") ) - test("canAskUser reflects the constructor flag"): + convTest("canAskUser reflects the constructor flag"): val http = new RecordingHttp val conv = new OpencodeConversation(empty, http, "ses_A", None, canAsk = false) diff --git a/opencode/src/test/scala/orca/tools/opencode/OpencodeIntegrationTest.scala b/opencode/src/test/scala/orca/tools/opencode/OpencodeIntegrationTest.scala index d555e291..a50030a4 100644 --- a/opencode/src/test/scala/orca/tools/opencode/OpencodeIntegrationTest.scala +++ b/opencode/src/test/scala/orca/tools/opencode/OpencodeIntegrationTest.scala @@ -1,7 +1,7 @@ package orca.tools.opencode import orca.backend.SupervisedBackend -import orca.llm.{BackendTag, LlmConfig, Model, SessionId, ToolSet} +import orca.agents.{BackendTag, AgentConfig, Model, SessionId, ToolSet} import orca.subprocess.OsProcCliRunner /** End-to-end tests against a real `opencode serve`. Gated on the @@ -28,9 +28,10 @@ class OpencodeIntegrationTest extends munit.FunSuite: private val model: Model = Model(sys.env.getOrElse("ORCA_OPENCODE_MODEL", "openai/gpt-4o-mini")) - private val config: LlmConfig = LlmConfig.default.copy(model = Some(model)) + private val config: AgentConfig = + AgentConfig.default.copy(model = Some(model)) - private def withBackend(body: OpencodeBackend => Unit): Unit = + private def withBackend(body: ox.Ox ?=> OpencodeBackend => Unit): Unit = SupervisedBackend.using(OpencodeBackend(OsProcCliRunner))(body) private def fresh = SessionId.fresh[BackendTag.Opencode.type] diff --git a/opencode/src/test/scala/orca/tools/opencode/OpencodeModelTest.scala b/opencode/src/test/scala/orca/tools/opencode/OpencodeModelTest.scala index 767b4968..7ead68f0 100644 --- a/opencode/src/test/scala/orca/tools/opencode/OpencodeModelTest.scala +++ b/opencode/src/test/scala/orca/tools/opencode/OpencodeModelTest.scala @@ -1,7 +1,7 @@ package orca.tools.opencode import orca.OrcaFlowException -import orca.llm.Model +import orca.agents.Model class OpencodeModelTest extends munit.FunSuite: diff --git a/opencode/src/test/scala/orca/tools/opencode/OpencodeServerTest.scala b/opencode/src/test/scala/orca/tools/opencode/OpencodeServerTest.scala index 97302fac..2ed9f52b 100644 --- a/opencode/src/test/scala/orca/tools/opencode/OpencodeServerTest.scala +++ b/opencode/src/test/scala/orca/tools/opencode/OpencodeServerTest.scala @@ -96,8 +96,19 @@ class OpencodeServerTest extends munit.FunSuite: val _ = server.http.postJson("/x", "{}") // force the spawn assertEquals( runner.lastArgs, - Seq("ollama", "launch", "opencode", "--model", "qwen3-coder", "--", - "serve", "--port", "0", "--log-level", "WARN") + Seq( + "ollama", + "launch", + "opencode", + "--model", + "qwen3-coder", + "--", + "serve", + "--port", + "0", + "--log-level", + "WARN" + ) ) test("a server that exits without binding surfaces its stderr"): @@ -118,3 +129,49 @@ class OpencodeServerTest extends munit.FunSuite: ex.getMessage.contains("model \"gemma4\" not found"), ex.getMessage ) + + test("shutdown destroys the process + closes the client; drains unblock"): + // The drain forks block on a non-interruptible read for the server's life; + // `shutdown`'s `destroyForcibly` is what EOFs them so the scope can join. + // This process leaves stdout/stderr OPEN after the bind line (unlike the + // others), so the drains are genuinely blocked until shutdown runs. + supervised: + val proc = new FakePipedCliProcess() + proc.enqueueStdout("opencode server listening on http://127.0.0.1:4096") + // deliberately NOT closing stdout/stderr — the drains stay blocked + class TrackingHttp extends OpencodeHttp: + @volatile var closed: Boolean = false + def postJson(path: String, body: String): String = "ok" + def events(): StreamSource = throw new UnsupportedOperationException + override def close(): Unit = closed = true + val client = new TrackingHttp + val server = + new OpencodeServer( + new RecordingRunner(proc), + os.temp.dir(), + httpFor = (_, _) => client + ) + + val _ = server.http // force start: spawns, reads bind line, forks drains + assert(proc.isAlive) + server.shutdown() + assert(!proc.isAlive, "shutdown must destroy the serve process") + assert(client.closed, "shutdown must close the http client") + server.shutdown() // idempotent: no exception, no double effect + // The scope then joins the drain forks. (The fake's queue read is + // interruptible, unlike a real native readLine, so this can't reproduce the + // production hang — the destroy/close assertions above are the real teeth; + // OpencodeServerTest's value is shutdown's effects + idempotency.) + + test("shutdown is a no-op when the server was never started"): + supervised: + val proc = new FakePipedCliProcess() + val runner = new RecordingRunner(proc) + val server = + new OpencodeServer( + runner, + os.temp.dir(), + httpFor = (_, _) => fail("unused") + ) + server.shutdown() // never forced `http` + assertEquals(runner.spawns.get(), 0) diff --git a/pi/src/main/scala/orca/tools/pi/DefaultPiAgent.scala b/pi/src/main/scala/orca/tools/pi/DefaultPiAgent.scala new file mode 100644 index 00000000..78546279 --- /dev/null +++ b/pi/src/main/scala/orca/tools/pi/DefaultPiAgent.scala @@ -0,0 +1,43 @@ +package orca.tools.pi + +import orca.backend.{Interaction, AgentBackend} +import orca.events.OrcaListener +import orca.agents.{BackendTag, AgentConfig, PiAgent, Prompts} +import orca.agents.BaseAgent + +/** Default [[PiAgent]] implementation. Inherits the autonomous-text and + * structured-output plumbing from [[BaseAgent]]; Pi model selection is left to + * generic [[AgentConfig.model]] values because Pi supports many providers and + * fuzzy model patterns through its own CLI. + */ +private[orca] class DefaultPiAgent( + backend: AgentBackend[BackendTag.Pi.type], + config: AgentConfig, + prompts: Prompts, + workDir: os.Path, + events: OrcaListener, + interaction: Interaction, + val name: String = "pi" +) extends BaseAgent[BackendTag.Pi.type, PiAgent]( + backend, + config, + prompts, + workDir, + events, + interaction + ) + with PiAgent: + + protected def copyTool( + config: AgentConfig = config, + name: String = name + ): PiAgent = + new DefaultPiAgent( + backend, + config, + prompts, + workDir, + events, + interaction, + name + ) diff --git a/pi/src/main/scala/orca/tools/pi/DefaultPiTool.scala b/pi/src/main/scala/orca/tools/pi/DefaultPiTool.scala deleted file mode 100644 index 9044eea1..00000000 --- a/pi/src/main/scala/orca/tools/pi/DefaultPiTool.scala +++ /dev/null @@ -1,43 +0,0 @@ -package orca.tools.pi - -import orca.backend.{Interaction, LlmBackend} -import orca.events.OrcaListener -import orca.llm.{BackendTag, LlmConfig, PiTool, Prompts} -import orca.llm.BaseLlmTool - -/** Default [[PiTool]] implementation. Inherits the autonomous-text and - * structured-output plumbing from [[BaseLlmTool]]; Pi model selection is left - * to generic [[LlmConfig.model]] values because Pi supports many providers and - * fuzzy model patterns through its own CLI. - */ -private[orca] class DefaultPiTool( - backend: LlmBackend[BackendTag.Pi.type], - config: LlmConfig, - prompts: Prompts, - workDir: os.Path, - events: OrcaListener, - interaction: Interaction, - val name: String = "pi" -) extends BaseLlmTool[BackendTag.Pi.type, PiTool]( - backend, - config, - prompts, - workDir, - events, - interaction - ) - with PiTool: - - protected def copyTool( - config: LlmConfig = config, - name: String = name - ): PiTool = - new DefaultPiTool( - backend, - config, - prompts, - workDir, - events, - interaction, - name - ) diff --git a/pi/src/main/scala/orca/tools/pi/PiArgs.scala b/pi/src/main/scala/orca/tools/pi/PiArgs.scala index 7948a75c..3fdbddc3 100644 --- a/pi/src/main/scala/orca/tools/pi/PiArgs.scala +++ b/pi/src/main/scala/orca/tools/pi/PiArgs.scala @@ -1,7 +1,7 @@ package orca.tools.pi import orca.backend.CliArgs -import orca.llm.{LlmConfig, ToolSet} +import orca.agents.{AgentConfig, ToolSet} /** Maps Orca backend configuration to Pi CLI arguments. The backend drives Pi * through RPC mode and sends prompts over stdin, so the argv carries only @@ -27,7 +27,7 @@ private[pi] object PiArgs: def rpc( sessionDir: os.Path, resume: Boolean, - config: LlmConfig, + config: AgentConfig, systemPromptFile: Option[os.Path], askUserExtension: Option[os.Path] = None ): Seq[String] = @@ -41,13 +41,13 @@ private[pi] object PiArgs: private def systemPromptArgs(file: Option[os.Path]): Seq[String] = file.toSeq.flatMap(f => Seq("--append-system-prompt", f.toString)) - /** Maps [[LlmConfig.tools]] to pi's `--tools` allowlist. `Full` omits the + /** Maps [[AgentConfig.tools]] to pi's `--tools` allowlist. `Full` omits the * flag (all built-in tools enabled); `ReadOnly` restricts to * [[ReadOnlyTools]]; `NetworkOnly` adds [[NetworkTool]] (`bash`) for network * access. The ask-user extension tool is appended when present. */ private def toolsArgs( - config: LlmConfig, + config: AgentConfig, includeAskUser: Boolean ): Seq[String] = config.tools match diff --git a/pi/src/main/scala/orca/tools/pi/PiBackend.scala b/pi/src/main/scala/orca/tools/pi/PiBackend.scala index a075d031..453b7879 100644 --- a/pi/src/main/scala/orca/tools/pi/PiBackend.scala +++ b/pi/src/main/scala/orca/tools/pi/PiBackend.scala @@ -1,22 +1,23 @@ package orca.tools.pi import orca.events.OrcaListener -import orca.llm.{BackendTag, LlmConfig, SessionId} -import orca.{AgentTurnFailed, OrcaFlowException} +import orca.agents.{BackendTag, AgentConfig, SessionId} import orca.backend.{ Conversation, Conversations, Dispatch, - LlmBackend, - LlmResult, + AgentBackend, + AgentResult, SessionMode, SessionRegistry, + SubprocessSpawn, SystemPromptComposer } import orca.subprocess.CliRunner +import ox.{Ox, supervised} + import scala.collection.mutable.ListBuffer -import scala.util.control.NonFatal /** Pi backend driven through `pi --mode rpc` JSONL over stdio. * @@ -25,15 +26,15 @@ import scala.util.control.NonFatal * bidirectional channel (needed within a turn for `ask_user` extension-UI * replies). * - * Lifecycle is deliberately per-call: each Orca call spawns its own - * `pi --mode rpc` process, sends one `prompt`, reads to `agent_end`, then lets - * the process exit. Context carries across calls through a per-session + * Lifecycle is deliberately per-call: each Orca call spawns its own `pi --mode + * rpc` process, sends one `prompt`, reads to `agent_end`, then lets the + * process exit. Context carries across calls through a per-session * `--session-dir` (one dir per Orca session id) that Pi creates on the first * turn and `--continue` resumes on later turns — rather than a long-lived * process. */ private[orca] class PiBackend(cli: CliRunner) - extends LlmBackend[BackendTag.Pi.type]: + extends AgentBackend[BackendTag.Pi.type]: // Pi persists each session in a directory; one dir per Orca session id gives // caller-stable continuity. The registry tracks fresh-vs-resume and is @@ -46,36 +47,37 @@ private[orca] class PiBackend(cli: CliRunner) def runAutonomous( prompt: String, session: SessionId[BackendTag.Pi.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener = OrcaListener.noop, outputSchema: Option[String] = None - ): LlmResult[BackendTag.Pi.type] = - val conv = openConversation( - prompt = prompt, - mode = SessionMode.Autonomous, - session = session, - config = config, - workDir = workDir, - outputSchema = outputSchema - ) - try - val result = Conversations.drainAutonomous(conv, events) - sessions.commitSuccess(session, session) // now resumable - result.copy(sessionId = session) - catch - case e: AgentTurnFailed => throw e - case e: OrcaFlowException => - throw new OrcaFlowException(s"pi CLI failed: ${e.getMessage}") + ): AgentResult[BackendTag.Pi.type] = + // Self-scoped: the conversation forks its workers into this per-call Ox, the + // drain consumes them, and `cancel` (the `finally`) tears the subprocess + + // forks (and the per-turn temp resources) down before the scope joins. + supervised: + val conv = openConversation( + prompt = prompt, + mode = SessionMode.Autonomous, + session = session, + config = config, + workDir = workDir, + outputSchema = outputSchema + ) + try + Conversations + .drainAndCommit("pi", conv, session, sessions, events) + .copy(sessionId = session) + finally conv.cancel() def runInteractive( prompt: String, session: SessionId[BackendTag.Pi.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Pi.type] = + )(using Ox): Conversation[BackendTag.Pi.type] = openConversation( prompt = prompt, mode = SessionMode.Interactive(displayPrompt), @@ -86,21 +88,26 @@ private[orca] class PiBackend(cli: CliRunner) ) /** Marks an interactive session resumable once its turn has succeeded — the - * framework calls this after driving the returned conversation to completion. + * framework calls this after driving the returned conversation to + * completion. */ override def registerSession( client: SessionId[BackendTag.Pi.type], serverSession: SessionId[BackendTag.Pi.type] ): Unit = sessions.commitSuccess(client, serverSession) + // No `resumeWireId` override: pi's sessions live in a `deleteOnExit` temp dir + // (gone across runs), so there is nothing durable to persist or rehydrate — pi + // always re-seeds (ADR 0018 §2.6). The default `None` keeps pi unaffected by + // the persist/rehydrate path. private def openConversation( prompt: String, mode: SessionMode, session: SessionId[BackendTag.Pi.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): PiConversation = + )(using Ox): PiConversation = // Temp files (ask-user extension, system prompt) Pi reads for the whole // turn. Ownership passes to the conversation once it's constructed — it // closes them in `onFinalize` when the turn ends; `closeResources` here is @@ -112,16 +119,16 @@ private[orca] class PiBackend(cli: CliRunner) resources += resource resource - def closeResources(): Unit = resources.reverseIterator.foreach(closeQuietly) - - try - val (displayPrompt, askUserExtension, extraHint) = mode match - case SessionMode.Autonomous => - ("", None, None) - case SessionMode.Interactive(p) => - val extension = register(PiAskUserExtension.allocate()) - (p, Some(extension), Some(PiAskUserExtension.Hint)) + val (displayPrompt, askUserExtension, extraHint) = mode match + case SessionMode.Autonomous => + ("", None, None) + case SessionMode.Interactive(p) => + val extension = register(PiAskUserExtension.allocate()) + (p, Some(extension), Some(PiAskUserExtension.Hint)) + // `resources` is accumulated above (and as the argv is built); SubprocessSpawn + // reads it (by-name) only on a spawn/build failure to release it. + SubprocessSpawn.open("pi RPC", resources.toList) { val systemPromptFile = writeSystemPromptIfPresent(config, extraHint) .map(register) @@ -135,32 +142,22 @@ private[orca] class PiBackend(cli: CliRunner) systemPromptFile = systemPromptFile.map(_.file), askUserExtension = askUserExtension.map(_.file) ) - val process = cli.spawnPiped(args, cwd = workDir, pipeStderr = true) - try - val conversation = new PiConversation( - process = process, - clientSession = session, - initialPrompt = displayPrompt, - outputSchema = outputSchema, - askUserEnabled = askUserExtension.isDefined, - resources = resources.toList - ) - conversation.sendPrompt(prompt) - conversation - catch - case e: Exception => - process.sendSigInt() - closeResources() - throw OrcaFlowException( - s"Failed to open pi RPC session: ${e.getMessage}" - ) - catch - case NonFatal(e) => - closeResources() - throw e + cli.spawnPiped(args, cwd = workDir, pipeStderr = true) + } { process => + val conversation = new PiConversation( + process = process, + clientSession = session, + initialPrompt = displayPrompt, + outputSchema = outputSchema, + askUserEnabled = askUserExtension.isDefined, + resources = resources.toList + ) + conversation.sendPrompt(prompt) + conversation + } private def writeSystemPromptIfPresent( - config: LlmConfig, + config: AgentConfig, extraHint: Option[String] ): Option[TempFileResource] = SystemPromptComposer diff --git a/pi/src/main/scala/orca/tools/pi/PiConversation.scala b/pi/src/main/scala/orca/tools/pi/PiConversation.scala index f8d761fd..4b478d84 100644 --- a/pi/src/main/scala/orca/tools/pi/PiConversation.scala +++ b/pi/src/main/scala/orca/tools/pi/PiConversation.scala @@ -1,10 +1,14 @@ package orca.tools.pi import orca.events.Usage -import orca.llm.{BackendTag, Model, SessionId} +import orca.agents.{BackendTag, Model, SessionId} import orca.{OrcaFlowException} -import orca.backend.{ConversationEvent, LlmResult} -import orca.backend.{BufferedStderrDiagnostics, StreamConversation, StreamSource} +import orca.backend.{ConversationEvent, AgentResult} +import orca.backend.{ + BufferedStderrDiagnostics, + ForkedConversation, + StreamSource +} import orca.subprocess.PipedCliProcess import orca.util.TerminalControl import orca.tools.pi.rpc.{ @@ -14,16 +18,18 @@ import orca.tools.pi.rpc.{ OutboundMessage } +import ox.Ox + import scala.util.control.NonFatal /** Drives one `pi --mode rpc` process for a single Orca LLM call. The backend * sends one `prompt` command, this conversation translates Pi RPC events into * Orca conversation events, and `agent_end` becomes the terminal - * [[LlmResult]]. + * [[AgentResult]]. * * Pi has no native structured-output / JSON-schema flag, so `outputSchema` is * carried only for the framework's parsing: the schema is enforced through the - * prompt (`DefaultLlmCall` injects the rules), not the Pi CLI. + * prompt (`DefaultAgentCall` injects the rules), not the Pi CLI. */ private[pi] class PiConversation( process: PipedCliProcess, @@ -32,7 +38,8 @@ private[pi] class PiConversation( val outputSchema: Option[String] = None, askUserEnabled: Boolean = false, resources: List[AutoCloseable] = Nil -) extends StreamConversation[BackendTag.Pi.type]( +)(using Ox) + extends ForkedConversation[BackendTag.Pi.type]( StreamSource.fromProcess(process), backendName = "pi", initialPrompt = initialPrompt, @@ -44,12 +51,12 @@ private[pi] class PiConversation( /** Turn state, accrued by the single reader thread — `handleLine` and the * handlers it drives all run there, so a plain `var` over an immutable - * snapshot is safe and avoids cross-thread machinery; `awaitResult` reads the - * outcome only after joining the reader, which publishes these writes. + * snapshot is safe and avoids cross-thread machinery; `awaitResult` reads + * the outcome only after joining the reader, which publishes these writes. * * `textStreamedThisMessage` lets `message_end` emit the completed text as a - * fallback only when no `text_delta` already streamed it; `sawAssistantMessage` - * gates the single `AssistantTurnEnd` at `agent_end`. + * fallback only when no `text_delta` already streamed it; + * `sawAssistantMessage` gates the single `AssistantTurnEnd` at `agent_end`. */ private case class TurnState( lastAssistantMessage: String = "", @@ -61,13 +68,14 @@ private[pi] class PiConversation( private var turnState: TurnState = TurnState() // All stdin writes funnel through this lock: `sendPrompt` runs on the caller's - // thread, the ask-user reply on the event consumer's, and the reader thread - // may write an extension cancel. `writeLine` is an unsynchronised write+flush, - // so concurrent callers would otherwise interleave JSONL frames. Declared - // before `start()` so the reader thread never observes a null lock. + // thread, the ask-user reply on the event consumer's, and the reader fork may + // write an extension cancel. `writeLine` is an unsynchronised write+flush, so + // concurrent callers would otherwise interleave JSONL frames. private val stdinLock = new AnyRef - start() + // No `start()`: the base spawns its reader / stderr forks lazily on first + // touch of the conversation surface, after this subclass's fields (incl. + // `stdinLock`) are initialised. def sendPrompt(prompt: String): Unit = sendLine(OutboundMessage.prompt(prompt)) @@ -78,12 +86,6 @@ private[pi] class PiConversation( private def closeStdin(): Unit = stdinLock.synchronized(process.closeStdin()) - /** Pi RPC prompts are command messages rather than a writable chat stdin. - * Orca's interactive Pi support currently routes human input through the - * ask_user extension UI bridge, so unsolicited user turns are a no-op. - */ - def sendUserMessage(text: String): Unit = () - override protected def handleLine(line: String): Unit = handle(InboundEvent.parse(line)) @@ -163,7 +165,7 @@ private[pi] class PiConversation( private def handleAgentEnd(): Unit = if turnState.sawAssistantMessage then eventQueue.enqueue(ConversationEvent.AssistantTurnEnd) - val result = LlmResult( + val result = AgentResult( sessionId = clientSession, output = turnState.lastAssistantMessage, usage = turnState.usage, diff --git a/pi/src/main/scala/orca/tools/pi/rpc/InboundEvent.scala b/pi/src/main/scala/orca/tools/pi/rpc/InboundEvent.scala index 095af7e3..86460fab 100644 --- a/pi/src/main/scala/orca/tools/pi/rpc/InboundEvent.scala +++ b/pi/src/main/scala/orca/tools/pi/rpc/InboundEvent.scala @@ -118,9 +118,9 @@ private[pi] object InboundEvent: .getOrElse(method) ExtensionUiRequest(wire.id, method, question) - /** Pi's `message.content` is polymorphic: either a JSON string, or an array of - * content blocks (of which we keep the `text` ones). Decode by shape; fall - * back to the raw trimmed value if it's neither, or fails to parse. + /** Pi's `message.content` is polymorphic: either a JSON string, or an array + * of content blocks (of which we keep the `text` ones). Decode by shape; + * fall back to the raw trimmed value if it's neither, or fails to parse. */ private def renderContent(raw: RawJson): String = val trimmed = raw.value.trim @@ -144,7 +144,9 @@ private[pi] object InboundEvent: * dropped. */ private def renderTextBlocks(blocks: List[ContentBlockWire]): String = - blocks.flatMap(b => Option.when(b.`type` == "text")(b.text).flatten).mkString + blocks + .flatMap(b => Option.when(b.`type` == "text")(b.text).flatten) + .mkString private def renderJsonString(raw: RawJson): Option[String] = val trimmed = raw.value.trim diff --git a/pi/src/test/scala/orca/tools/pi/PiArgsTest.scala b/pi/src/test/scala/orca/tools/pi/PiArgsTest.scala index 2a3880db..8b16239e 100644 --- a/pi/src/test/scala/orca/tools/pi/PiArgsTest.scala +++ b/pi/src/test/scala/orca/tools/pi/PiArgsTest.scala @@ -1,13 +1,13 @@ package orca.tools.pi -import orca.llm.{LlmConfig, Model, ToolSet} +import orca.agents.{AgentConfig, Model, ToolSet} class PiArgsTest extends munit.FunSuite: private val dir: os.Path = os.Path("/tmp/orca-pi-session") test("a fresh turn opens the session dir without --continue"): - val args = PiArgs.rpc(dir, resume = false, LlmConfig.default, None) + val args = PiArgs.rpc(dir, resume = false, AgentConfig.default, None) assertEquals( args.take(5), Seq("pi", "--mode", "rpc", "--session-dir", dir.toString) @@ -15,7 +15,7 @@ class PiArgsTest extends munit.FunSuite: assert(!args.contains("--continue"), args) test("a resumed turn adds --continue for the same session dir"): - val args = PiArgs.rpc(dir, resume = true, LlmConfig.default, None) + val args = PiArgs.rpc(dir, resume = true, AgentConfig.default, None) assert(args.containsSlice(Seq("--session-dir", dir.toString)), args) assert(args.contains("--continue"), args) @@ -23,7 +23,7 @@ class PiArgsTest extends munit.FunSuite: val args = PiArgs.rpc( dir, resume = false, - LlmConfig.default.copy(model = Some(Model("openai/gpt-5"))), + AgentConfig.default.copy(model = Some(Model("openai/gpt-5"))), Some(os.Path("/tmp/system.md")) ) assert(args.containsSlice(Seq("--model", "openai/gpt-5")), args) @@ -36,7 +36,7 @@ class PiArgsTest extends munit.FunSuite: val args = PiArgs.rpc( dir, resume = false, - LlmConfig.default.copy(tools = ToolSet.ReadOnly), + AgentConfig.default.copy(tools = ToolSet.ReadOnly), None ) assert(args.containsSlice(Seq("--tools", "read,grep,find,ls")), args) @@ -45,7 +45,7 @@ class PiArgsTest extends munit.FunSuite: val args = PiArgs.rpc( dir, resume = false, - LlmConfig.default.copy(tools = ToolSet.NetworkOnly), + AgentConfig.default.copy(tools = ToolSet.NetworkOnly), None ) assert(args.containsSlice(Seq("--tools", "read,grep,find,ls,bash")), args) @@ -54,7 +54,7 @@ class PiArgsTest extends munit.FunSuite: val args = PiArgs.rpc( dir, resume = false, - LlmConfig.default.copy(tools = ToolSet.ReadOnly), + AgentConfig.default.copy(tools = ToolSet.ReadOnly), None, askUserExtension = Some(os.Path("/tmp/ask-user.ts")) ) diff --git a/pi/src/test/scala/orca/tools/pi/PiBackendTest.scala b/pi/src/test/scala/orca/tools/pi/PiBackendTest.scala index b21b29ca..40f09280 100644 --- a/pi/src/test/scala/orca/tools/pi/PiBackendTest.scala +++ b/pi/src/test/scala/orca/tools/pi/PiBackendTest.scala @@ -2,7 +2,7 @@ package orca.tools.pi import orca.backend.SystemPromptComposer import orca.events.Usage -import orca.llm.{BackendTag, LlmConfig, Model, SessionId, ToolSet} +import orca.agents.{BackendTag, AgentConfig, Model, SessionId, ToolSet} import orca.subprocess.{FakePipedCliProcess, SpawnStubCliRunner} class PiBackendTest extends munit.FunSuite: @@ -33,7 +33,8 @@ class PiBackendTest extends munit.FunSuite: val backend = new PiBackend(runner) val workDir = os.temp.dir() - val result = backend.runAutonomous("do it", sid, LlmConfig.default, workDir) + val result = + backend.runAutonomous("do it", sid, AgentConfig.default, workDir) assertEquals(result.sessionId, sid) assertEquals(result.output, "answer") @@ -57,8 +58,8 @@ class PiBackendTest extends munit.FunSuite: val backend = new PiBackend(runner) val workDir = os.temp.dir() - val _ = backend.runAutonomous("one", sid, LlmConfig.default, workDir) - val _ = backend.runAutonomous("two", sid, LlmConfig.default, workDir) + val _ = backend.runAutonomous("one", sid, AgentConfig.default, workDir) + val _ = backend.runAutonomous("two", sid, AgentConfig.default, workDir) val Seq(first, second) = runner.spawnCalls.take(2): @unchecked assert(!first.args.contains("--continue"), first.args) @@ -76,9 +77,9 @@ class PiBackendTest extends munit.FunSuite: val workDir = os.temp.dir() val _ = intercept[Exception]( - backend.runAutonomous("one", sid, LlmConfig.default, workDir) + backend.runAutonomous("one", sid, AgentConfig.default, workDir) ) - val _ = backend.runAutonomous("two", sid, LlmConfig.default, workDir) + val _ = backend.runAutonomous("two", sid, AgentConfig.default, workDir) val Seq(first, second) = runner.spawnCalls.take(2): @unchecked assert(!first.args.contains("--continue"), first.args) @@ -93,7 +94,7 @@ class PiBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "q", sid, - LlmConfig.default.copy( + AgentConfig.default.copy( model = Some(Model("anthropic/claude-sonnet")), tools = ToolSet.ReadOnly ), @@ -110,26 +111,29 @@ class PiBackendTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(process)) val backend = new PiBackend(runner) - val conv = backend.runInteractive( - "q", - sid, - displayPrompt = "q", - LlmConfig.default.copy(tools = ToolSet.ReadOnly), - os.temp.dir(), - outputSchema = Some("{}") - ) - assert(conv.canAskUser) - assertEquals(conv.outputSchema, Some("{}")) - - val args = runner.calls.head - assert( - args.containsSlice(Seq("--tools", "read,grep,find,ls,ask_user")), - args - ) - assert(args.contains("--extension"), args) - - val _ = conv.events.toList - val _ = conv.awaitResult() + // The conversation forks its workers into the surrounding Ox scope, so it + // must be created AND consumed within the same `supervised` block. + ox.supervised: + val conv = backend.runInteractive( + "q", + sid, + displayPrompt = "q", + AgentConfig.default.copy(tools = ToolSet.ReadOnly), + os.temp.dir(), + outputSchema = Some("{}") + ) + assert(conv.canAskUser) + assertEquals(conv.outputSchema, Some("{}")) + + val args = runner.calls.head + assert( + args.containsSlice(Seq("--tools", "read,grep,find,ls,ask_user")), + args + ) + assert(args.contains("--extension"), args) + + val _ = conv.events.toList + val _ = conv.awaitResult() test( "interactive system prompt file contains configured prompt, hint, and git rule" @@ -138,33 +142,39 @@ class PiBackendTest extends munit.FunSuite: val runner = new SpawnStubCliRunner(List(process)) val backend = new PiBackend(runner) - val conv = backend.runInteractive( - "q", - sid, - displayPrompt = "q", - LlmConfig.default.copy(systemPrompt = Some("be terse")), - os.temp.dir(), - outputSchema = None - ) - - val args = runner.calls.head - val promptFile = args(args.indexOf("--append-system-prompt") + 1) - val promptText = os.read(os.Path(promptFile)) - assert(promptText.contains("be terse"), promptText) - assert(promptText.contains(PiAskUserExtension.Hint), promptText) - assert(promptText.contains(SystemPromptComposer.RuntimeOwnsGit), promptText) - - val extensionFile = os.Path(args(args.indexOf("--extension") + 1)) - assert(os.exists(extensionFile)) - - process.enqueueStdout( - """{"type":"message_end","message":{"role":"assistant","content":[{"type":"text","text":"done"}]}}""" - ) - process.enqueueStdout("""{"type":"agent_end","messages":[]}""") - val _ = conv.events.toList - val _ = conv.awaitResult() - assert(!os.exists(os.Path(promptFile))) - assert(!os.exists(extensionFile)) + // The conversation forks its workers into the surrounding Ox scope, so it + // must be created AND consumed within the same `supervised` block. + ox.supervised: + val conv = backend.runInteractive( + "q", + sid, + displayPrompt = "q", + AgentConfig.default.copy(systemPrompt = Some("be terse")), + os.temp.dir(), + outputSchema = None + ) + + val args = runner.calls.head + val promptFile = args(args.indexOf("--append-system-prompt") + 1) + val promptText = os.read(os.Path(promptFile)) + assert(promptText.contains("be terse"), promptText) + assert(promptText.contains(PiAskUserExtension.Hint), promptText) + assert( + promptText.contains(SystemPromptComposer.RuntimeOwnsGit), + promptText + ) + + val extensionFile = os.Path(args(args.indexOf("--extension") + 1)) + assert(os.exists(extensionFile)) + + process.enqueueStdout( + """{"type":"message_end","message":{"role":"assistant","content":[{"type":"text","text":"done"}]}}""" + ) + process.enqueueStdout("""{"type":"agent_end","messages":[]}""") + val _ = conv.events.toList + val _ = conv.awaitResult() + assert(!os.exists(os.Path(promptFile))) + assert(!os.exists(extensionFile)) test("self-managed git suppresses the runtime git rule"): val process = successfulProcess() @@ -174,9 +184,13 @@ class PiBackendTest extends munit.FunSuite: val _ = backend.runAutonomous( "q", sid, - LlmConfig.default.copy(selfManagedGit = true), + AgentConfig.default.copy(selfManagedGit = true), os.temp.dir() ) val args = runner.calls.head assert(!args.contains("--append-system-prompt"), args) + + test("sessionExists always returns false (Pi has no server-side probe)"): + val backend = new PiBackend(new SpawnStubCliRunner(Nil)) + assert(!backend.sessionExists(sid)) diff --git a/pi/src/test/scala/orca/tools/pi/PiConversationTest.scala b/pi/src/test/scala/orca/tools/pi/PiConversationTest.scala index cb3a0af5..7bab2402 100644 --- a/pi/src/test/scala/orca/tools/pi/PiConversationTest.scala +++ b/pi/src/test/scala/orca/tools/pi/PiConversationTest.scala @@ -2,16 +2,26 @@ package orca.tools.pi import orca.backend.ConversationEvent import orca.events.Usage -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} import orca.{OrcaFlowException, OrcaInteractiveCancelled} import orca.subprocess.FakePipedCliProcess +import ox.{Ox, supervised} class PiConversationTest extends munit.FunSuite: private val sid: SessionId[BackendTag.Pi.type] = SessionId[BackendTag.Pi.type]("pi-session") - test("text deltas complete with AssistantTurnEnd and produce LlmResult"): + /** `PiConversation` forks its reader/stderr workers into the caller's + * per-turn Ox, so construction needs a `using Ox`. Run each test body in a + * fresh supervised scope that provides it. + */ + private def convTest(name: String)(body: Ox ?=> Unit): Unit = + test(name)(supervised(body)) + + convTest( + "text deltas complete with AssistantTurnEnd and produce AgentResult" + ): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -38,7 +48,7 @@ class PiConversationTest extends munit.FunSuite: assertEquals(process.sigIntCount, 1) assert(process.isStdinClosed) - test("message_end emits assistant text when no text delta streamed"): + convTest("message_end emits assistant text when no text delta streamed"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -57,7 +67,7 @@ class PiConversationTest extends munit.FunSuite: val Right(result) = conv.awaitResult(): @unchecked assertEquals(result.output, "fallback") - test("thinking delta becomes AssistantThinkingDelta"): + convTest("thinking delta becomes AssistantThinkingDelta"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -76,7 +86,7 @@ class PiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("tool execution events become tool call and tool result"): + convTest("tool execution events become tool call and tool result"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -102,7 +112,7 @@ class PiConversationTest extends munit.FunSuite: case other => fail(s"expected ToolResult, got $other") val _ = conv.awaitResult() - test("unknown events are ignored"): + convTest("unknown events are ignored"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -117,7 +127,7 @@ class PiConversationTest extends munit.FunSuite: val Right(result) = conv.awaitResult(): @unchecked assertEquals(result.output, "ok") - test("usage accumulates across assistant messages"): + convTest("usage accumulates across assistant messages"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -136,7 +146,7 @@ class PiConversationTest extends munit.FunSuite: assertEquals(result.output, "second") assertEquals(result.usage, Usage(5L, 7L, None, 9L)) - test("failed prompt response fails the conversation"): + convTest("failed prompt response fails the conversation"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -153,7 +163,9 @@ class PiConversationTest extends munit.FunSuite: val ex = intercept[OrcaFlowException](conv.awaitResult()) assert(ex.getMessage.contains("model unavailable")) - test("extension UI input request becomes UserQuestion and writes response"): + convTest( + "extension UI input request becomes UserQuestion and writes response" + ): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid, askUserEnabled = true) assert(conv.canAskUser) @@ -176,7 +188,7 @@ class PiConversationTest extends munit.FunSuite: case other => fail(s"expected cancellation after test cleanup, got $other") - test("fire-and-forget extension UI requests are ignored"): + convTest("fire-and-forget extension UI requests are ignored"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -193,7 +205,9 @@ class PiConversationTest extends munit.FunSuite: assert(!process.writes.exists(_.contains("extension_ui_response"))) val _ = conv.awaitResult() - test("an extension_ui_request without a method is cancelled, not dropped"): + convTest( + "an extension_ui_request without a method is cancelled, not dropped" + ): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -204,10 +218,15 @@ class PiConversationTest extends munit.FunSuite: val _ = conv.events.toList // A cancel is written so Pi doesn't block waiting on a reply. - assert(process.writes.exists(_.contains("extension_ui_response")), process.writes) + assert( + process.writes.exists(_.contains("extension_ui_response")), + process.writes + ) val _ = conv.awaitResult() - test("message_end without content surfaces the error, not a parse failure"): + convTest( + "message_end without content surfaces the error, not a parse failure" + ): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -233,7 +252,7 @@ class PiConversationTest extends munit.FunSuite: ) val _ = conv.awaitResult() - test("clean exit before agent_end fails"): + convTest("clean exit before agent_end fails"): val process = new FakePipedCliProcess(initiallyAlive = false) val conv = new PiConversation(process, sid) process.closeStdout() @@ -243,7 +262,7 @@ class PiConversationTest extends munit.FunSuite: val ex = intercept[OrcaFlowException](conv.awaitResult()) assert(ex.getMessage.contains("agent_end")) - test("stderr diagnostics are attached to failures"): + convTest("stderr diagnostics are attached to failures"): val process = new FakePipedCliProcess(initiallyAlive = false): override def tryExitCode: Option[Int] = Some(7) val conv = new PiConversation(process, sid) @@ -255,7 +274,7 @@ class PiConversationTest extends munit.FunSuite: val ex = intercept[OrcaFlowException](conv.awaitResult()) assert(ex.getMessage.contains("Pi auth failed"), ex.getMessage) - test("terminal notification stderr noise is ignored"): + convTest("terminal notification stderr noise is ignored"): val process = new FakePipedCliProcess() val conv = new PiConversation(process, sid) @@ -269,7 +288,7 @@ class PiConversationTest extends munit.FunSuite: assertEquals(events, Nil) val _ = conv.awaitResult() - test("stderr strips terminal controls before surfacing diagnostics"): + convTest("stderr strips terminal controls before surfacing diagnostics"): val process = new FakePipedCliProcess(initiallyAlive = false): override def tryExitCode: Option[Int] = Some(7) val conv = new PiConversation(process, sid) diff --git a/pi/src/test/scala/orca/tools/pi/PiIntegrationTest.scala b/pi/src/test/scala/orca/tools/pi/PiIntegrationTest.scala index 350cb854..7d2bcbe0 100644 --- a/pi/src/test/scala/orca/tools/pi/PiIntegrationTest.scala +++ b/pi/src/test/scala/orca/tools/pi/PiIntegrationTest.scala @@ -1,6 +1,6 @@ package orca.tools.pi -import orca.llm.{BackendTag, LlmConfig, SessionId, ToolSet} +import orca.agents.{BackendTag, AgentConfig, SessionId, ToolSet} import orca.subprocess.OsProcCliRunner /** End-to-end smoke test against the real `pi` CLI. Gated on `ORCA_INTEGRATION` @@ -24,7 +24,7 @@ class PiIntegrationTest extends munit.FunSuite: val result = backend.runAutonomous( prompt = "Reply with the single word: READY", session = fresh, - config = LlmConfig.default.copy(tools = ToolSet.ReadOnly), + config = AgentConfig.default.copy(tools = ToolSet.ReadOnly), workDir = os.temp.dir() ) assert( diff --git a/plan.md b/plan.md index 3d8387d8..394f1de3 100644 --- a/plan.md +++ b/plan.md @@ -48,14 +48,14 @@ All traits, types, and signatures — compilable, no implementations yet. This i | # | Task | Description | Status | |---|---|---|---| -| 2.1 | Base types | `Backend` enum, `SessionId` opaque type, `LlmConfig`, `AutoApprove`, `UnapprovedPolicy`, `Usage`. Verify `derives` works for enums. | ✅ | +| 2.1 | Base types | `Backend` enum, `SessionId` opaque type, `AgentConfig`, `AutoApprove`, `UnapprovedPolicy`, `Usage`. Verify `derives` works for enums. | ✅ | | 2.2 | Tool traits | `GitTool`, `GitHubTool`, `FsTool` traits. Supporting types: `PrHandle`, `BuildStatus`, `Comment`, `CommitInfo`. | ✅ | -| 2.3 | LLM traits | `LlmTool[B]`, `ClaudeTool`, `CodexTool`, `LlmCall[B, O]`. Verify the type parameter + opaque type + method chaining compiles. | ✅ | +| 2.3 | LLM traits | `Agent[B]`, `ClaudeAgent`, `CodexAgent`, `AgentCall[B, O]`. Verify the type parameter + opaque type + method chaining compiles. | ✅ | | 2.4 | AgentInput typeclass | `AgentInput` trait, `given` instances for `String` and `ConfiguredJsonValueCodec`. Unit test: serialization of String and a case class. | ✅ | | 2.5 | Structured types | `ReviewIssue`, `Severity`, `ReviewResult`, `IgnoredIssue`, `IgnoredIssues`, `ReviewContext`, `SelectedReviewers`. Verify `derives Schema, ConfiguredJsonValueCodec` compiles. Unit test: serialize/deserialize round-trip. | ✅ | | 2.6 | Event types | `OrcaEvent` enum, `OrcaListener` trait, `Interaction` trait. | ✅ | | 2.7 | FlowContext & helpers | `FlowContext` trait, `OrcaFlowException`. Signatures for `stage`, `fail`, `fixLoop`, `reviewAndFix`, `lint`. `PromptTemplate` trait. | ✅ | -| 2.8 | Backend abstraction | `LlmBackend[B]` trait, `LlmResult[B]`, `InteractiveHandle[B]`. | ✅ | +| 2.8 | Backend abstraction | `AgentBackend[B]` trait, `AgentResult[B]`, `InteractiveHandle[B]`. | ✅ | **Exit criteria**: `sbt compile` green across all modules. All types and traits exist with correct signatures. Unit tests for AgentInput and structured type round-trips pass. @@ -84,12 +84,12 @@ First real backend. Subprocess-based, testable with a `CliRunner` abstraction. | # | Task | Description | Status | |---|---|---|---| | 4.1 | CliRunner abstraction | Trait with `run(args, stdin, env, cwd)` returning `(exitCode, stdout, stderr)`. Real implementation via `os.proc`. Test stub returning canned responses. | ✅ | -| 4.2 | Headless invocation | Implement `runHeadless`: construct `claude -p` command with flags (`--output-format json`, `--append-system-prompt-file`, `--allowedTools`, `--permission-mode`). Parse JSON response into `LlmResult`. Unit test with stubbed CliRunner. | ✅ | +| 4.2 | Headless invocation | Implement `runHeadless`: construct `claude -p` command with flags (`--output-format json`, `--append-system-prompt-file`, `--allowedTools`, `--permission-mode`). Parse JSON response into `AgentResult`. Unit test with stubbed CliRunner. | ✅ | | 4.3 | Session management | Implement `continueHeadless` with `--resume <id>`. Parse `session_id` from response. Unit test: verify `--resume` flag is passed, session ID extracted. | ✅ | | 4.4 | NDJSON streaming | Parse `stream-json` output line by line. Emit `OrcaEvent.LlmOutput` for each text event. Unit test with canned NDJSON. | ✅ | | 4.5 | Stop hook generation | `prepareWorkspace`: write `.claude/settings.json` with the Stop hook that detects `<<<ORCA_DONE>>>` and writes the sentinel file. Unit test: verify generated JSON is valid, hook script is correct. | ✅ | | 4.6 | Interactive mode | `launchInteractive`: spawn `claude` (no `-p`) with `--session-id` and `--append-system-prompt-file`. Terminal handoff via `os.Inherit`. `awaitTermination`: watch for sentinel file, then SIGINT. Unit test with stubbed CliRunner. | ✅ | -| 4.7 | Config mapping | Map `LlmConfig` fields to Claude CLI flags: `autoApprove` → `--permission-mode`, model → `--model`, etc. Unit test: config → args mapping. | ✅ | +| 4.7 | Config mapping | Map `AgentConfig` fields to Claude CLI flags: `autoApprove` → `--permission-mode`, model → `--model`, etc. Unit test: config → args mapping. | ✅ | | 4.8 | Integration tests | Real `claude -p` calls (gated). Test: headless prompt returns valid JSON, session resume works, streaming emits events. | ✅ | **Exit criteria**: Claude Code backend passes unit tests with stubbed CLI. Integration tests pass against real `claude` (when available). @@ -164,7 +164,7 @@ Higher-level flow combinators, built on all previous epics. Second backend. Mirrors the Claude architecture established in Epics 4 and 11 — `PipedCliProcess` subprocess, daemon reader thread producing -`ConversationEvent`s, `DefaultLlmCall` driving retries unchanged. The +`ConversationEvent`s, `DefaultAgentCall` driving retries unchanged. The plan's original WebSocket/app-server path may no longer be necessary: if `codex exec --json` streams per-turn events and tool calls, we get interactive support over the same stdio shape Claude uses, for free. @@ -174,8 +174,8 @@ Task 9.1 decides. |---|---|---|---| | 9.1 | Capability probe | Drive `codex exec --json --full-auto` with a tool-using, multi-turn prompt and inspect the JSONL stream. Does it emit partial text deltas, tool calls, and a mid-turn tool-approval channel — or only a batched final result? Answer determines whether 9.2 follows the Claude stdio pattern or falls back to app-server + WebSocket. Document findings in a short ADR. See [ADR 0007](adr/0007-codex-exec-jsonl-driver.md). | ✅ | | 9.2 | Conversation driver | `CodexConversation` built on the same skeleton as `ClaudeConversation`: `PipedCliProcess`, daemon reader thread, `LinkedBlockingQueue[Option[ConversationEvent]]`, stderr drain thread, `outcomeRef` + `cancelled` gates, SIGINT on `cancel()`, `finalizeLoop` resolving the terminal outcome. Translate Codex's JSONL events (`thread.started`, `item.completed`, `turn.completed`, tool events) into `ConversationEvent.*`. Unit tests via `FakePipedCliProcess`. Only add a WebSocket + app-server path if 9.1 rules stdio out. | ✅ | -| 9.3 | Headless + interactive surface | `CodexBackend.runHeadless` / `continueHeadless` parse the final result out of the same JSONL stream (or use `--resume` for continuation). `runInteractive` / `continueInteractive` return `Conversation[Backend.Codex.type]` and accept both `prompt` (wire) and `displayPrompt` (renderer-facing), enqueuing a `ConversationEvent.UserMessage(displayPrompt)` at startup — contract established for Claude and already wired into `LlmBackend`. Tool approvals route through `ConversationEvent.ApproveTool(name, input, respond)` when the CLI exposes mid-turn approvals; otherwise `LlmConfig.autoApprove` is pre-baked into the spawn args. | ✅ | -| 9.4 | Config mapping | `LlmConfig.autoApprove` / `model` / `systemPrompt` → Codex CLI flags. Reuse `LlmConfig.defaultRetrySchedule` and `DefaultLlmCall`'s retry-with-corrective-prompt loop — no backend-specific retry logic. | ✅ | +| 9.3 | Headless + interactive surface | `CodexBackend.runHeadless` / `continueHeadless` parse the final result out of the same JSONL stream (or use `--resume` for continuation). `runInteractive` / `continueInteractive` return `Conversation[Backend.Codex.type]` and accept both `prompt` (wire) and `displayPrompt` (renderer-facing), enqueuing a `ConversationEvent.UserMessage(displayPrompt)` at startup — contract established for Claude and already wired into `AgentBackend`. Tool approvals route through `ConversationEvent.ApproveTool(name, input, respond)` when the CLI exposes mid-turn approvals; otherwise `AgentConfig.autoApprove` is pre-baked into the spawn args. | ✅ | +| 9.4 | Config mapping | `AgentConfig.autoApprove` / `model` / `systemPrompt` → Codex CLI flags. Reuse `AgentConfig.defaultRetrySchedule` and `DefaultAgentCall`'s retry-with-corrective-prompt loop — no backend-specific retry logic. | ✅ | | 9.5 | Prompt template | Verify `DefaultPromptTemplate` applies as-is. It now tells the agent to "deliver the final answer as a JSON-only message" and relies on `ResponseParser.parse` (fence-stripping + right-to-left balanced-`{...}` extraction + `MalformedAgentOutputException`). If Codex enforces a different structured-output convention, write a Codex-specific `PromptTemplate` rather than bending the shared one. | ✅ | | 9.6 | Integration tests | Real `codex` calls gated on `ORCA_INTEGRATION`. Headless round-trip, `continueHeadless` resume, streaming deltas + `AssistantTurnEnd`, tool-approval denial (if supported). Mirror `ClaudeIntegrationTest`. | ✅ | @@ -200,10 +200,10 @@ Task 9.1 decides. - **Cancellation is SIGINT.** The reader thread sees EOF, `finalizeLoop` compare-and-sets `outcomeRef` to `Outcome.Cancelled`, the event queue closes. Don't force-close the queue from another thread. -- **`LlmConfig` val order.** `default` reads `defaultRetrySchedule`; Scala +- **`AgentConfig` val order.** `default` reads `defaultRetrySchedule`; Scala evaluates object vals in source order, so the schedule must appear first or the case-class default latches on null and downstream `retry` - calls NPE. Already fixed in the shared `LlmConfig` but worth knowing + calls NPE. Already fixed in the shared `AgentConfig` but worth knowing when touching that file. **UX parity checklist** (so swapping `claude` for `codex` in a flow @@ -225,7 +225,7 @@ requires zero script changes): **Exit criteria**: Codex backend passes unit tests with stubbed CLI. Integration tests pass against a real `codex`. Swapping `claude` for -`codex` in `DefaultLlmCall.autonomous` / `continueSession` / `interactive` +`codex` in `DefaultAgentCall.autonomous` / `continueSession` / `interactive` paths requires no flow-script changes. --- @@ -285,7 +285,7 @@ Follow-up to Epics 3 & 4. Replaces the `<<<ORCA_DONE>>>` marker + stop-hook + se | 11.2 | Bidirectional CliProcess | `PipedCliProcess` with `writeLine`, `stdoutLines` / `stderrLines` iterators, `tryExitCode`, `closeStdin`. `FakePipedCliProcess` for tests. | ✅ | | 11.3 | Conversation contract | `Conversation[B]` (events + awaitResult + sendUserMessage + cancel), `ConversationEvent` enum, `ApprovalDecision`, `OrcaInteractiveCancelled`. `Interaction.drive(conversation)` replaces `runInteractive(handle)`. | ✅ | | 11.4 | ClaudeConversation driver | Reader thread, event queue, autoapprove gating, per-message translation, cancel via SIGINT. | ✅ | -| 11.5 | Backend rewrite | `ClaudeBackend.runInteractive` / `continueInteractive` return `Conversation[B]`. `ClaudeArgs.streamJson` replaces the old `interactive`. `DefaultLlmCall.interactive` delegates to `interaction.drive`. | ✅ | +| 11.5 | Backend rewrite | `ClaudeBackend.runInteractive` / `continueInteractive` return `Conversation[B]`. `ClaudeArgs.streamJson` replaces the old `interactive`. `DefaultAgentCall.interactive` delegates to `interaction.drive`. | ✅ | | 11.6 | Prompt template | Drop `<<<ORCA_DONE>>>` from `DefaultPromptTemplate.interactive`; thread `--json-schema` via `outputSchema: Option[String]`. | ✅ | | 11.7 | Stop-hook cleanup | Delete `ClaudeStopHook`, `ClaudeInteractiveHandle`, `DoneMarkerExtractor`, `InteractiveHandle` trait, the inherited-stdio `spawn` path, `prepareWorkspace` on the backend trait. | ✅ | | 11.8 | TerminalConversationRenderer | Per-event rendering (streaming text, tool calls, tool results, errors), spinner coordination, approval prompts via a `Prompter` seam. | ✅ | @@ -302,14 +302,14 @@ Items deferred during the Epic 11 reviews. Independent, order-agnostic, each ~ha | # | Item | Description | Status | |---|---|---|---| -| 12.1 | `ConversationEvent.Usage` | Emit an event as `result.usage` arrives (and, if upstream ever adds it, mid-session usage deltas) so Slack/HTTP channels can show live cost/token readouts. `DefaultLlmCall` keeps translating to `OrcaEvent.TokensUsed` for `CostTracker`. | | +| 12.1 | `ConversationEvent.Usage` | Emit an event as `result.usage` arrives (and, if upstream ever adds it, mid-session usage deltas) so Slack/HTTP channels can show live cost/token readouts. `DefaultAgentCall` keeps translating to `OrcaEvent.TokensUsed` for `CostTracker`. | | | 12.2 | Ctrl-C-during-streaming → graceful cancel | Today Ctrl-C kills the JVM when no readline prompt is active; only approval prompts cancel gracefully (via `UserInterruptException`). Install a `Signal("INT")` handler around `TerminalConversationRenderer.render` that calls `conversation.cancel()` instead of exiting, then removes itself on return. | | | 12.3 | Real-subprocess cancel integration test | `ClaudeConversation.cancel()` is unit-tested against `FakePipedCliProcess` only. Add a gated integration case that spawns real `claude` with a long-running prompt, calls `cancel`, asserts `awaitResult` throws `OrcaInteractiveCancelled` within a timeout, and the subprocess exits. | | -| 12.4 | Interactive-path schema-threading test | `DefaultLlmCall.interactive` generates `JsonSchemaGen[O]` and forwards it as `Some(schema)` to `backend.runInteractive`. No unit test pins this; add one that captures `outputSchema` on a stub `LlmBackend.runInteractive` and asserts the schema is present + shaped for the case-class `O`. | | +| 12.4 | Interactive-path schema-threading test | `DefaultAgentCall.interactive` generates `JsonSchemaGen[O]` and forwards it as `Some(schema)` to `backend.runInteractive`. No unit test pins this; add one that captures `outputSchema` on a stub `AgentBackend.runInteractive` and asserts the schema is present + shaped for the case-class `O`. | | | 12.5 | `writeOutbound` I/O failures fail the session | `ClaudeConversation.handleControlRequest`'s stdin writes can throw `IOException` if the child died; today that surfaces as a `ConversationEvent.Error` and the reader keeps polling. Let the exception propagate so the reader's `NonFatal` catch records `Outcome.Failed` and `awaitResult` rethrows. | | | 12.6 | `ClaudeConversation` safe-publication factory | The reader thread starts in the constructor — safe in practice given final-field + `Thread.start` happens-before, not structurally enforced. Split into a private constructor + `object ClaudeConversation.open(process, config)` that constructs then starts the thread. | | | 12.7 | Reader-thread join timeout on `awaitResult` | If the child ignores SIGINT (hung GC, attached debugger), the reader leaks forever. `readerThread.join(timeout)` with a sensible default (30s?) + a `Failed(OrcaFlowException("reader did not terminate"))` path. | | -| 12.8 | ACP client adapter | N-backend portability play. Once Codex work (Epic 9) lands, wrap `LlmBackend` dispatch behind an Agent Client Protocol JSON-RPC client: one Scala client covers Claude/Codex/Gemini/Copilot CLIs (native or via sidecar adapters). See the tradeoffs discussion in ADR 0006's "Alternatives" section. Bigger task — budget 3–5 days. | | +| 12.8 | ACP client adapter | N-backend portability play. Once Codex work (Epic 9) lands, wrap `AgentBackend` dispatch behind an Agent Client Protocol JSON-RPC client: one Scala client covers Claude/Codex/Gemini/Copilot CLIs (native or via sidecar adapters). See the tradeoffs discussion in ADR 0006's "Alternatives" section. Bigger task — budget 3–5 days. | | | 12.9 | `ConversationEvent.Error` at WARN, not ERROR | Unknown `InboundMessage` types currently surface as `Error` events; semantically they're "protocol drift, not actionable". Add a `Warning` variant (or a severity field) so the channel can render them at a lower level. | | **Exit criteria**: none as a gate — each item stands alone. 12.2 and 12.8 are the highest-user-facing impact; 12.5 and 12.7 are the highest-reliability impact. diff --git a/runner/src/main/scala/orca/exports.scala b/runner/src/main/scala/orca/exports.scala index 80cb60c9..8f4d6855 100644 --- a/runner/src/main/scala/orca/exports.scala +++ b/runner/src/main/scala/orca/exports.scala @@ -12,18 +12,18 @@ package orca // re-export needed. export orca.events.{OrcaEvent, OrcaListener, Pricing, PriceList, ModelPricing} -export orca.llm.{ - LlmTool, - ClaudeTool, - CodexTool, - OpencodeTool, - PiTool, - GeminiTool, - LlmCall, +export orca.agents.{ + Agent, + ClaudeAgent, + CodexAgent, + OpencodeAgent, + PiAgent, + GeminiAgent, + AgentCall, AutonomousTextCall, - AutonomousLlmCall, - InteractiveLlmCall, - LlmConfig, + AutonomousAgentCall, + InteractiveAgentCall, + AgentConfig, AutoApprove, ToolSet, BackendTag, @@ -35,17 +35,7 @@ export orca.llm.{ schemaFromJsonData, codecFromJsonData } -export orca.plan.{ - BugReportMatch, - Plan, - PlanLike, - PlanWithBrief, - Sessioned, - Task, - Title, - Triage, - Verdict -} +export orca.plan.{BugReportMatch, Plan, Sessioned, Task, Title, Triage, Verdict} export orca.pr.{summarisePr, PrSummary} export orca.review.{ allReviewers, diff --git a/runner/src/main/scala/orca/flow.scala b/runner/src/main/scala/orca/flow.scala index 1f30f7e9..0d8e8d9a 100644 --- a/runner/src/main/scala/orca/flow.scala +++ b/runner/src/main/scala/orca/flow.scala @@ -9,14 +9,32 @@ import orca.events.{ PriceList, Pricing } -import orca.llm.{ClaudeTool, DefaultPrompts, OpencodeTool, PiTool, Prompts} +import orca.agents.{ + Agent, + BackendTag, + ClaudeAgent, + CodexAgent, + DefaultPrompts, + GeminiAgent, + OpencodeAgent, + PiAgent, + Prompts +} +import orca.progress.ProgressStore import orca.tools.opencode.OpencodeLauncher -import orca.runner.{DefaultFlowContext, LoggingListener, OrcaBanner, OrcaLog} +import orca.runner.{ + DefaultFlowContext, + FlowLifecycle, + LoggingListener, + OrcaBanner, + OrcaLog +} import orca.runner.terminal.TerminalInteraction import org.slf4j.LoggerFactory import orca.tools.FsTool import orca.tools.GitTool import orca.tools.GitHubTool +import orca.tools.OsGitTool import orca.util.OrcaDebug import ox.supervised @@ -27,8 +45,8 @@ import scala.util.control.NonFatal * given. * * ``` - * flow(OrcaArgs(args)): - * val plan = claude.resultAs[Plan].autonomous.run(userPrompt) + * flow(OrcaArgs(args), _.claude): + * val plan = agent.resultAs[Plan].autonomous.run(userPrompt) * ... * ``` * @@ -43,26 +61,45 @@ import scala.util.control.NonFatal * ... * ``` * + * The leading agent is named by a required `agent` selector resolved against + * the built `FlowContext`: the only way to name an agent is the accessor on + * the context, which isn't in scope at the `flow(...)` argument position, so + * the selector defers resolution until the context exists. + * `flow(OrcaArgs(args), _.claude)` runs against claude; `flow(OrcaArgs(args), + * _.codex)` against codex, etc. Inside the body, reference the resolved lead + * via the backend-agnostic [[agent]] accessor (not a concrete + * `claude`/`codex`) so switching the selector switches the whole flow. + * + * `B` is the leading agent's backend tag, inferred from the selector + * (`_.claude` ⇒ `ClaudeCode`) and never written by callers; the runtime pins + * it into `FlowContext.LeadB` so `agent` is concretely typed and sessions + * thread. + * * Overrides default to `None` so the runtime can build the default lazily — * `TerminalInteraction`, in particular, takes the resolved `workDir` which * can't be threaded through a Scala 3 default-arg expression. */ -def flow( +def flow[B <: BackendTag]( args: OrcaArgs, + agent: FlowContext => Agent[B], workDir: os.Path = os.pwd, interaction: Option[Interaction] = None, extraListeners: List[OrcaListener] = Nil, - claude: Option[ClaudeTool] = None, - opencode: Option[OpencodeTool] = None, + branchNaming: Option[BranchNamingStrategy] = None, + returnToStartBranch: Boolean = false, + progressStore: Option[ProgressStore] = None, + claude: Option[ClaudeAgent] = None, + codex: Option[CodexAgent] = None, + opencode: Option[OpencodeAgent] = None, opencodeLauncher: OpencodeLauncher = OpencodeLauncher.default, - pi: Option[PiTool] = None, + pi: Option[PiAgent] = None, + gemini: Option[GeminiAgent] = None, git: Option[GitTool] = None, gh: Option[GitHubTool] = None, fs: Option[FsTool] = None, prompts: Prompts = DefaultPrompts, pricing: PriceList = Pricing.default -)(body: FlowContext ?=> Unit): Unit = - val debug = OrcaDebug.enabled || args.verbose.value +)(body: FlowControl ?=> Unit): Unit = // Per-run trace file: captures every stage, prompt, tool/subprocess call and // result at DEBUG. Started before anything logs so the whole run is caught; // the path is printed by the banner and the detail stays in the file. @@ -83,55 +120,28 @@ def flow( // `try/finally` so the cost summary always lands — even when a fatal // throwable (OOM, StackOverflow) escapes the NonFatal catch below. // Tokens may have already been spent; the user deserves to see what. - // Default TerminalInteraction is built inside `supervised:` because its - // worker is a `forkUser` bound to that scope; close() in the body's - // `finally` lets the worker drain and exit before the scope joins it. try try - supervised: - val effectiveInteraction = interaction.getOrElse( - TerminalInteraction.start(workDir = Some(workDir)) - ) - try - val dispatcher = new EventDispatcher( - effectiveInteraction.listeners ++ List( - costTracker, - new LoggingListener - ) ++ extraListeners - ) - val ctx = DefaultFlowContext.withDefaults( - userPrompt = args.userPrompt, - dispatcher = dispatcher, - workDir = workDir, - interaction = effectiveInteraction, - claude = claude, - opencode = opencode, - opencodeLauncher = opencodeLauncher, - pi = pi, - git = git, - gh = gh, - fs = fs, - prompts = prompts - ) - // The whole flow body runs as a top-level stage: an otherwise - // unhandled exception surfaces as a single Error event (the same - // message a stage failure shows). A nested stage / `fail` marks the - // exception `alreadyEmitted` once it has reported it, so we don't - // re-report it here. The stack goes to the trace file only (DEBUG, - // below the console's WARN threshold); `--verbose` also prints it to - // stderr. - try body(using ctx) - catch - case NonFatal(e) => - val alreadyEmitted = e match - case fe: OrcaFlowException => fe.alreadyEmitted - case _ => false - if !alreadyEmitted then - ctx.emit(OrcaEvent.Error(throwableMessage(e))) - flowLog.debug("flow aborted", e) - if debug then e.printStackTrace(System.err) - throw e - finally effectiveInteraction.close() + runFlow( + args, + agent, + workDir, + interaction, + extraListeners ++ List(costTracker), + branchNaming, + returnToStartBranch, + progressStore, + claude, + codex, + opencode, + opencodeLauncher, + pi, + gemini, + git, + gh, + fs, + prompts + )(body) catch // The failure was already surfaced inside the scope (the flow body runs // as a top-level stage): the message went to the console, the stack to @@ -147,6 +157,147 @@ def flow( // finished it before exiting. orcaLog.finish() +/** Exit-free flow lifecycle: builds the interaction/context, runs setup, then + * runs the body as a top-level stage with disjoint success/failure teardown. + * Unlike [[flow]], a `NonFatal` failure in `body` is **propagated** (after + * failure teardown), not turned into a `System.exit` — so the + * crash→`resetHard`→resume wiring is directly testable end-to-end. [[flow]] + * wraps this to keep the observable CLI behaviour (cost summary, OrcaLog, + * `System.exit(1)`). + * + * `extraListeners` is the full listener set this run should observe beyond the + * interaction's own (the CLI wrapper adds its [[CostTracker]] here); a + * [[LoggingListener]] is always appended. + */ +private[orca] def runFlow[B <: BackendTag]( + args: OrcaArgs, + agent: FlowContext => Agent[B], + workDir: os.Path, + interaction: Option[Interaction], + extraListeners: List[OrcaListener], + branchNaming: Option[BranchNamingStrategy], + returnToStartBranch: Boolean, + progressStore: Option[ProgressStore], + claude: Option[ClaudeAgent], + codex: Option[CodexAgent], + opencode: Option[OpencodeAgent], + opencodeLauncher: OpencodeLauncher, + pi: Option[PiAgent], + gemini: Option[GeminiAgent], + git: Option[GitTool], + gh: Option[GitHubTool], + fs: Option[FsTool], + prompts: Prompts +)(body: FlowControl ?=> Unit): Unit = + val debug = OrcaDebug.enabled || args.verbose.value + val flowLog = LoggerFactory.getLogger("orca.flow") + // Default TerminalInteraction is built inside `supervised:` because its + // worker is a `forkUser` bound to that scope; close() in the body's + // `finally` lets the worker drain and exit before the scope joins it. + supervised: + val effectiveInteraction = interaction.getOrElse( + TerminalInteraction.start(workDir = Some(workDir)) + ) + // Set once the context exists; called in the body's `finally` (below) so + // context-owned background forks — the opencode `serve` drains — are torn + // down BEFORE this `supervised` scope joins them. Ox runs `releaseAfterScope` + // after the join, so this must be a body-finally, not a finalizer. + var closeContext: () => Unit = () => () + try + val dispatcher = new EventDispatcher( + effectiveInteraction.listeners ++ List( + new LoggingListener + ) ++ extraListeners + ) + // Resolve the git tool up-front: the lifecycle's setup (stash, branch + // checkout, header commit) and teardown (cleanup, restore) run before + // and after the body, outside any user stage, so they need git + // directly. The same instance is handed to the context. + val effectiveGit = git.getOrElse(new OsGitTool(workDir, dispatcher)) + // Order matters (and is delicate): the leading agent is a selector + // resolved against the context, and branch setup needs the resolved agent + // for branch naming. So the store and the context must exist BEFORE setup + // runs: + // 1. Resolve the progress store (pure — no git effect, no agent). + // 2. Build the context (pure construction — backends are created but + // no subprocess spawns until the first gated `run`). + // 3. Resolve the leading agent (`agent(ctx)`) and run branch setup + // (stash → resume-vs-fresh → checkout → header commit) using it for + // branch naming. + // Teardown is unchanged (the `bodySucceeded` gate + disjoint + // success/failure paths below), so CORE's invariants are preserved. + val store = + progressStore.getOrElse( + ProgressStore.default(workDir, args.userPrompt) + ) + val ctx = DefaultFlowContext.withDefaults( + userPrompt = args.userPrompt, + dispatcher = dispatcher, + workDir = workDir, + interaction = effectiveInteraction, + progressStore = store, + agentSelector = agent, + claude = claude, + codex = codex, + opencode = opencode, + opencodeLauncher = opencodeLauncher, + pi = pi, + gemini = gemini, + git = Some(effectiveGit), + gh = gh, + fs = fs, + prompts = prompts + ) + closeContext = () => ctx.close() + // The context resolved the leading agent (lazily, against itself) and + // exposes it as `ctx.agent`. The runtime needs it (erased) for branch + // naming and session rehydration, which run before the body. + val setup = + FlowLifecycle.setup(args, ctx.agent, effectiveGit, branchNaming, store) + // Rehydrate the client→server session map: the registry is + // in-memory, so on resume the leading agent's mapping is empty. Replay + // the persisted records into it (after the context + log exist, before + // the body) so `dispatchFor` resumes the right server thread and the + // server-id existence probes work. Only the leading agent is rehydrated — + // the common case; multi-tool flows are a known limitation (see report). + FlowLifecycle.rehydrateSessions(ctx.agent, store) + // The whole flow body runs as a top-level stage: an otherwise + // unhandled exception surfaces as a single Error event (the same + // message a stage failure shows). A nested stage / `fail` marks the + // exception `alreadyEmitted` once it has reported it, so we don't + // re-report it here. The stack goes to the trace file only (DEBUG, + // below the console's WARN threshold); `--verbose` also prints it to + // stderr. + // + // Teardown separation: body-failure and body-success teardowns are + // completely disjoint. A success-teardown error (e.g. a cosmetic + // cleanup-commit failure) must NOT trigger the failure teardown + // (`resetHard`), and must NOT strand the user on the feature branch. + var bodySucceeded = false + try + // The body reads the lead via the `agent` accessor; `ctx.LeadB` (pinned + // to `B` at construction) keeps it concretely typed so sessions thread. + body(using ctx) + bodySucceeded = true + catch + case NonFatal(e) => + val alreadyEmitted = e match + case fe: OrcaFlowException => fe.alreadyEmitted + case _ => false + if !alreadyEmitted then ctx.emit(OrcaEvent.Error(throwableMessage(e))) + flowLog.debug("flow aborted", e) + if debug then e.printStackTrace(System.err) + FlowLifecycle.teardownFailure(effectiveGit) + throw e + if bodySucceeded then + FlowLifecycle.teardownSuccess(effectiveGit, setup, returnToStartBranch) + finally + // Both run before the `supervised` scope joins its forks. closeContext + // first: it destroys the opencode `serve` process so its drain forks' + // reads EOF and the join can't hang. + try closeContext() + finally effectiveInteraction.close() + private def installUncaughtExceptionHandler(): Unit = // Idempotent across nested or repeated `flow(...)` calls — we only install // our handler if no app-specific one is already in place. The `orca` logger diff --git a/runner/src/main/scala/orca/runner/DefaultFlowContext.scala b/runner/src/main/scala/orca/runner/DefaultFlowContext.scala index 4bf50bcf..bf383645 100644 --- a/runner/src/main/scala/orca/runner/DefaultFlowContext.scala +++ b/runner/src/main/scala/orca/runner/DefaultFlowContext.scala @@ -1,31 +1,34 @@ package orca.runner -import orca.{FlowContext} +import orca.{FlowContext, FlowControl} +import orca.progress.ProgressStore import orca.tools.{GitTool} import orca.tools.{GitHubTool} import orca.tools.{FsTool} -import orca.llm.{ - ClaudeTool, - CodexTool, - GeminiTool, - LlmConfig, - OpencodeTool, - PiTool, +import orca.agents.{ + Agent, + AgentConfig, + BackendTag, + ClaudeAgent, + CodexAgent, + GeminiAgent, + OpencodeAgent, + PiAgent, Prompts } import orca.events.{EventDispatcher, OrcaEvent} import orca.backend.Interaction -import orca.tools.claude.{ClaudeBackend, DefaultClaudeTool} -import orca.tools.codex.{CodexBackend, DefaultCodexTool} +import orca.tools.claude.{ClaudeBackend, DefaultClaudeAgent} +import orca.tools.codex.{CodexBackend, DefaultCodexAgent} import orca.tools.opencode.{ - DefaultOpencodeTool, + DefaultOpencodeAgent, OpencodeBackend, OpencodeLauncher } -import orca.tools.pi.{DefaultPiTool, PiBackend} -import orca.tools.gemini.{GeminiBackend, DefaultGeminiTool} -import orca.llm.DefaultPrompts +import orca.tools.pi.{DefaultPiAgent, PiBackend} +import orca.tools.gemini.{GeminiBackend, DefaultGeminiAgent} +import orca.agents.DefaultPrompts import orca.subprocess.OsProcCliRunner import orca.tools.OsFsTool import orca.tools.OsGitTool @@ -35,53 +38,122 @@ import orca.tools.OsGitHubTool * `flow(args, ...)`, which supplies defaults for all tools. Individual tools * can be replaced by passing overrides as named arguments to `flow`. */ -private[orca] class DefaultFlowContext( +private[orca] class DefaultFlowContext[B <: BackendTag]( val userPrompt: String, dispatcher: EventDispatcher, - val claude: ClaudeTool, - val codex: CodexTool, - val opencode: OpencodeTool, - val pi: PiTool, - val gemini: GeminiTool, + agentSelector: FlowContext => Agent[B], + val claude: ClaudeAgent, + val codex: CodexAgent, + val opencode: OpencodeAgent, + val pi: PiAgent, + val gemini: GeminiAgent, val git: GitTool, val gh: GitHubTool, - val fs: FsTool -) extends FlowContext: + val fs: FsTool, + val progressStore: ProgressStore, + closeHook: () => Unit = () => () +) extends FlowControl: + + /** Tear down context-owned background resources (currently the shared + * opencode `serve` process and its drain forks). The runner calls this in + * the flow body's `finally`, BEFORE the flow scope joins its forks — Ox runs + * `releaseAfterScope` finalizers after the join, so a fork blocked on a + * non-interruptible read must be unblocked here (by destroying the process) + * rather than via `releaseAfterScope`. Idempotent / no-op when nothing + * started. + */ + def close(): Unit = closeHook() + + // The leading agent's backend tag, pinned from the type parameter `B` (which + // `flow` inferred from the selector). Concrete here, so `agent` is concretely + // typed and sessions thread; abstract when the context is seen as `FlowContext` + // in a body, where the path-dependent `ctx.LeadB` is still stable. + type LeadB = B + + // The leading agent, resolved by the selector against this context (the only + // way to name an agent is an accessor on it — `_.claude`, `_.codex`, …). + // Resolved lazily so the selector sees a fully-built context. + lazy val agent: Agent[B] = agentSelector(this) def emit(event: OrcaEvent): Unit = dispatcher.onEvent(event) + // Per-run occurrence counter. A ConcurrentHashMap + AtomicInteger is pure + // atomic state (no class-level `var`); stages run sequentially, so this is + // really just sequential bookkeeping made safe by construction. + private val occurrences = + new java.util.concurrent.ConcurrentHashMap[ + String, + java.util.concurrent.atomic.AtomicInteger + ] + + def nextOccurrence(stageName: String): Int = + occurrences + .computeIfAbsent( + stageName, + _ => new java.util.concurrent.atomic.AtomicInteger(0) + ) + .getAndIncrement() + + // Independent of the stage counter so sessions can be obtained outside stages + // without perturbing stage occurrence indices. + private val sessionOccurrences = + new java.util.concurrent.atomic.AtomicInteger(0) + + def nextSessionOccurrence(): Int = sessionOccurrences.getAndIncrement() + private[orca] object DefaultFlowContext: /** Build a context with Orca's default tool implementations, filling in any * `None` override with the production default. */ - def withDefaults( + def withDefaults[B <: BackendTag]( userPrompt: String, dispatcher: EventDispatcher, workDir: os.Path, interaction: Interaction, - claude: Option[ClaudeTool] = None, - codex: Option[CodexTool] = None, - opencode: Option[OpencodeTool] = None, + progressStore: ProgressStore, + agentSelector: FlowContext => Agent[B], + claude: Option[ClaudeAgent] = None, + codex: Option[CodexAgent] = None, + opencode: Option[OpencodeAgent] = None, opencodeLauncher: OpencodeLauncher = OpencodeLauncher.default, - pi: Option[PiTool] = None, - gemini: Option[GeminiTool] = None, + pi: Option[PiAgent] = None, + gemini: Option[GeminiAgent] = None, git: Option[GitTool] = None, gh: Option[GitHubTool] = None, fs: Option[FsTool] = None, prompts: Prompts = DefaultPrompts - )(using ox.Ox, ox.channels.BufferCapacity): DefaultFlowContext = - new DefaultFlowContext( + )(using ox.Ox, ox.channels.BufferCapacity): DefaultFlowContext[B] = + // Build the opencode agent up-front so the default backend's `shutdown` can + // be wired into the context's `close` (the runner calls it in the flow body's + // finally). A caller-supplied opencode agent owns its own lifecycle, so the + // hook is a no-op then. + val (opencodeAgent, opencodeClose): (OpencodeAgent, () => Unit) = + opencode match + case Some(a) => (a, () => ()) + case None => + val backend = OpencodeBackend(OsProcCliRunner, opencodeLauncher) + val a = new DefaultOpencodeAgent( + backend = backend, + config = AgentConfig.default, + prompts = prompts, + workDir = workDir, + events = dispatcher, + interaction = interaction + ) + (a, () => backend.shutdown()) + new DefaultFlowContext[B]( userPrompt = userPrompt, dispatcher = dispatcher, + agentSelector = agentSelector, claude = claude.getOrElse( - new DefaultClaudeTool( + new DefaultClaudeAgent( backend = new ClaudeBackend(OsProcCliRunner), // Bare `claude` defaults to Opus with the 1M context window — the // implementer session is long-lived, so it needs the big window. // `claude.sonnet` / `claude.haiku` opt down for cheap one-shots. config = - LlmConfig.default.copy(model = Some(DefaultClaudeTool.Opus1M)), + AgentConfig.default.copy(model = Some(DefaultClaudeAgent.Opus1M)), prompts = prompts, workDir = workDir, events = dispatcher, @@ -89,29 +161,20 @@ private[orca] object DefaultFlowContext: ) ), codex = codex.getOrElse( - new DefaultCodexTool( + new DefaultCodexAgent( backend = new CodexBackend(OsProcCliRunner), - config = LlmConfig.default, - prompts = prompts, - workDir = workDir, - events = dispatcher, - interaction = interaction - ) - ), - opencode = opencode.getOrElse( - new DefaultOpencodeTool( - backend = OpencodeBackend(OsProcCliRunner, opencodeLauncher), - config = LlmConfig.default, + config = AgentConfig.default, prompts = prompts, workDir = workDir, events = dispatcher, interaction = interaction ) ), + opencode = opencodeAgent, pi = pi.getOrElse( - new DefaultPiTool( + new DefaultPiAgent( backend = new PiBackend(OsProcCliRunner), - config = LlmConfig.default, + config = AgentConfig.default, prompts = prompts, workDir = workDir, events = dispatcher, @@ -119,12 +182,13 @@ private[orca] object DefaultFlowContext: ) ), gemini = gemini.getOrElse( - new DefaultGeminiTool( + new DefaultGeminiAgent( backend = new GeminiBackend(OsProcCliRunner), // Bare `gemini` pins Gemini Pro (the strong model, like claude // defaults to Opus for the long-lived implementer); `gemini.flash` // opts down for cheap one-shots. - config = LlmConfig.default.copy(model = Some(DefaultGeminiTool.Pro)), + config = + AgentConfig.default.copy(model = Some(DefaultGeminiAgent.Pro)), prompts = prompts, workDir = workDir, events = dispatcher, @@ -135,5 +199,7 @@ private[orca] object DefaultFlowContext: gh = gh.getOrElse( new OsGitHubTool(OsProcCliRunner, workDir, events = dispatcher) ), - fs = fs.getOrElse(new OsFsTool(workDir)) + fs = fs.getOrElse(new OsFsTool(workDir)), + progressStore = progressStore, + closeHook = opencodeClose ) diff --git a/runner/src/main/scala/orca/runner/FlowLifecycle.scala b/runner/src/main/scala/orca/runner/FlowLifecycle.scala new file mode 100644 index 00000000..52e39a12 --- /dev/null +++ b/runner/src/main/scala/orca/runner/FlowLifecycle.scala @@ -0,0 +1,227 @@ +package orca.runner + +import orca.{BranchNamingStrategy, InStage, OrcaArgs, OrcaFlowException} +import orca.agents.{BackendTag, Agent, SessionId} +import orca.progress.{ProgressHeader, ProgressStore, RecoveryCheck} +import orca.tools.GitTool + +import scala.util.control.NonFatal + +/** Flow setup/teardown/recovery lifecycle (ADR 0018 §2.4/§2.5). Extracted from + * the `flow` entry point so `flow.scala` holds only the entry point and + * orchestration: this object owns the privileged, outside-any-user-stage git + * and progress-store mutations that bracket the body. + */ +object FlowLifecycle: + + /** Replay the persisted resume-wire-id map (ADR 0018 §2.6) into the leading + * agent's in-memory registry, so a resumed run resumes against the right + * wire id and the existence probes target the right id. Reads every + * [[orca.progress.SessionRecord]] that carries a `resumeWireId` and + * registers it via [[orca.agents.Agent.registerResumeWireId]]. + * + * The type parameter `B` binds the wildcard backend tag from the `agent` + * parameter (`Agent[?]`) so the client/wire [[orca.agents.SessionId]]s share + * its type. Only the leading agent is rehydrated (the common case); a flow + * that drives a second tool's sessions across resume is a known limitation. + */ + private[orca] def rehydrateSessions[B <: BackendTag]( + agent: Agent[B], + store: ProgressStore + ): Unit = + for + log <- store.load().toList + record <- log.sessions + wireId <- record.resumeWireId + do + agent.registerResumeWireId( + SessionId[B](record.id), + SessionId[B](wireId) + ) + + /** Outcome of [[setup]]: the resolved progress store, the feature branch the + * run is bound to, and the starting branch to restore on success. + */ + private[orca] case class FlowSetup( + store: ProgressStore, + featureBranch: String, + startBranch: String + ) + + /** Bind the run to a branch + progress log before the body runs (ADR 0018 + * §2.4/§2.5). Records the starting branch, snapshots the log file, stashes a + * dirty tree, then either resumes an existing log or starts fresh (resolve a + * branch name, create it, write + commit the header). All git/store + * mutations run with a runtime-minted `InStage` — setup is privileged, + * predating any user stage. + * + * The progress header is **untrusted input** on load (the log is + * human-visible and pushable), so a resumed run: + * - Snapshots the log file BEFORE `ensureClean` and restores it if the + * stash removed it, so the header is always readable. + * - Validates the header before any destructive action (safe refs, + * prompt-hash match, no protected feature branch). A + * parseable-but-invalid header is a HARD abort (`OrcaFlowException`), + * not a silent fresh start — it signals tampering or a mismatch. (An + * *unparseable* log stays `store.load() == None` → fresh run; that path + * is separate.) + * - Cross-checks that the current branch is the one the header records + * (the in-place invariant): a log that surfaced on a branch it does not + * name (e.g. its feature branch was merged, carrying the log along) + * aborts rather than resuming against the wrong branch. + * + * On resume `startBranch` is the header's recorded `startingBranch` (the + * ORIGINAL branch at first run), so when a run does return to start (a PR + * flow via `returnToStartBranch`, or a throwaway) it goes to that original + * branch — exactly like a fresh run — not the re-run's current (feature) + * branch. + */ + private[orca] def setup( + args: OrcaArgs, + agent: Agent[?], + git: GitTool, + branchNaming: Option[BranchNamingStrategy], + store: ProgressStore + ): FlowSetup = + given InStage = InStage.unsafe + val startBranch = git.currentBranch() + // Snapshot the log file before the stash, restore it if the stash + // removed it — so an uncommitted/untracked log is still readable below. + val snapshot = snapshotLog(store.path) + val _ = git.ensureClean("orca: starting flow") + restoreLogIfMissing(store.path, snapshot) + store.load() match + case Some(log) => + val header = log.header + // Validate the untrusted header before any destructive action. The + // protected set is the main/master floor plus the repo's ACTUAL default + // branch (best-effort), so a tampered header naming e.g. `trunk` as a + // feature branch is refused too. + val protectedBranches = + Set("main", "master") ++ git.defaultBranch().map(_.toLowerCase) + RecoveryCheck.validateHeader( + header, + args.userPrompt, + protectedBranches + ) match + case Left(reason) => + throw new OrcaFlowException( + s"refusing to resume: progress log header failed validation ($reason)" + ) + case Right(()) => () + // Only resume IN PLACE. If the log surfaced on a branch it does not + // name, it was likely carried here by a merge — abort, don't replay. + val current = git.currentBranch() + if current != header.branch then + throw new OrcaFlowException( + s"progress log for branch '${header.branch}' found while on " + + s"'$current' — was it merged? aborting rather than resuming " + + "against the wrong branch" + ) + // Resume in place: already on header.branch. The recorded start branch + // (where a PR flow / throwaway returns to) is the ORIGINAL one, not this + // feature branch. + FlowSetup(store, header.branch, header.startingBranch) + case None => + // Fresh run: resolve + create the branch, then commit the header so it + // is the branch's first commit. + val strategy = + branchNaming.getOrElse(BranchNamingStrategy.shortenPrompt) + val branch = strategy.resolve(args.userPrompt, agent) + git.checkoutOrCreate(branch) + store.writeHeader( + ProgressHeader( + startingBranch = startBranch, + branch = branch, + promptHash = ProgressStore.hashPrompt(args.userPrompt) + ) + ) + git.forceAdd(store.path) + val _ = git.commit("orca: progress log") + FlowSetup(store, branch, startBranch) + + /** Read the bytes of the progress-log file if it exists. Returns `None` when + * the file is absent — the normal fresh-run case and the case where the log + * is committed (so the stash can't remove it). + */ + private[runner] def snapshotLog(path: os.Path): Option[Array[Byte]] = + if os.exists(path) then Some(os.read.bytes(path)) else None + + /** Restore the progress-log file from a pre-stash snapshot if the stash + * removed it, so the header is always readable. A no-op when there was + * nothing to snapshot or the file still exists. + */ + private[runner] def restoreLogIfMissing( + path: os.Path, + snapshot: Option[Array[Byte]] + ): Unit = + snapshot.foreach: bytes => + if !os.exists(path) then os.write.over(path, bytes, createFolders = true) + + /** Successful teardown (ADR 0018 §2.5): remove the progress-log file in a + * final commit so a merged branch is clean, then hand off to + * [[finishBranch]] for where HEAD lands — stay on the feature branch + * (default), return to the starting branch (`returnToStartBranch`), or + * delete a throwaway and return to start. + * + * Errors during log removal, the cleanup commit, or the branch handoff are + * cosmetic — swallowed (in a `finally`) so they don't trigger the failure + * path or strand the user. + */ + private[orca] def teardownSuccess( + git: GitTool, + setup: FlowSetup, + returnToStartBranch: Boolean + ): Unit = + // Teardown is runtime code running outside any user stage, so it mints its + // own `InStage` — the runtime is the privileged token constructor. + given InStage = InStage.unsafe + try + // Best-effort: a missing file (already gone) or a failing cleanup commit is + // cosmetic on an already-successful run, so neither must escape teardown. + try os.remove(setup.store.path) + catch + case _: java.nio.file.NoSuchFileException => () + // `add -A` in commit picks up the removal; NothingToCommit (a Left) means + // it was never committed — harmless. A genuine commit failure is swallowed: + // the run already succeeded and the progress file is gone from the tree. + try + val _ = git.commit("orca: remove progress log") + catch case NonFatal(_) => () + finally + try finishBranch(git, setup, returnToStartBranch) + catch case NonFatal(_) => () + + /** Where HEAD ends up after a successful run. A throwaway feature branch + * (only orca bookkeeping, no user code vs the start branch — diff excluding + * `.orca/` is empty) is always deleted and HEAD returns to the starting + * branch: there's nothing to keep. Otherwise the feature branch is kept, and + * `returnToStartBranch` chooses where HEAD lands — stay on the feature + * branch (the default, so the user ends on the work) or return to the + * starting branch (PR flows, done with the branch once the PR is up). + * Best-effort and success-path-only; never deletes start/protected branches. + */ + private def finishBranch( + git: GitTool, + setup: FlowSetup, + returnToStartBranch: Boolean + )(using InStage): Unit = + val throwaway = + setup.featureBranch != setup.startBranch && + git + .diffBranchExcludingOrca(setup.startBranch, setup.featureBranch) + .isBlank + if throwaway then + git.checkoutOrCreate(setup.startBranch) + git.deleteBranch(setup.featureBranch) + else if returnToStartBranch then git.checkoutOrCreate(setup.startBranch) + // else: stay on the feature branch (the default). + + /** Failure teardown (ADR 0018 §2.5): discard the failed stage's uncommitted + * partial edits with `git reset --hard` (which restores the last committed + * log), and stay on the feature branch so the next run resumes in place. + */ + private[orca] def teardownFailure(git: GitTool): Unit = + // Runtime teardown mints its own token, as in `teardownSuccess`. + given InStage = InStage.unsafe + git.resetHard() diff --git a/runner/src/main/scala/orca/runner/LoggingListener.scala b/runner/src/main/scala/orca/runner/LoggingListener.scala index 9fe7a219..968e6860 100644 --- a/runner/src/main/scala/orca/runner/LoggingListener.scala +++ b/runner/src/main/scala/orca/runner/LoggingListener.scala @@ -9,9 +9,9 @@ import org.slf4j.LoggerFactory * uses, steps (git / gh / recovery / …), structured results, token usage, and * errors. Wired into the flow's `EventDispatcher` alongside the cost tracker. * - * This is a trace mirror, not a console channel: the whole `orca.*` logger tree - * is routed to the trace file only (`OrcaLog` makes it non-additive), so even - * the `Error` line — logged at ERROR for greppability — never reaches the + * This is a trace mirror, not a console channel: the whole `orca.*` logger + * tree is routed to the trace file only (`OrcaLog` makes it non-additive), so + * even the `Error` line — logged at ERROR for greppability — never reaches the * console. The terminal renderer owns the console (it shows the `✖`). * * Messages are plain ASCII on purpose — the trace file is read back and dumped diff --git a/runner/src/main/scala/orca/runner/OrcaLog.scala b/runner/src/main/scala/orca/runner/OrcaLog.scala index 2d0dcc30..207c7761 100644 --- a/runner/src/main/scala/orca/runner/OrcaLog.scala +++ b/runner/src/main/scala/orca/runner/OrcaLog.scala @@ -21,9 +21,9 @@ import scala.util.control.NonFatal * reaches the console's WARN appender, unaffected. * * The file is intentionally NOT deleted on exit, so it can be inspected after - * the run. If logback isn't the active slf4j backend, or the temp file can't be - * created, file logging is skipped (best-effort) rather than failing the flow — - * [[file]] is then `None`. + * the run. If logback isn't the active slf4j backend, or the temp file can't + * be created, file logging is skipped (best-effort) rather than failing the + * flow — [[file]] is then `None`. */ private[orca] final class OrcaLog private ( val file: Option[os.Path], diff --git a/runner/src/main/scala/orca/runner/terminal/ConversationRenderer.scala b/runner/src/main/scala/orca/runner/terminal/ConversationRenderer.scala index 931316c4..d5b43698 100644 --- a/runner/src/main/scala/orca/runner/terminal/ConversationRenderer.scala +++ b/runner/src/main/scala/orca/runner/terminal/ConversationRenderer.scala @@ -1,12 +1,12 @@ package orca.runner.terminal import orca.OrcaInteractiveCancelled -import orca.llm.BackendTag +import orca.agents.BackendTag import orca.backend.{ ApprovalDecision, Conversation, ConversationEvent, - LlmResult + AgentResult } import org.jline.reader.{LineReader, LineReaderBuilder, UserInterruptException} import org.jline.terminal.TerminalBuilder @@ -77,7 +77,7 @@ private[terminal] class ConversationRenderer( */ def render[B <: BackendTag]( conversation: Conversation[B] - ): Either[OrcaInteractiveCancelled, LlmResult[B]] = + ): Either[OrcaInteractiveCancelled, AgentResult[B]] = try conversation.events.foreach(dispatch(_, conversation)) // A well-behaved backend ends each turn with AssistantTurnEnd, diff --git a/runner/src/main/scala/orca/runner/terminal/TerminalInteraction.scala b/runner/src/main/scala/orca/runner/terminal/TerminalInteraction.scala index b721749b..a03cada3 100644 --- a/runner/src/main/scala/orca/runner/terminal/TerminalInteraction.scala +++ b/runner/src/main/scala/orca/runner/terminal/TerminalInteraction.scala @@ -1,8 +1,8 @@ package orca.runner.terminal -import orca.backend.{Conversation, Interaction, LlmResult} +import orca.backend.{Conversation, Interaction, AgentResult} import orca.events.OrcaListener -import orca.llm.BackendTag +import orca.agents.BackendTag import ox.Ox import ox.channels.BufferCapacity import ox.either.orThrow @@ -52,7 +52,7 @@ class TerminalInteraction private[terminal] ( * when the conversation finishes. Backend errors surface as * `OrcaInteractiveCancelled` or other throwables from `awaitResult`. */ - def drive[B <: BackendTag](conversation: Conversation[B]): LlmResult[B] = + def drive[B <: BackendTag](conversation: Conversation[B]): AgentResult[B] = new ConversationRenderer( useColor = useColor, output = output, diff --git a/runner/src/test/scala/flowtests/FlowCompilesTest.scala b/runner/src/test/scala/flowtests/FlowCompilesTest.scala index 5abc4a5c..11f037a1 100644 --- a/runner/src/test/scala/flowtests/FlowCompilesTest.scala +++ b/runner/src/test/scala/flowtests/FlowCompilesTest.scala @@ -20,6 +20,7 @@ import orca.{*, given} // failure, referenced by name in `examples/implement-enhanced.sc`. Pinning it // here keeps the "import it explicitly" requirement honest. import orca.tools.PrAlreadyExists +import orca.agents.AgentConfig case class PlanTask(branchName: String, description: String) derives JsonData case class FlowPlan(tasks: List[PlanTask]) derives JsonData @@ -27,13 +28,18 @@ case class BranchSlug(name: String) derives JsonData object FlowCanary: + // The leading model is named by a `flow(...)` selector resolved against the + // flow context (ADR 0018 §2.5). A positional `agent` selector is + // required: `_.claude`, `_.codex`, etc. These canaries use the real shapes + // the `examples/*.sc` files use — no hand-built stub. + /** Structured output via `derives JsonData` must be reachable through the * `resultAs[O]` path without any extra imports. */ def structuredResult(): Unit = - flow(OrcaArgs()): + flow(OrcaArgs(), _.claude): stage("plan"): - val session = claude.newSession + val session = claude.session(seed = userPrompt) val _ = claude.resultAs[FlowPlan].interactive.run(userPrompt, session) val _ = claude.resultAs[FlowPlan].interactive.run("refine", session) val _ = claude.resultAs[FlowPlan].autonomous.run(userPrompt, session) @@ -43,9 +49,9 @@ object FlowCanary: * promises for per-task implementation. */ def continuedSession(): Unit = - flow(OrcaArgs()): + flow(OrcaArgs(), _.claude): stage("impl"): - val session = claude.newSession + val session = claude.session(seed = userPrompt) val _ = claude.autonomous.run("kick off", session) val _ = claude.autonomous.run("keep going", session) val _ = claude.autonomous.run("one-shot") @@ -53,7 +59,7 @@ object FlowCanary: /** Every top-level accessor must resolve from `import orca.*` alone. */ def accessors(): Unit = - flow(OrcaArgs()): + flow(OrcaArgs(), _.claude): stage("tools"): val _ = git.createBranch("x") val _ = git.commit("msg") @@ -65,14 +71,14 @@ object FlowCanary: val _ = claude.withReadOnly.withSelfManagedGit val _ = codex.withSelfManagedGit val _ = pi.withConfig( - LlmConfig.default.copy(model = Some(Model("gpt-5.5"))) + AgentConfig.default.copy(model = Some(Model("gpt-5.5"))) ) - /** Review-and-fix loop; pulls in `allReviewers` and the internal `stage`/fork - * machinery. + /** Review-and-fix loop; pulls in `allReviewers` and the internal `display`/ + * fork machinery (which now runs under the caller's stage). */ def reviewLoop(): Unit = - flow(OrcaArgs()): + flow(OrcaArgs(), _.claude): val (sessionId, plan) = stage("plan"): claude.resultAs[FlowPlan].interactive.run(userPrompt) for task <- plan.tasks do @@ -81,18 +87,18 @@ object FlowCanary: coder = claude, sessionId = sessionId, reviewers = allReviewers(claude), - reviewerSelection = ReviewerSelector.llmDriven(claude.haiku), + reviewerSelection = ReviewerSelector.agentDriven(claude.haiku), task = task.description, formatCommand = Some("mvn -q spotless:apply"), lintCommand = Some("mvn -q test"), - lintLlm = Some(claude.haiku) + lintAgent = Some(claude.haiku) ) /** Config overrides must be reachable as unqualified names so users can write * `flow(args = ..., workDir = ...)` straight from `import orca.*`. */ def configured(): Unit = - flow(args = OrcaArgs("hello"), workDir = os.pwd): + flow(args = OrcaArgs("hello"), agent = _.claude, workDir = os.pwd): stage("cfg"): val _ = claude.autonomous.run(userPrompt) @@ -101,20 +107,20 @@ object FlowCanary: * Array[String]`. */ def fromCliArgs(args: Array[String]): Unit = - flow(OrcaArgs(args)): + flow(OrcaArgs(args), _.claude): stage("start"): val _ = claude.autonomous.run(userPrompt) /** `summarisePr` + `PrSummary` surface; exercised by `examples/issue-pr.sc`. - * Pins the call shape (`llm`, `diff`, optional `context`, optional + * Pins the call shape (`agent`, `diff`, optional `context`, optional * `instructions`) and the result type so a rename or signature drift * surfaces in this test instead of at the next live run. */ def summarisePrSurface(): Unit = - flow(OrcaArgs()): + flow(OrcaArgs(), _.claude): stage("pr"): val summary: PrSummary = summarisePr( - llm = claude.haiku, + agent = claude.haiku, diff = git.diff(), context = Some("Originating issue: acme/widgets#7") ) @@ -125,7 +131,7 @@ object FlowCanary: * `examples/`. If any of these signatures move, the canary fails. */ def issueAndPrSurface(): Unit = - flow(OrcaArgs()): + flow(OrcaArgs(), _.claude): stage("gh"): val issueHandle = IssueHandle.parseOrThrow("acme/widgets#7") val _: Either[String, IssueHandle] = IssueHandle.parse("acme/widgets#7") @@ -140,32 +146,24 @@ object FlowCanary: gh.updatePr(pr, "new title", "new body") /** Branch + PR surface — exercised by `examples/implement-enhanced.sc`. Pins - * the resumable-branch ops (`recover` guard, `ensureClean`, - * `checkoutOrCreate`, `currentBranch`, `checkout`), the `createPr` `Either` - * with its recoverable `PrAlreadyExists`, and the merge-base diff that feeds - * `summarisePr`. A signature drift surfaces here instead of at the next live - * run. + * the branch ops the runtime still exposes to flow scripts and the + * `createPr` `Either` with its recoverable `PrAlreadyExists`. The manual + * `Plan.recover`/`ensureClean`/`checkoutOrCreate` resume guard is gone — the + * flow runtime now owns branch + resume (ADR 0018 §2.5); the per-flow + * branching ceremony is restored in Epic F's example conversion. */ def branchAndPrSurface(): Unit = - flow(OrcaArgs()): - val planFile = Plan.defaultPath(userPrompt) - val startBranch = git.currentBranch() - if Plan.recover(planFile).isEmpty then - val _ = git.ensureClean("orca: pre-implement stash") - val branch = - claude.haiku.resultAs[BranchSlug].autonomous.run(userPrompt)._2.name - git.checkoutOrCreate(branch) + flow(OrcaArgs(), _.claude): stage("pr"): git.push().orThrow val summary = summarisePr( - llm = claude.haiku, + agent = claude.haiku, diff = git.diffVsBase(git.defaultBase()) ) gh.createPr(title = summary.title, body = summary.body) match case Left(_: PrAlreadyExists) => () case Left(e) => throw e case Right(_) => () - git.checkout(startBranch).orThrow /** Planning grid surface; exercised across `examples/`. Pins the full `mode × * operation` grid: every cell returns `Sessioned[B, <result>]` where the @@ -174,7 +172,7 @@ object FlowCanary: * rename/case removal surfaces here instead of at the next live run. */ def planningGridSurface(): Unit = - flow(OrcaArgs()): + flow(OrcaArgs(), _.claude): stage("grid"): // --- from → Sessioned[B, Plan], both modes --- val autoFrom: Sessioned[?, Plan] = @@ -218,35 +216,283 @@ object FlowCanary: case Triage.Untestable(_, _) => () case Triage.Testable(_, _, _) => () - /** Post-planning steps (`reviewed` / `briefed`) and the brief-aware - * persistence surface — exercised by `examples/implement-enhanced.sc`. Pins - * that the `Sessioned[B, Plan]` / `Sessioned[B, PlanWithBrief]` extensions - * resolve through `import orca.*` alone (implicit scope = the `Plan` / - * `PlanWithBrief` companions), and that both step orders type-check. + /** Post-planning step (`reviewed`) plus the per-task stage loop — exercised + * by `examples/implement-enhanced.sc`. Pins that the `Sessioned[B, Plan]` + * extension resolves through `import orca.*` alone. Plans are always briefed + * (the `brief` rides in the structured output, so `plan.brief` / + * `plan.taskPrompt` are always available — no `.briefed` step, no + * `PlanWithBrief`). The `Plan.recoverOrCreate` / `implementTaskLoop` + * persistence calls are gone — resume is now the stage log (ADR 0018 §2.8), + * and the task loop is a plain per-task `stage(...)`. */ def planReviewAndBriefSurface(): Unit = - flow(OrcaArgs()): - stage("review+brief"): - // review-then-brief and brief-then-review both yield a PlanWithBrief. - val reviewedThenBriefed: Sessioned[?, PlanWithBrief] = + flow(OrcaArgs(), _.claude): + val plan: Plan = + stage("plan"): Plan.autonomous .from(userPrompt, claude) .reviewed(claude) - .briefed(claude) - val briefedThenReviewed: Sessioned[?, PlanWithBrief] = - Plan.autonomous - .from(userPrompt, claude) - .briefed(claude) - .reviewed(claude) - val _ = briefedThenReviewed - // review alone stays a bare Plan. - val reviewedOnly: Sessioned[?, Plan] = - Plan.autonomous.from(userPrompt, claude).reviewed(claude) - val _ = reviewedOnly - - val planFile = Plan.defaultPath(userPrompt) - val plan: PlanLike = - Plan.recoverOrCreate(planFile)(reviewedThenBriefed.value) - Plan.implementTaskLoop(planFile, plan): task => - val _ = - claude.autonomous.run(plan.taskPrompt(task), claude.newSession) + .value + + for task <- plan.tasks do + stage(s"task: ${task.title.value}"): + // omitting the session arg gives a fresh one-shot session + val _ = claude.autonomous.run(plan.taskPrompt(task)) + + // ----------------------------------------------------------------------- + // Example-shape canaries (ADR 0018 §3, Task F2) + // Each def mirrors the distinct pattern in one of the `examples/*.sc` + // files so a signature drift or missing API surfaces here instead of at + // the next live run. Nothing is invoked at runtime. + // ----------------------------------------------------------------------- + + /** `implement.sc`: autonomous plan → session seeded from brief → task loop + * with `runSeeded` + `reviewAndFixLoop`. The session-based shapes + * (`session(seed=)`, `runSeeded`) are the core new-API additions. + */ + def implementFlowShape(): Unit = + flow(OrcaArgs(), _.claude): + val plan: Plan = stage("Plan"): + Plan.autonomous.from(userPrompt, claude).value + + val session = claude.session(seed = plan.brief) + + for task <- plan.tasks do + stage(s"task: ${task.title}"): + val _ = claude.runSeeded(task.description, session) + reviewAndFixLoop( + coder = claude, + sessionId = session, + reviewers = allReviewers(claude), + reviewerSelection = ReviewerSelector.agentDriven(claude.haiku), + task = task.title.value, + formatCommand = Some("cargo fmt"), + lintCommand = Some("cargo check --tests"), + lintAgent = Some(claude.haiku) + ) + + /** `implement-interactive.sc`: interactive plan → session → task loop. Only + * the planning call differs from `implementFlowShape`. + */ + def interactivePlanFlowShape(): Unit = + flow(OrcaArgs(), _.claude): + val plan: Plan = stage("Plan"): + Plan.interactive.from(userPrompt, claude).value + + val session = claude.session(seed = plan.brief) + + for task <- plan.tasks do + stage(s"task: ${task.title}"): + val _ = claude.runSeeded(task.description, session) + reviewAndFixLoop( + coder = claude, + sessionId = session, + reviewers = allReviewers(claude), + reviewerSelection = ReviewerSelector.agentDriven(claude.haiku), + task = task.title.value, + formatCommand = Some("cargo fmt") + ) + + /** `implement-enhanced.sc`: plan → `.reviewed` → session seeded from brief → + * task loop with `taskPrompt` → push → PR via `createPr`. + */ + def enhancedImplementFlowShape(): Unit = + flow(OrcaArgs(), _.claude): + val plan: Plan = stage("Plan"): + Plan.autonomous.from(userPrompt, claude).reviewed(claude).value + + val session = claude.session(seed = plan.brief) + + for task <- plan.tasks do + stage(s"task: ${task.title}"): + val _ = claude.runSeeded(plan.taskPrompt(task), session) + reviewAndFixLoop( + coder = claude, + sessionId = session, + reviewers = allReviewers(claude), + reviewerSelection = ReviewerSelector.agentDriven(claude.haiku), + task = task.title.value, + formatCommand = Some("cargo fmt") + ) + + stage("Push branch"): + git.push().orThrow + + val prSum = stage("Generate PR title and description"): + summarisePr( + agent = claude.haiku, + diff = git.diffVsBase(git.defaultBase()) + ) + + val _ = stage("Open PR"): + gh.createPr(title = prSum.title, body = prSum.body).orThrow + + /** The leading-model selector resolves against the flow context (ADR 0018 + * §2.5): a positional `agent` selector is required (`_.claude`, `_.codex`, + * …). Pins the `_.codex` positional shape so a selector regression surfaces + * here. + */ + def agentSelector(): Unit = + flow(OrcaArgs(), _.codex): + stage("lead"): + val _ = codex.autonomous.run(userPrompt) + + /** `epic.sc`: cross-backend review — claude implements, codex reviews. + * Exercises the `allReviewers(codex)` shape and `claude.opus` planning. + */ + def epicFlowShape(): Unit = + flow(OrcaArgs(), _.claude): + val plan: Plan = stage("Plan"): + Plan.autonomous.from(userPrompt, claude.opus).value + + val session = claude.session(seed = plan.brief) + val reviewers: List[Agent[?]] = allReviewers(codex) + + for task <- plan.tasks do + stage(s"task: ${task.title}"): + val _ = claude.runSeeded(task.description, session) + reviewAndFixLoop( + coder = claude, + sessionId = session, + reviewers = reviewers, + reviewerSelection = ReviewerSelector.agentDriven(claude.haiku), + task = task.title.value, + formatCommand = Some("mvn -q spotless:apply") + ) + + stage("Update documentation"): + val _ = claude.runSeeded( + "Update project docs based on the changes made.", + session + ) + + /** `issue-pr.sc`: read issue outside stage, `assessThenPlan`, optional plan, + * session from plan brief, task loop, push, PR. Also exercises + * `BranchNamingStrategy.issue` and the `Verdict` match. + */ + def issuePrFlowShape(): Unit = + val orcaArgs = OrcaArgs("acme/widgets#42") + val issueHandle = IssueHandle.parseOrThrow(orcaArgs.userPrompt) + flow( + orcaArgs, + _.claude, + branchNaming = Some(BranchNamingStrategy.issue(issueHandle)) + ): + // Read outside stage (no InStage needed). + val issue: Issue = gh.readIssue(issueHandle) + + val (maybePlan, rejectionBody) = stage("Assess and plan"): + Plan.autonomous.assessThenPlan(issue.body, claude.opus).value match + case Verdict.Rejection(_, body) => (None: Option[Plan], body) + case Verdict.Proceed(plan) => (Some(plan), "") + + if maybePlan.isEmpty then + stage("Comment: rejection"): + gh.writeComment(issueHandle, rejectionBody) + + maybePlan.foreach: plan => + val session = claude.session(seed = plan.brief) + + for task <- plan.tasks do + stage(s"task: ${task.title}"): + val _ = claude.runSeeded(task.description, session) + reviewAndFixLoop( + coder = claude, + sessionId = session, + reviewers = allReviewers(claude), + reviewerSelection = ReviewerSelector.agentDriven(claude.haiku), + task = task.title.value, + formatCommand = Some("npx prettier --write .") + ) + + stage("Push branch"): + git.push().orThrow + + val _ = stage("Open PR"): + gh.createPr(title = "fix", body = s"Closes ${issueHandle.shortRef}.") + .orThrow + + /** `issue-pr-bugfix.sc`: the push-after-edit authoring rule (ADR 0018). + * "Write failing test" commits the test; a LATER "Push + open PR" stage + * pushes it. Also covers `triage`, `waitForBuild` outside a stage, + * `claude.runSeeded` in a nested helper, and the final push+updatePr stage. + */ + def bugfixFlowShape(): Unit = + import scala.concurrent.duration.DurationInt + val orcaArgs = OrcaArgs("acme/widgets#42") + val issueHandle = IssueHandle.parseOrThrow(orcaArgs.userPrompt) + flow( + orcaArgs, + _.claude, + branchNaming = Some(BranchNamingStrategy.issue(issueHandle)) + ): + // Pure read outside any stage. + val issue: Issue = gh.readIssue(issueHandle) + + val session = claude.session(seed = issue.body) + + val triage: Triage = stage("Triage"): + Plan.autonomous.triage(issue.body, claude).value + + triage match + case Triage.NotABug(explanation) => + stage("Comment: not a bug"): + gh.writeComment(issueHandle, explanation) + + case Triage.Untestable(_, steps) => + stage("Comment: repro steps"): + gh.writeComment(issueHandle, steps) + + case Triage.Testable(summary, _, failingTestPath) => + // Stage 1: write + commit the test. + stage("Write failing test"): + val _ = claude.runSeeded( + s"Write the failing test at $failingTestPath.", + session + ) + + // Stage 2: LATER stage — push the already-committed test, open PR. + // (Authoring rule R8: push must be in a later stage than the edit.) + val pr: PrHandle = stage("Push + open tentative PR"): + git.push().orThrow + gh.createPr(title = summary, body = "Failing test only.").orThrow + + // `waitForBuild` is a pure polling read — outside any stage. + if gh + .waitForBuild(pr, 30.minutes) + .orThrow + .outcome == BuildOutcome.Success + then + fail( + "CI passed on the failing-test commit — reproduction is wrong." + ) + display(s"CI red on ${pr.shortRef} — confirmed") + + // Implement the fix in per-task stages. + val fixPlan: Plan = stage("Plan the fix"): + Plan.autonomous + .from(s"Fix ${issueHandle.shortRef}", claude) + .reviewed(claude) + .value + for task <- fixPlan.tasks do + stage(s"task: ${task.title}"): + val _ = claude.runSeeded(fixPlan.taskPrompt(task), session) + reviewAndFixLoop( + coder = claude, + sessionId = session, + reviewers = allReviewers(claude), + reviewerSelection = ReviewerSelector.agentDriven(claude.haiku), + task = task.title.value, + formatCommand = Some("sbt scalafmtAll"), + lintCommand = Some("sbt Test/compile"), + lintAgent = Some(claude.haiku) + ) + + // Stage: push fix + finalise PR (later than the fix-task stages). + stage("Push fix + finalise PR"): + git.push().orThrow + gh.updatePr( + pr, + title = "fix: " + summary, + body = "Failing test + fix." + ) diff --git a/runner/src/test/scala/orca/runner/FlowContextAgentTest.scala b/runner/src/test/scala/orca/runner/FlowContextAgentTest.scala new file mode 100644 index 00000000..220f9997 --- /dev/null +++ b/runner/src/test/scala/orca/runner/FlowContextAgentTest.scala @@ -0,0 +1,53 @@ +package orca.runner + +import orca.{OrcaArgs, agent, runFlow} +import orca.agents.Agent +import orca.runner.terminal.TerminalInteraction +import orca.tools.opencode.OpencodeLauncher +import orca.agents.DefaultPrompts +import ox.supervised + +import java.io.{ByteArrayOutputStream, PrintStream} + +/** Verifies that the `agent` selector passed to `runFlow` is resolved against + * the flow context and reachable inside the body via the `agent` accessor (the + * lead is no longer exposed as a `FlowContext` member). + */ +class FlowContextAgentTest extends munit.FunSuite: + + test("the `agent` accessor resolves the selector passed to runFlow"): + val workDir = TempRepo.create() + var seen: Option[Agent[?]] = None + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + runFlow( + args = OrcaArgs("test-agent"), + agent = _ => StubAgent.claude, + workDir = workDir, + interaction = Some(interaction), + extraListeners = Nil, + branchNaming = None, + returnToStartBranch = false, + progressStore = None, + claude = None, + codex = None, + opencode = None, + opencodeLauncher = OpencodeLauncher.default, + pi = None, + gemini = None, + git = None, + gh = None, + fs = None, + prompts = DefaultPrompts + ): + seen = Some(agent) + assert( + seen.exists(_ eq StubAgent.claude), + s"expected the `agent` accessor to be StubAgent.claude but got: $seen" + ) + +end FlowContextAgentTest diff --git a/runner/src/test/scala/orca/runner/FlowLifecycleTest.scala b/runner/src/test/scala/orca/runner/FlowLifecycleTest.scala new file mode 100644 index 00000000..82a37f86 --- /dev/null +++ b/runner/src/test/scala/orca/runner/FlowLifecycleTest.scala @@ -0,0 +1,713 @@ +package orca.runner + +import orca.{ + BranchNamingStrategy, + FlowContext, + InStage, + OrcaArgs, + runFlow, + stage, + flow +} +import orca.events.OrcaEvent +import orca.agents.{ + Announce, + AutonomousTextCall, + BackendTag, + ClaudeAgent, + DefaultPrompts, + JsonData, + AgentCall, + AgentConfig, + Model, + SessionId, + ToolSet +} +import orca.progress.{ProgressHeader, ProgressStore, SessionRecord, StageEntry} +import orca.runner.terminal.TerminalInteraction +import orca.tools.OsGitTool +import orca.tools.opencode.OpencodeLauncher +import ox.supervised + +import java.io.{ByteArrayOutputStream, PrintStream} +import java.util.concurrent.atomic.AtomicInteger + +/** Tests for the flow lifecycle: success teardown, failure teardown, and resume + * across two calls. Each test uses a real temp git repo via `TempRepo` and a + * null-sink `TerminalInteraction` so no TTY is required. + * + * The first three tests cover teardown/resume through the public `flow(...)` + * and by hand-building state — `flow()` calls `System.exit(1)` on body + * failure, so they can't drive a failing invocation directly. + * + * The last two tests exercise the genuine end-to-end crash→resume path via the + * exit-free `runFlow(...)` seam: a body that throws in stage 2 propagates (no + * `System.exit`), failure teardown keeps HEAD on the feature branch with stage + * 1 recorded, and a second `runFlow` over the same store resumes — replaying + * stage 1 instead of re-running it. + */ +class FlowLifecycleTest extends munit.FunSuite: + + test("success teardown: ends on start branch and removes progress-log file"): + val workDir = TempRepo.create() + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + flow( + args = OrcaArgs("lifecycle-success"), + agent = _ => StubAgent.claude, + workDir = workDir, + interaction = Some(interaction) + ): + summon[FlowContext].emit(OrcaEvent.Step("body ran")) + // After flow returns, HEAD must be back on main (the starting branch). + val branch = + os.proc("git", "rev-parse", "--abbrev-ref", "HEAD") + .call(cwd = workDir) + .out + .text() + .trim + assertEquals(branch, "main") + // The progress-log file must have been removed. + val store = ProgressStore.default(workDir, "lifecycle-success") + assert(!os.exists(store.path), s"progress log ${store.path} should be gone") + + test( + "failure teardown: stays on feature branch with clean working tree and earlier commit present" + ): + // Build the pre-failure state manually: feature branch with a committed + // progress header + one completed stage entry, then staged (but not yet + // committed) partial work from a second stage. + // Then apply failure teardown (resetHard) and assert the resulting state. + val workDir = TempRepo.create() + val git = new OsGitTool(workDir) + val prompt = "lifecycle-failure" + val store = ProgressStore.default(workDir, prompt) + + given InStage = InStage.unsafe + + // Mirror flowSetup: create feature branch, commit progress header. + val _ = git.createBranch("feat/lifecycle-failure") + store.writeHeader( + ProgressHeader( + startingBranch = "main", + branch = "feat/lifecycle-failure", + promptHash = ProgressStore.hashPrompt(prompt) + ) + ) + git.forceAdd(store.path) + val _ = git.commit("orca: progress log") + + // Simulate stage-one completing: write and commit code + stage entry. + os.write(workDir / "one.txt", "content") + store.appendEntry(StageEntry("stage-one#0", "stage-one", "\"done\"")) + git.forceAdd(store.path) + val _ = git.commit("stage: stage-one") + + // Simulate stage-two leaving staged but uncommitted work. + // Writing + staging makes this a modified-in-index file that `reset --hard` + // will remove (unlike untracked files which reset --hard leaves alone). + os.write(workDir / "two.txt", "partial") + val _ = os.proc("git", "add", "two.txt").call(cwd = workDir) + val featureBranch = git.currentBranch() + + // Apply failure teardown (git reset --hard, stay on feature branch). + git.resetHard() + + // HEAD must still be on the feature branch. + assertEquals(git.currentBranch(), featureBranch) + // Working tree must be clean (staged partial work was discarded). + val status = + os.proc("git", "status", "--porcelain") + .call(cwd = workDir) + .out + .text() + .trim + assertEquals( + status, + "", + "working tree must be clean after failure teardown" + ) + // Stage one's result must survive in the progress log. + val ids = store.load().get.entries.map(_.id) + assert(ids.contains("stage-one#0"), "stage one must remain recorded") + // Stage two's partial file must be gone (reset --hard wiped it from index). + assert( + !os.exists(workDir / "two.txt"), + "staged partial file must be wiped by reset --hard" + ) + + test( + "setup resume: second flow call resumes feature branch; a stage body runs only once" + ): + // Build the on-disk state that an aborted first run leaves: the feature + // branch has a committed progress header + one completed stage entry, and + // the progress-log file is present on disk (failure teardown stays on the + // feature branch without deleting the log). + val workDir = TempRepo.create() + val prompt = "lifecycle-resume" + val store = ProgressStore.default(workDir, prompt) + val invocations = new AtomicInteger(0) + + given InStage = InStage.unsafe + + val git = new OsGitTool(workDir) + val _ = git.createBranch("feat/lifecycle-resume") + store.writeHeader( + ProgressHeader( + startingBranch = "main", + branch = "feat/lifecycle-resume", + promptHash = ProgressStore.hashPrompt(prompt) + ) + ) + git.forceAdd(store.path) + val _ = git.commit("orca: progress log") + store.appendEntry( + StageEntry("resumable-stage#0", "resumable-stage", "\"ok\"") + ) + git.forceAdd(store.path) + val _ = git.commit("stage: resumable-stage") + // We are on the feature branch (as failure teardown would leave us) and the + // progress-log file is present on disk — flow() can read it via store.load(). + + // Second run: flow detects the header in the store, resumes the feature + // branch (already there), and skips the already-recorded stage body. + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + flow( + args = OrcaArgs(prompt), + agent = _ => StubAgent.claude, + workDir = workDir, + progressStore = Some(store), + interaction = Some(interaction) + ): + val _ = stage("resumable-stage"): + invocations.incrementAndGet() + "ok" + + assertEquals( + invocations.get(), + 0, + "body must NOT run on a resumed call where the stage is already recorded" + ) + // Success teardown on a resumed run returns to the ORIGINAL start branch + // recorded in the header (main), not the re-run's current feature branch. + val branch = + os.proc("git", "rev-parse", "--abbrev-ref", "HEAD") + .call(cwd = workDir) + .out + .text() + .trim + assertEquals( + branch, + "main", + "a resumed run returns to the header's original start branch" + ) + + test( + "runFlow propagates a body failure: stays on the feature branch with stage-one recorded" + ): + // End-to-end crash path: a body that completes stage 1 then THROWS in stage + // 2. `runFlow` (exit-free) must propagate the exception; failure teardown + // leaves us on the feature branch with stage 1's commit + log entry intact. + val workDir = TempRepo.create() + val prompt = "crash-feature" + val store = ProgressStore.default(workDir, prompt) + val git = new OsGitTool(workDir) + val startBranch = git.currentBranch() + + val thrown = intercept[RuntimeException]: + runFlowForTest(workDir, prompt, store): + val _ = stage("stage-one"): + os.write(workDir / "one.txt", "content") + "one-done" + val _ = stage[String]("stage-two"): + throw new RuntimeException("boom in stage two") + assertEquals(thrown.getMessage, "boom in stage two") + + // HEAD must be on the feature branch, not the start branch. + val branch = git.currentBranch() + assertNotEquals(branch, startBranch) + assertEquals(branch, store.load().get.header.branch) + + // Stage one's commit + log entry must survive. + val ids = store.load().get.entries.map(_.id) + assert(ids.contains("stage-one#0"), "stage one must be recorded") + assert( + os.exists(workDir / "one.txt"), + "stage one's committed file must survive failure teardown" + ) + + test( + "runFlow resumes after a crash: stage one replays once and ends on the original start branch" + ): + // Two runs over the SAME repo/prompt/store. The first crashes in stage 2 + // after stage 1 runs; the second resumes — stage 1's body must NOT run again + // (the recorded result is replayed), so the counter ends at 1, and the + // successful second run returns to the start branch. + val workDir = TempRepo.create() + val prompt = "resume-feature" + val store = ProgressStore.default(workDir, prompt) + val git = new OsGitTool(workDir) + val startBranch = git.currentBranch() + val stageOneRuns = new AtomicInteger(0) + + // First run: crashes in stage two. + val _ = intercept[RuntimeException]: + runFlowForTest(workDir, prompt, store): + val _ = stage("stage-one"): + stageOneRuns.incrementAndGet() + "one-done" + val _ = stage[String]("stage-two"): + throw new RuntimeException("boom") + assertEquals(stageOneRuns.get(), 1, "stage one runs once in the first run") + + // Failure teardown leaves the repo on the feature branch — which is exactly + // the resume entry point: a re-run "in place" (the next invocation inherits + // the repo's HEAD, which the crash left on the feature branch) finds the + // committed progress log in the working tree and resumes from it. + val featureBranch = git.currentBranch() + assertNotEquals(featureBranch, startBranch) + + // Second run from the feature branch: resumes; stage one is replayed from the + // log (body skipped), stage two runs fresh. + runFlowForTest(workDir, prompt, store): + val _ = stage("stage-one"): + stageOneRuns.incrementAndGet() + "one-done" + val _ = stage("stage-two"): + "two-done" + + assertEquals( + stageOneRuns.get(), + 1, + "stage one must replay (not re-run) on resume: counter stays at 1" + ) + assertEquals( + git.currentBranch(), + startBranch, + "a successful in-place resumed run returns to the original start branch" + ) + + test( + "R20: snapshot-before-stash restores the log file if the stash removed it" + ): + // The end-to-end stash hazard is belt-and-suspenders (the log is normally + // committed, so the stash can't remove it), so cover the helper directly: + // a snapshot taken while the file exists restores its exact bytes after the + // file is gone, and is a no-op when the file still exists. + val dir = os.temp.dir() + val path = dir / ".orca" / "progress-x.json" + os.write(path, "{\"header\":true}", createFolders = true) + val snapshot = FlowLifecycle.snapshotLog(path) + assert(snapshot.isDefined, "snapshot must capture an existing file") + + // Simulate the stash removing the file, then restore it. + val _ = os.remove(path) + FlowLifecycle.restoreLogIfMissing(path, snapshot) + assert(os.exists(path), "log must be restored from the snapshot") + assertEquals(os.read(path), "{\"header\":true}") + + // Restore is a no-op when the file is still present (does not overwrite). + os.write.over(path, "untouched") + FlowLifecycle.restoreLogIfMissing(path, snapshot) + assertEquals(os.read(path), "untouched") + + // A snapshot of a missing file is None; restore then does nothing. + val missing = dir / ".orca" / "absent.json" + assertEquals(FlowLifecycle.snapshotLog(missing), None) + FlowLifecycle.restoreLogIfMissing(missing, None) + assert(!os.exists(missing)) + + test( + "R30: a log whose recorded branch differs from the current branch aborts" + ): + // Simulate a merged feature branch: the committed log records branch X, but + // HEAD is on Y (as if X was merged into Y, carrying the log along). Resuming + // must abort rather than replay against the wrong branch. + val workDir = TempRepo.create() + val prompt = "merged-hazard" + val store = ProgressStore.default(workDir, prompt) + val git = new OsGitTool(workDir) + + given InStage = InStage.unsafe + // Commit the log on `main` (HEAD) while it names a different feature branch. + store.writeHeader( + ProgressHeader( + startingBranch = "main", + branch = "feat/merged-hazard", + promptHash = ProgressStore.hashPrompt(prompt) + ) + ) + git.forceAdd(store.path) + val _ = git.commit("orca: progress log") + val currentBranch = git.currentBranch() + + val thrown = intercept[orca.OrcaFlowException]: + runFlowForTest(workDir, prompt, store): + val _ = stage("never-runs"): + "x" + assert( + thrown.getMessage.contains("feat/merged-hazard") && + thrown.getMessage.contains(currentBranch) && + thrown.getMessage.contains("merged"), + s"abort message must name both branches and the merge hazard: ${thrown.getMessage}" + ) + + test( + "R32: a tampered header (prompt-hash mismatch) aborts rather than resuming" + ): + // A committed log on the feature branch whose promptHash does not match the + // current prompt — a hand-edited/mismatched header. Resume must hard-abort. + val workDir = TempRepo.create() + val prompt = "tampered-feature" + val store = ProgressStore.default(workDir, prompt) + val git = new OsGitTool(workDir) + + given InStage = InStage.unsafe + val _ = git.createBranch("feat/tampered-feature") + store.writeHeader( + ProgressHeader( + startingBranch = "main", + branch = "feat/tampered-feature", + promptHash = "deadbeefcafe" + ) + ) + git.forceAdd(store.path) + val _ = git.commit("orca: progress log") + + val thrown = intercept[orca.OrcaFlowException]: + runFlowForTest(workDir, prompt, store): + val _ = stage("never-runs"): + "x" + assert( + thrown.getMessage.contains("failed validation"), + s"abort message must mention validation failure: ${thrown.getMessage}" + ) + + test( + "rehydrate: persisted client→server map is replayed into the leading model before the body" + ): + // An aborted run left a session record carrying a learned resumeWireId. On + // resume, flow setup must replay it into the leading model's registry via + // registerResumeWireId BEFORE the body runs. + val workDir = TempRepo.create() + val prompt = "rehydrate-feature" + val store = ProgressStore.default(workDir, prompt) + val git = new OsGitTool(workDir) + + given InStage = InStage.unsafe + val _ = git.createBranch("feat/rehydrate-feature") + store.writeHeader( + ProgressHeader( + startingBranch = "main", + branch = "feat/rehydrate-feature", + promptHash = ProgressStore.hashPrompt(prompt) + ) + ) + git.forceAdd(store.path) + val _ = git.commit("orca: progress log") + store.upsertSession( + SessionRecord( + index = 0, + id = "client-uuid", + seed = "brief", + resumeWireId = Some("ses_server_1") + ) + ) + git.forceAdd(store.path) + val _ = git.commit("orca: session record") + + val recorder = new RecordingClaude + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + runFlow( + args = OrcaArgs(prompt), + agent = _ => recorder, + workDir = workDir, + interaction = Some(interaction), + extraListeners = Nil, + branchNaming = None, + returnToStartBranch = false, + progressStore = Some(store), + claude = Some(recorder), + codex = None, + opencode = None, + opencodeLauncher = OpencodeLauncher.default, + pi = None, + gemini = None, + git = None, + gh = None, + fs = None, + prompts = DefaultPrompts + ): + // The body observes the already-rehydrated mapping. + assertEquals( + recorder.registered, + List(("client-uuid", "ses_server_1")), + "registerResumeWireId must be called once with the persisted mapping" + ) + + /** Drive `runFlow` directly (exit-free) with a null-sink interaction so no + * TTY is needed and a body failure surfaces as a thrown exception rather + * than a `System.exit`. + */ + private def runFlowForTest( + workDir: os.Path, + prompt: String, + store: ProgressStore + )(body: orca.FlowControl ?=> Unit): Unit = + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + runFlow( + args = OrcaArgs(prompt), + agent = _ => StubAgent.claude, + workDir = workDir, + interaction = Some(interaction), + extraListeners = Nil, + branchNaming = None, + returnToStartBranch = false, + progressStore = Some(store), + claude = None, + codex = None, + opencode = None, + opencodeLauncher = OpencodeLauncher.default, + pi = None, + gemini = None, + git = None, + gh = None, + fs = None, + prompts = DefaultPrompts + )(body) + + test( + "R5: success teardown auto-deletes feature branch when only orca commits exist" + ): + // A flow whose body does nothing besides getting staged (only the orca + // progress header + removal commits are on the feature branch). On success, + // the branch should be gone. + val workDir = TempRepo.create() + val prompt = "throwaway-flow" + val git = new OsGitTool(workDir) + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + flow( + args = OrcaArgs(prompt), + agent = _ => StubAgent.claude, + workDir = workDir, + interaction = Some(interaction) + ): + // body does nothing — no code changes + summon[orca.FlowContext].emit(OrcaEvent.Step("no-op")) + // Back on main. + assertEquals(git.currentBranch(), "main") + // The feature branch must be gone (auto-deleted as throwaway). + // Verify by checking git branch list: no branch other than main exists. + val branches = os + .proc("git", "branch", "--format=%(refname:short)") + .call(cwd = workDir) + .out + .text() + .linesIterator + .map(_.trim) + .filter(_.nonEmpty) + .toSet + assertEquals(branches, Set("main"), s"expected only main, got: $branches") + + test( + "success teardown (default): stays on the feature branch when code landed" + ): + val workDir = TempRepo.create() + val prompt = "code-flow" + val git = new OsGitTool(workDir) + var featureBranchName = "" + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + flow( + args = OrcaArgs(prompt), + agent = _ => StubAgent.claude, + workDir = workDir, + interaction = Some(interaction) + ): + // Record the feature branch name before it commits (during stage body). + featureBranchName = summon[orca.FlowContext].git.currentBranch() + val _ = stage("write code"): + os.write(workDir / "code.txt", "real code") + "done" + // Default behaviour: stay on the feature branch (the user ends on the work). + assertEquals(git.currentBranch(), featureBranchName) + assert(featureBranchName.nonEmpty, "must have captured feature branch name") + val branches = os + .proc("git", "branch", "--format=%(refname:short)") + .call(cwd = workDir) + .out + .text() + .linesIterator + .map(_.trim) + .filter(_.nonEmpty) + .toSet + assert( + branches.contains(featureBranchName), + s"feature branch '$featureBranchName' must be kept; branches: $branches" + ) + + test( + "success teardown with returnToStartBranch=true returns to start, keeps branch" + ): + val workDir = TempRepo.create() + val prompt = "code-flow-return" + val git = new OsGitTool(workDir) + var featureBranchName = "" + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + flow( + args = OrcaArgs(prompt), + agent = _ => StubAgent.claude, + workDir = workDir, + interaction = Some(interaction), + returnToStartBranch = true + ): + featureBranchName = summon[orca.FlowContext].git.currentBranch() + val _ = stage("write code"): + os.write(workDir / "code.txt", "real code") + "done" + // PR-flow behaviour: HEAD returns to the starting branch… + assertEquals(git.currentBranch(), "main") + // …but the feature branch is kept (it holds the work / backs the PR). + val branches = os + .proc("git", "branch", "--format=%(refname:short)") + .call(cwd = workDir) + .out + .text() + .linesIterator + .map(_.trim) + .filter(_.nonEmpty) + .toSet + assert( + branches.contains(featureBranchName), + s"feature branch '$featureBranchName' must be kept; branches: $branches" + ) + + test("R5: failure teardown keeps feature branch regardless of code changes"): + // A flow that crashes must NOT delete the branch — it needs to stay for resume. + val workDir = TempRepo.create() + val prompt = "failure-keeps-branch" + val store = ProgressStore.default(workDir, prompt) + val git = new OsGitTool(workDir) + var featureBranchName = "" + val _ = intercept[RuntimeException]: + runFlowForTest(workDir, prompt, store): + // Capture the feature branch name before the crash. + featureBranchName = summon[orca.FlowControl].git.currentBranch() + val _ = stage[String]("crash"): + throw new RuntimeException("boom") + // On the feature branch, not main. + assertNotEquals(git.currentBranch(), "main") + assert(featureBranchName.nonEmpty, "must have captured feature branch name") + // Feature branch still exists (not deleted). + val branches = os + .proc("git", "branch", "--format=%(refname:short)") + .call(cwd = workDir) + .out + .text() + .linesIterator + .map(_.trim) + .filter(_.nonEmpty) + .toSet + assert( + branches.contains(featureBranchName), + s"feature branch '$featureBranchName' must survive failure: $branches" + ) + + test( + "default branchNaming (None) resolves via shortenPrompt: branch name equals slug(prompt)" + ): + // When `branchNaming = None` (the default), `flowSetup` uses + // `BranchNamingStrategy.shortenPrompt`. With `StubAgent.claude`, `cheap` + // returns `this` (haiku = this) and `autonomous` throws + // `UnsupportedOperationException`; `shortenPrompt` catches the failure and + // falls back to `slug(userPrompt)`. This pins that the default is + // `shortenPrompt`, not the old `fromText`. + val workDir = TempRepo.create() + val prompt = "default-naming" + val expectedBranch = BranchNamingStrategy.slug(prompt) + var observedBranch = "" + supervised: + val interaction = TerminalInteraction.start( + out = new PrintStream(new ByteArrayOutputStream()), + useColor = false, + animated = false + ) + // branchNaming defaults to None — do not pass it. + flow( + args = OrcaArgs(prompt), + agent = _ => StubAgent.claude, + workDir = workDir, + interaction = Some(interaction) + ): + observedBranch = summon[orca.FlowContext].git.currentBranch() + assertEquals( + observedBranch, + expectedBranch, + s"default branchNaming must use shortenPrompt (slug fallback); got '$observedBranch'" + ) + + /** A `ClaudeAgent` that records `registerResumeWireId` calls, to assert the + * lifecycle rehydrates the persisted resume-wire-id map. All LLM methods + * throw — the rehydration test never invokes the model. + */ + private class RecordingClaude extends ClaudeAgent: + private var _registered: List[(String, String)] = Nil + def registered: List[(String, String)] = _registered + + override def registerResumeWireId( + client: SessionId[BackendTag.ClaudeCode.type], + wireId: SessionId[BackendTag.ClaudeCode.type] + ): Unit = + _registered = _registered :+ (client.value -> wireId.value) + + val name = "recording-claude" + def haiku = this + def sonnet = this + def opus = this + def fable = this + def withModel(model: Model) = this + def withNetworkTools(t: Seq[String]) = this + def withConfig(c: AgentConfig) = this + def withSystemPrompt(p: String) = this + def withName(n: String) = this + def withTools(tools: ToolSet) = this + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + throw new UnsupportedOperationException + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + throw new UnsupportedOperationException + +end FlowLifecycleTest diff --git a/runner/src/test/scala/orca/runner/OpencodeFlowTest.scala b/runner/src/test/scala/orca/runner/OpencodeFlowTest.scala index f482b40c..7dc5c424 100644 --- a/runner/src/test/scala/orca/runner/OpencodeFlowTest.scala +++ b/runner/src/test/scala/orca/runner/OpencodeFlowTest.scala @@ -1,41 +1,46 @@ package orca.runner import orca.{FlowContext, OrcaArgs, flow} -import orca.backend.{Conversation, Interaction, LlmBackend, LlmResult} +import orca.backend.{Conversation, Interaction, AgentBackend, AgentResult} import orca.events.{OrcaListener, Usage} -import orca.llm.{ +import orca.agents.{ AgentInput, Announce, - AutonomousLlmCall, + AutonomousAgentCall, AutonomousTextCall, BackendTag, DefaultPrompts, - InteractiveLlmCall, + InteractiveAgentCall, JsonData, - LlmCall, - LlmConfig, - OpencodeTool, + AgentCall, + AgentConfig, + OpencodeAgent, SessionId, ToolSet } import orca.plan.{Plan, Task, Title} -import orca.tools.opencode.DefaultOpencodeTool +import orca.tools.opencode.DefaultOpencodeAgent import _root_.orca.runner.terminal.TerminalInteraction import ox.supervised import java.io.{ByteArrayOutputStream, PrintStream} /** End-to-end flow coverage for the OpenCode tool without a live server: - * 1. the backend-agnostic Plan DSL runs through a wired `OpencodeTool`, and + * 1. the backend-agnostic Plan DSL runs through a wired `OpencodeAgent`, and * 2. a structured `resultAs[O]` call parses the backend's output via the - * real `DefaultLlmCall` (not a short-circuiting stub). + * real `DefaultAgentCall` (not a short-circuiting stub). */ class OpencodeFlowTest extends munit.FunSuite: + // These tests drive gated LLM calls directly in the flow body (not inside a + // `stage`), so mint the in-stage token for the suite (package `orca.runner`). + private given orca.InStage = orca.InStage.unsafe + private val samplePlan = Plan( epicId = "x", description = "d", - tasks = List(Task(Title("t1"), "body")) + tasks = List(Task(Title("t1"), "body")), + brief = "the brief" ) test( @@ -48,9 +53,12 @@ class OpencodeFlowTest extends munit.FunSuite: useColor = false, animated = false ) + val canned = new CannedOpencode(samplePlan) flow( args = OrcaArgs(), - opencode = Some(new CannedOpencode(samplePlan)), + agent = _.opencode, + workDir = TempRepo.create(), + opencode = Some(canned), interaction = Some(interaction) ): observed = Some( @@ -60,10 +68,10 @@ class OpencodeFlowTest extends munit.FunSuite: ) assertEquals(observed, Some(samplePlan)) - test("resultAs[O] parses the backend output through DefaultOpencodeTool"): - val tool = new DefaultOpencodeTool( + test("resultAs[O] parses the backend output through DefaultOpencodeAgent"): + val tool = new DefaultOpencodeAgent( new CannedBackend("""{"decision":"go","score":7}"""), - LlmConfig.default, + AgentConfig.default, DefaultPrompts, os.temp.dir(), OrcaListener.noop, @@ -77,66 +85,68 @@ class OpencodeFlowTest extends munit.FunSuite: private case class Verdict(decision: String, score: Int) derives JsonData /** Returns a fixed JSON string as the autonomous output; the tool's - * `DefaultLlmCall` does the real parsing. + * `DefaultAgentCall` does the real parsing. */ private class CannedBackend(json: String) - extends LlmBackend[BackendTag.Opencode.type]: + extends AgentBackend[BackendTag.Opencode.type]: def runAutonomous( prompt: String, session: SessionId[BackendTag.Opencode.type], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener, outputSchema: Option[String] - ): LlmResult[BackendTag.Opencode.type] = - LlmResult(session, json, Usage(0L, 0L, None)) + ): AgentResult[BackendTag.Opencode.type] = + AgentResult(session, json, Usage(0L, 0L, None)) def runInteractive( prompt: String, session: SessionId[BackendTag.Opencode.type], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[BackendTag.Opencode.type] = + )(using ox.Ox): Conversation[BackendTag.Opencode.type] = throw new UnsupportedOperationException private val noInteraction: Interaction = new Interaction: def listeners: List[OrcaListener] = Nil def drive[B <: BackendTag]( conversation: Conversation[B] - ): LlmResult[B] = throw new UnsupportedOperationException + ): AgentResult[B] = throw new UnsupportedOperationException /** OpenCode-typed canned tool whose `resultAs[O]` hands back `value` directly - * (bypassing parsing) — proves the generic Plan DSL accepts an OpencodeTool. + * (bypassing parsing) — proves the generic Plan DSL accepts an + * OpencodeAgent. */ - private class CannedOpencode[T](value: T) extends OpencodeTool: + private class CannedOpencode[T](value: T) extends OpencodeAgent: val name: String = "canned" - def anthropicOpus: OpencodeTool = this - def anthropicSonnet: OpencodeTool = this - def anthropicHaiku: OpencodeTool = this - def openaiGpt5: OpencodeTool = this - def openaiGpt5Codex: OpencodeTool = this - def openaiGpt5Mini: OpencodeTool = this - def withModel(providerModel: String): OpencodeTool = this - def withConfig(c: LlmConfig): OpencodeTool = this - def withSystemPrompt(p: String): OpencodeTool = this - def withName(n: String): OpencodeTool = this - def withTools(tools: ToolSet): OpencodeTool = this + def anthropicOpus: OpencodeAgent = this + def anthropicSonnet: OpencodeAgent = this + def anthropicHaiku: OpencodeAgent = this + def openaiGpt5: OpencodeAgent = this + def openaiGpt5Codex: OpencodeAgent = this + def openaiGpt5Mini: OpencodeAgent = this + def withModel(providerModel: String): OpencodeAgent = this + def withConfig(c: AgentConfig): OpencodeAgent = this + def withSystemPrompt(p: String): OpencodeAgent = this + def withName(n: String): OpencodeAgent = this + def withTools(tools: ToolSet): OpencodeAgent = this def autonomous: AutonomousTextCall[BackendTag.Opencode.type] = throw new UnsupportedOperationException - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.Opencode.type, O] = - new LlmCall[BackendTag.Opencode.type, O]: - val autonomous: AutonomousLlmCall[BackendTag.Opencode.type, O] = - new AutonomousLlmCall[BackendTag.Opencode.type, O]: + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.Opencode.type, O] = + new AgentCall[BackendTag.Opencode.type, O]: + val autonomous: AutonomousAgentCall[BackendTag.Opencode.type, O] = + new AutonomousAgentCall[BackendTag.Opencode.type, O]: def run[I: AgentInput]( input: I, session: SessionId[BackendTag.Opencode.type], - config: LlmConfig, + config: AgentConfig, emitPrompt: Boolean - ): (SessionId[BackendTag.Opencode.type], O) = + )(using orca.InStage): (SessionId[BackendTag.Opencode.type], O) = ( SessionId[BackendTag.Opencode.type]("stub-sid"), value.asInstanceOf[O] ) - def interactive: InteractiveLlmCall[BackendTag.Opencode.type, O] = + def interactive: InteractiveAgentCall[BackendTag.Opencode.type, O] = throw new UnsupportedOperationException diff --git a/runner/src/test/scala/orca/runner/OrcaLogTest.scala b/runner/src/test/scala/orca/runner/OrcaLogTest.scala index d0dd3494..61a0e4e7 100644 --- a/runner/src/test/scala/orca/runner/OrcaLogTest.scala +++ b/runner/src/test/scala/orca/runner/OrcaLogTest.scala @@ -12,7 +12,8 @@ class OrcaLogTest extends munit.FunSuite: orcaLog.finish() - val file = orcaLog.file.getOrElse(fail("trace file should have been created")) + val file = + orcaLog.file.getOrElse(fail("trace file should have been created")) assert(os.exists(file), "trace file should exist") val onDisk = os.read(file) assert(onDisk.contains("trace-marker-abc"), onDisk) diff --git a/runner/src/test/scala/orca/runner/OrcaOverridesTest.scala b/runner/src/test/scala/orca/runner/OrcaOverridesTest.scala index 9a9e93cb..8b4b8ceb 100644 --- a/runner/src/test/scala/orca/runner/OrcaOverridesTest.scala +++ b/runner/src/test/scala/orca/runner/OrcaOverridesTest.scala @@ -2,17 +2,18 @@ package orca.runner import orca.{FlowContext, OrcaArgs, flow, fs, pi} import orca.tools.{FsTool} -import orca.llm.{ +import orca.agents.{ Announce, AutonomousTextCall, BackendTag, - ClaudeTool, + ClaudeAgent, JsonData, - PiTool, - LlmCall, - LlmConfig, + Agent, + PiAgent, + AgentCall, + AgentConfig, Model, - OpencodeTool, + OpencodeAgent, SessionId, ToolSet } @@ -24,10 +25,19 @@ import java.io.{ByteArrayOutputStream, PrintStream} class OrcaOverridesTest extends munit.FunSuite: + // These tests drive gated LLM calls directly in the flow body (not inside a + // `stage`), so mint the in-stage token for the suite (package `orca.runner`). + private given orca.InStage = orca.InStage.unsafe + + // The leading-model selector defaults to `_.claude` (ADR 0018 §2.5); these + // tests assert tool-override wiring, not LLM behaviour, so they resolve a stub + // via a `_ => StubAgent.claude` selector. + private val stubLead: FlowContext => ClaudeAgent = _ => StubAgent.claude + test("flow uses a custom FsTool when supplied"): val fake = new FsTool: def read(path: String): Option[String] = Some("canned content") - def write(path: String, content: String): Unit = () + def write(path: String, content: String)(using orca.InStage): Unit = () def list(glob: String): List[String] = List("custom") var observed: Option[String] = None supervised: @@ -36,19 +46,26 @@ class OrcaOverridesTest extends munit.FunSuite: useColor = false, animated = false ) - flow(args = OrcaArgs(), fs = Some(fake), interaction = Some(interaction)): + flow( + args = OrcaArgs(), + agent = stubLead, + workDir = TempRepo.create(), + fs = Some(fake), + interaction = Some(interaction) + ): observed = fs.read("ignored") assertEquals(observed, Some("canned content")) - test("flow uses a custom ClaudeTool when supplied"): - val fakeClaude = new ClaudeTool: + test("flow uses a custom ClaudeAgent when supplied"): + val fakeClaude = new ClaudeAgent: val name = "fake" def haiku = this def sonnet = this def opus = this def fable = this + def withModel(model: Model) = this def withNetworkTools(t: Seq[String]) = this - def withConfig(c: LlmConfig) = this + def withConfig(c: AgentConfig) = this def withSystemPrompt(p: String) = this def withName(n: String) = this def withTools(tools: ToolSet) = this @@ -57,12 +74,14 @@ class OrcaOverridesTest extends munit.FunSuite: def run( p: String, session: SessionId[BackendTag.ClaudeCode.type], - c: LlmConfig, + c: AgentConfig, emitPrompt: Boolean + )(using + orca.InStage ): (SessionId[BackendTag.ClaudeCode.type], String) = (SessionId[BackendTag.ClaudeCode.type]("fake-sid"), s"echo: $p") def resultAs[O: JsonData: Announce] - : LlmCall[BackendTag.ClaudeCode.type, O] = + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? var observed: String = "" supervised: @@ -73,14 +92,16 @@ class OrcaOverridesTest extends munit.FunSuite: ) flow( args = OrcaArgs(), + agent = _ => fakeClaude, + workDir = TempRepo.create(), claude = Some(fakeClaude), interaction = Some(interaction) ): observed = summon[FlowContext].claude.autonomous.run("hi")._2 assertEquals(observed, "echo: hi") - test("flow uses a custom OpencodeTool when supplied"): - val fakeOpencode = new OpencodeTool: + test("flow uses a custom OpencodeAgent when supplied"): + val fakeOpencode = new OpencodeAgent: val name = "fake" def anthropicOpus = this def anthropicSonnet = this @@ -89,7 +110,7 @@ class OrcaOverridesTest extends munit.FunSuite: def openaiGpt5Codex = this def openaiGpt5Mini = this def withModel(providerModel: String) = this - def withConfig(c: LlmConfig) = this + def withConfig(c: AgentConfig) = this def withSystemPrompt(p: String) = this def withName(n: String) = this def withTools(tools: ToolSet) = this @@ -98,12 +119,12 @@ class OrcaOverridesTest extends munit.FunSuite: def run( p: String, session: SessionId[BackendTag.Opencode.type], - c: LlmConfig, + c: AgentConfig, emitPrompt: Boolean - ): (SessionId[BackendTag.Opencode.type], String) = + )(using orca.InStage): (SessionId[BackendTag.Opencode.type], String) = (SessionId[BackendTag.Opencode.type]("fake-sid"), s"opencode: $p") def resultAs[O: JsonData: Announce] - : LlmCall[BackendTag.Opencode.type, O] = ??? + : AgentCall[BackendTag.Opencode.type, O] = ??? var observed: String = "" supervised: val interaction = TerminalInteraction.start( @@ -113,16 +134,19 @@ class OrcaOverridesTest extends munit.FunSuite: ) flow( args = OrcaArgs(), + agent = stubLead, + workDir = TempRepo.create(), opencode = Some(fakeOpencode), interaction = Some(interaction) ): observed = summon[FlowContext].opencode.autonomous.run("hi")._2 assertEquals(observed, "opencode: hi") - test("flow uses a custom PiTool when supplied"): - val fakePi = new PiTool: + test("flow uses a custom PiAgent when supplied"): + val fakePi = new PiAgent: val name = "fake-pi" - def withConfig(c: LlmConfig) = this + def withModel(model: Model) = this + def withConfig(c: AgentConfig) = this def withSystemPrompt(p: String) = this def withName(n: String) = this def withTools(tools: ToolSet) = this @@ -131,11 +155,11 @@ class OrcaOverridesTest extends munit.FunSuite: def run( p: String, session: SessionId[BackendTag.Pi.type], - c: LlmConfig, + c: AgentConfig, emitPrompt: Boolean - ): (SessionId[BackendTag.Pi.type], String) = + )(using orca.InStage): (SessionId[BackendTag.Pi.type], String) = (SessionId[BackendTag.Pi.type]("fake-pi-sid"), s"pi: $p") - def resultAs[O: JsonData: Announce]: LlmCall[BackendTag.Pi.type, O] = + def resultAs[O: JsonData: Announce]: AgentCall[BackendTag.Pi.type, O] = ??? var observed: String = "" supervised: @@ -146,6 +170,8 @@ class OrcaOverridesTest extends munit.FunSuite: ) flow( args = OrcaArgs(), + agent = stubLead, + workDir = TempRepo.create(), pi = Some(fakePi), interaction = Some(interaction) ): @@ -163,6 +189,8 @@ class OrcaOverridesTest extends munit.FunSuite: ) flow( args = OrcaArgs(), + agent = stubLead, + workDir = TempRepo.create(), interaction = Some(interaction), extraListeners = List(tracker) ): diff --git a/runner/src/test/scala/orca/runner/OrcaTest.scala b/runner/src/test/scala/orca/runner/OrcaTest.scala index c118194f..32add715 100644 --- a/runner/src/test/scala/orca/runner/OrcaTest.scala +++ b/runner/src/test/scala/orca/runner/OrcaTest.scala @@ -22,7 +22,12 @@ class OrcaTest extends munit.FunSuite: useColor = false, animated = false ) - flow(args = OrcaArgs("hello world"), interaction = Some(interaction)): + flow( + args = OrcaArgs("hello world"), + agent = _ => StubAgent.claude, + workDir = TempRepo.create(), + interaction = Some(interaction) + ): seen = userPrompt assertEquals(seen, "hello world") @@ -34,7 +39,12 @@ class OrcaTest extends munit.FunSuite: useColor = false, animated = false ) - flow(args = OrcaArgs(), interaction = Some(interaction)): + flow( + args = OrcaArgs(), + agent = _ => StubAgent.claude, + workDir = TempRepo.create(), + interaction = Some(interaction) + ): summon[FlowContext].emit(OrcaEvent.StageStarted("plan")) // By the time the outer supervised exits, the interaction's worker has // drained — `flow`'s finally closes the channel, and the supervised diff --git a/runner/src/test/scala/orca/runner/StubAgent.scala b/runner/src/test/scala/orca/runner/StubAgent.scala new file mode 100644 index 00000000..fd5d77a7 --- /dev/null +++ b/runner/src/test/scala/orca/runner/StubAgent.scala @@ -0,0 +1,36 @@ +package orca.runner + +import orca.agents.{ + Announce, + AutonomousTextCall, + BackendTag, + ClaudeAgent, + JsonData, + AgentCall, + AgentConfig, + Model, + ToolSet +} + +/** A `ClaudeAgent` stub for tests that must pass the now-mandatory leading + * model to `flow(...)` (ADR 0018 §2.5) but assert wiring/lifecycle, not LLM + * behaviour. Every call throws — no test reaches one. + */ +object StubAgent: + val claude: ClaudeAgent = new ClaudeAgent: + val name = "stub" + def haiku = this + def sonnet = this + def opus = this + def fable = this + def withModel(model: Model) = this + def withNetworkTools(t: Seq[String]) = this + def withConfig(c: AgentConfig) = this + def withSystemPrompt(p: String) = this + def withName(n: String) = this + def withTools(tools: ToolSet) = this + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = + throw new UnsupportedOperationException + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + throw new UnsupportedOperationException diff --git a/runner/src/test/scala/orca/runner/TempRepo.scala b/runner/src/test/scala/orca/runner/TempRepo.scala new file mode 100644 index 00000000..b885cce9 --- /dev/null +++ b/runner/src/test/scala/orca/runner/TempRepo.scala @@ -0,0 +1,11 @@ +package orca.runner + +import orca.testkit.GitRepo + +/** Test helper: a fresh temp git repo with one seed commit, so `flow(...)` — + * which now stashes, branches, and commits a progress log as part of its + * lifecycle (ADR 0018 §2.5) — runs in isolation instead of mutating the + * developer's checkout. + */ +object TempRepo: + def create(): os.Path = GitRepo.seeded() diff --git a/runner/src/test/scala/orca/runner/terminal/ConversationRendererTest.scala b/runner/src/test/scala/orca/runner/terminal/ConversationRendererTest.scala index 245c6a0f..6f801d07 100644 --- a/runner/src/test/scala/orca/runner/terminal/ConversationRendererTest.scala +++ b/runner/src/test/scala/orca/runner/terminal/ConversationRendererTest.scala @@ -1,13 +1,13 @@ package orca.runner.terminal -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} import orca.events.{Usage} import orca.{OrcaInteractiveCancelled} import orca.backend.{ ApprovalDecision, Conversation, ConversationEvent, - LlmResult + AgentResult } import java.io.{ByteArrayOutputStream, PrintStream} @@ -48,17 +48,16 @@ class ConversationRendererTest extends munit.FunSuite: */ private class ScriptedConversation[B <: BackendTag]( scripted: List[ConversationEvent], - outcome: Either[Throwable, LlmResult[B]], + outcome: Either[Throwable, AgentResult[B]], val outputSchema: Option[String] = None ) extends Conversation[B]: val cancelled = new AtomicReference[Boolean](false) def events: Iterator[ConversationEvent] = scripted.iterator - def awaitResult(): Either[OrcaInteractiveCancelled, LlmResult[B]] = + def awaitResult(): Either[OrcaInteractiveCancelled, AgentResult[B]] = outcome match case Right(r) => Right(r) case Left(c: OrcaInteractiveCancelled) => Left(c) case Left(t) => throw t - def sendUserMessage(text: String): Unit = () def canAskUser: Boolean = false def cancel(): Unit = cancelled.set(true) @@ -75,8 +74,8 @@ class ConversationRendererTest extends munit.FunSuite: next.getOrElse(throw new IllegalStateException("prompter exhausted")) def close(): Unit = () - private def sampleResult: LlmResult[BackendTag.ClaudeCode.type] = - LlmResult( + private def sampleResult: AgentResult[BackendTag.ClaudeCode.type] = + AgentResult( sessionId = SessionId[BackendTag.ClaudeCode.type]("sid"), output = """{"ok":true}""", usage = Usage(0L, 0L, None) diff --git a/runner/src/test/scala/orca/runner/terminal/TerminalEventListenerTest.scala b/runner/src/test/scala/orca/runner/terminal/TerminalEventListenerTest.scala index c71fa52e..272dda4b 100644 --- a/runner/src/test/scala/orca/runner/terminal/TerminalEventListenerTest.scala +++ b/runner/src/test/scala/orca/runner/terminal/TerminalEventListenerTest.scala @@ -1,7 +1,7 @@ package orca.runner.terminal import orca.events.{OrcaEvent, Usage} -import orca.llm.Model +import orca.agents.Model import java.io.{ByteArrayOutputStream, PrintStream} /** These tests exercise the listener's synchronous state-mutation behaviour by diff --git a/tools/src/main/resources/orca/llm/prompts/autonomous.md b/tools/src/main/resources/orca/agents/prompts/autonomous.md similarity index 100% rename from tools/src/main/resources/orca/llm/prompts/autonomous.md rename to tools/src/main/resources/orca/agents/prompts/autonomous.md diff --git a/tools/src/main/resources/orca/llm/prompts/interactive.md b/tools/src/main/resources/orca/agents/prompts/interactive.md similarity index 100% rename from tools/src/main/resources/orca/llm/prompts/interactive.md rename to tools/src/main/resources/orca/agents/prompts/interactive.md diff --git a/tools/src/main/resources/orca/llm/prompts/raw-json-rules.md b/tools/src/main/resources/orca/agents/prompts/raw-json-rules.md similarity index 100% rename from tools/src/main/resources/orca/llm/prompts/raw-json-rules.md rename to tools/src/main/resources/orca/agents/prompts/raw-json-rules.md diff --git a/tools/src/main/resources/orca/llm/prompts/retry.md b/tools/src/main/resources/orca/agents/prompts/retry.md similarity index 100% rename from tools/src/main/resources/orca/llm/prompts/retry.md rename to tools/src/main/resources/orca/agents/prompts/retry.md diff --git a/tools/src/main/scala/orca/InStage.scala b/tools/src/main/scala/orca/InStage.scala new file mode 100644 index 00000000..5d8d0851 --- /dev/null +++ b/tools/src/main/scala/orca/InStage.scala @@ -0,0 +1,36 @@ +package orca + +import scala.annotation.implicitNotFound + +/** In-stage mutation token. Opaque so that only runtime code (the `stage` + * implementation, which lives in package `orca`) can construct an instance. + * + * User code and tool wrappers receive an `InStage` as a `using` parameter but + * can never fabricate one themselves — the `private[orca]` constructor ensures + * this. This gates all stage-bound mutations so that they cannot accidentally + * be called outside a running stage (ADR 0018 §2.2). + * + * The representation is `Unit`; only the opaque boundary matters here. + * + * `@implicitNotFound` turns the missing-capability compile error into a + * user-facing instruction: a flow author never needs to know what `InStage` + * is, only that side-effecting calls belong inside a `stage(...)`. Without it + * the compiler reports a cryptic "No given instance of type orca.InStage" that + * names an internal type. (Scala 3 honours the annotation on an opaque type + * alias — verified on 3.8.3.) + * + * Note: `private[orca]` is the narrowest package-qualified scope available in + * Scala 3. Modules are not packages, so any code in the `orca` package across + * modules can technically call `unsafe` — this is an accepted guard-rail per + * ADR 0018 §5. The convention is that only the `stage` runtime does so. + */ +@implicitNotFound( + "side-effecting calls (git/file/GitHub writes, LLM runs) must be made inside a `stage(...)` body, which commits and checkpoints them. Move this call into a stage." +) +opaque type InStage = Unit + +object InStage: + /** Mint a fresh [[InStage]] token. Called exclusively by the `stage` runtime + * (package `orca`). Nothing outside the `orca` package can call this. + */ + private[orca] def unsafe: InStage = () diff --git a/tools/src/main/scala/orca/agents/Agent.scala b/tools/src/main/scala/orca/agents/Agent.scala new file mode 100644 index 00000000..1ed25feb --- /dev/null +++ b/tools/src/main/scala/orca/agents/Agent.scala @@ -0,0 +1,255 @@ +package orca.agents + +import orca.InStage + +import scala.util.control.NonFatal + +/** An LLM adapter usable from flow scripts — the handle you call from a + * `flow(...)` block (`claude`, `codex`, etc.). Two paths to invoke the model: + * + * - **`autonomous`** — free-form text, no structured output, no JSON schema + * wrapping. The agent's reply is returned verbatim. + * - **`resultAs[O]`** — fix the output type and obtain a call object that + * exposes both `autonomous` and `interactive` modes. + * + * Each mode has a single `run(input, session = …, config = …)` method that + * always returns `(SessionId[B], output)`. Pass a `SessionId[B]` across calls + * to keep one conversation alive — in a flow, obtain a resumable one with + * `agent.session(seed)` — or omit it to get a fresh one-shot session per call. + * + * The API never hides the autonomous-vs-interactive choice behind a default — + * it's always visible at the call site as the leftmost segment after the tool + * / call gateway. + * + * Parameterized by the concrete `BackendTag` so session ids and results carry + * the backend identity at the type level. + */ +trait Agent[B <: BackendTag]: + /** Label for this agent in the event stream (the `agent` axis of + * `OrcaEvent.TokensUsed`). Defaults to the backend name; set it with + * [[withName]] to distinguish roles in the cost report (e.g. "reviewer"). + */ + def name: String + + /** Free-form text autonomous calls. Use this when the agent's reply is prose + * / code / anything that doesn't need to parse as a structured `O`. For + * structured output (and the interactive-conversation path), use + * [[resultAs]]. + */ + def autonomous: AutonomousTextCall[B] + + /** Fix the output type of a structured call and obtain a gateway with both + * `autonomous` and `interactive` modes. `O` needs a `JsonData[O]` — `derives + * JsonData` on a case class is the normal way to provide one. + * + * An `Announce[O]` is also required; the library's default given returns + * `None` (no auto-announce), so callers don't need to do anything unless + * they want a friendly summary on the channel. See [[Announce]]. + */ + def resultAs[O: JsonData: Announce]: AgentCall[B, O] + + def withConfig(config: AgentConfig): Agent[B] + def withSystemPrompt(prompt: String): Agent[B] + def withName(name: String): Agent[B] + + /** Return a sibling tool whose config pins [[AgentConfig.tools]] to `tools` — + * the capability tier (see [[ToolSet]]). The primitive behind + * [[withReadOnly]] and [[withNetworkOnly]]; preserves the rest of the tool's + * config (model, system prompt, autoApprove). + */ + def withTools(tools: ToolSet): Agent[B] + + /** Sibling tool restricted to read-only tools ([[ToolSet.ReadOnly]]): no + * edits, no shell. Used by planning and review helpers so e.g. + * `claude.opus.withReadOnly` keeps the opus pin while gating writes. + */ + def withReadOnly: Agent[B] = withTools(ToolSet.ReadOnly) + + /** Sibling tool restricted to reads plus network ([[ToolSet.NetworkOnly]]) — + * for planner turns that must read an issue/PR. See [[ToolSet]] for the + * per-backend no-edit guarantee (hard on most; prompt-only on pi / codex). + */ + def withNetworkOnly: Agent[B] = withTools(ToolSet.NetworkOnly) + + /** A cheaper/faster variant of this model for incidental work (commit-message + * summaries, reviewer selection, prompt shortening). Returns the model + * pinned via [[withCheapModel]] if one was set, otherwise the backend's + * built-in cheap tier ([[defaultCheap]]). + */ + def cheap: Agent[B] = defaultCheap + + /** The backend's built-in cheap variant (claude→haiku, codex→mini, …), before + * any [[withCheapModel]] override; `this` when a backend has no cheaper + * tier. Backends override this — flow code calls [[cheap]]. + */ + protected def defaultCheap: Agent[B] = this + + /** Pin the model that [[cheap]] resolves to, overriding the backend default. + * Lets a flow specify both a leading and a cheap model, e.g. + * `_.opencode.anthropicSonnet.withCheapModel(Model("anthropic/claude-haiku-4-5"))`. + */ + def withCheapModel(model: Model): Agent[B] = this + + /** Best-effort one-line reply from the cheap model, for the runtime's own + * incidental text (branch naming, default commit messages). Runs `prompt` on + * `cheap.withReadOnly` (no prompt echo), returns the first non-blank line + * trimmed — or `fallback` if the reply is empty or the call fails for any + * non-fatal reason (markdown fence lines are skipped). Never throws: these + * calls must never break a flow. `private[orca]` — internal; flow scripts + * use `cheap.autonomous.run(...)` directly if they want a one-off cheap + * call. + */ + private[orca] def cheapOneShot(prompt: String, fallback: => String)(using + InStage + ): String = + try + val (_, text) = + cheap.withReadOnly.autonomous.run(prompt, emitPrompt = false) + val firstLine = text.linesIterator + .map(_.trim) + .filterNot(_.startsWith("```")) + .find(_.nonEmpty) + .getOrElse("") + if firstLine.isBlank then fallback else firstLine + catch case NonFatal(_) => fallback + + /** Return a sibling tool that manages git itself — flips + * [[AgentConfig.selfManagedGit]] on, suppressing the standing "runtime owns + * git" rule the runtime otherwise injects (don't `git commit`/`push`/branch; + * leave edits in the working tree). Use only for a flow that genuinely wants + * the agent to drive git; the default keeps git runtime-owned. + * + * Unlike [[withReadOnly]] this carries a no-op default (returns `this`), so + * a custom `Agent` that forgets to wire it simply keeps the safe + * runtime-owns-git behaviour rather than silently granting the escape hatch. + */ + def withSelfManagedGit: Agent[B] = this + + /** Best-effort, non-destructive: is a live, resumable backend conversation + * present for `session`? Delegates to the backend probe. Returns `false` by + * default — safe re-seed — when a concrete tool can't reach a backend + * instance (e.g. lightweight stubs). + */ + def sessionExists(session: SessionId[B]): Boolean = false + + /** The wire id to resume `client` against, or `None` if unknown (or the + * backend's sessions aren't durably resumable). Client and wire ids share + * one type (`SessionId[B]`): `client` is orca's stable handle; the result is + * whatever the backend actually resumes against — equal to `client` where + * the client id IS the wire id (claude), a learned server-thread id for + * codex/gemini/opencode, `None` for pi (ephemeral sessions). The flow + * runtime reads this after a run to persist the resume wire id into the + * progress log. Returns `None` by default for tools without a backend. + */ + def resumeWireId(client: SessionId[B]): Option[SessionId[B]] = None + + /** Record a resume wire id for `client` in the backend's registry. The flow + * runtime calls this on resume to rehydrate the map from the persisted log, + * so `dispatchFor` resumes against the right wire id and the probes target + * it. No-op by default for tools without a backend (stubs). + */ + def registerResumeWireId( + client: SessionId[B], + wireId: SessionId[B] + ): Unit = () + + /** Mint a fresh, unrecorded session id — used by the runtime for ephemeral + * one-off conversations (e.g. each reviewer's own turn). NOT resume-aware: + * it isn't recorded in the progress log, so a re-run mints a different id. + * Flow scripts that need a session to survive restarts use + * `agent.session(seed)` instead, which keys off the log. `private[orca]` — + * internal. + * + * Default implementation generates a UUID via [[SessionId.fresh]]; backends + * that need a different format (or eager server-side allocation) override. + */ + private[orca] def newSession: SessionId[B] = SessionId.fresh[B] + +/** Bare `claude` runs Opus with the 1M-token context window (the long-lived + * implementer); the accessors below pin a specific tier, e.g. + * `claude.haiku.autonomous.run("summarize this")._2` for a cheap fast + * one-shot. + */ +trait ClaudeAgent extends Agent[BackendTag.ClaudeCode.type]: + /** Pin the Claude model for subsequent calls, overriding `AgentConfig.model`. + */ + def haiku: ClaudeAgent + def sonnet: ClaudeAgent + def opus: ClaudeAgent + def fable: ClaudeAgent + + /** Pin any Claude model id beyond the named tiers, e.g. + * `claude.withModel(Model("claude-opus-4-1-some-snapshot"))`. + */ + def withModel(model: Model): ClaudeAgent + + override protected def defaultCheap: ClaudeAgent = haiku + + /** Set the read-only network allowlist used on [[ToolSet.NetworkOnly]] turns + * (claude `--allowedTools` syntax, e.g. `Bash(gh api:*)`, `WebFetch`). + * Claude-specific, so it's here rather than on `AgentConfig`; defaults to + * `ClaudeBackend.DefaultNetworkTools`. Pass it before handing the tool to a + * planning helper: `claude.opus.withNetworkTools(Seq("WebFetch"))`. + */ + def withNetworkTools(tools: Seq[String]): ClaudeAgent + +/** Bare `codex` runs the installed `codex-cli`'s default model; `codex.mini` + * opts down to the cheap tier, and `codex.withModel(Model("..."))` pins any + * other id the CLI offers. + */ +trait CodexAgent extends Agent[BackendTag.Codex.type]: + def mini: CodexAgent + + /** Pin any codex model the installed `codex-cli` offers, beyond `mini` — e.g. + * `codex.withModel(Model("gpt-5.4-pro"))`. + */ + def withModel(model: Model): CodexAgent + + override protected def defaultCheap: CodexAgent = mini + +/** OpenCode spans providers, so its model accessors are provider-prefixed (the + * prefix keeps the vendor explicit at the call site). [[withModel]] takes any + * `provider/model` id — including self-hosted ones, e.g. `ollama/llama3.1`. + */ +trait OpencodeAgent extends Agent[BackendTag.Opencode.type]: + def anthropicOpus: OpencodeAgent + def anthropicSonnet: OpencodeAgent + def anthropicHaiku: OpencodeAgent + def openaiGpt5: OpencodeAgent + def openaiGpt5Codex: OpencodeAgent + def openaiGpt5Mini: OpencodeAgent + + /** Base cheap variant is anthropic haiku; [[DefaultOpencodeAgent]] overrides + * this to match the leading provider, so incidental work on an openai-led + * tool doesn't pull in a second provider's auth. + */ + override protected def defaultCheap: OpencodeAgent = anthropicHaiku + + /** Pin any `provider/model` id (e.g. `ollama/llama3.1`, `myhost/qwen-coder`). + */ + def withModel(providerModel: String): OpencodeAgent + + /** Two-arg form of [[withModel]], e.g. `withModel("ollama", "llama3.1")`. The + * default joins with `/`; the concrete tool validates the parts. + */ + def withModel(provider: String, modelId: String): OpencodeAgent = + withModel(s"$provider/$modelId") + +trait PiAgent extends Agent[BackendTag.Pi.type]: + /** Pin a pi model id; pi otherwise selects the model via its own CLI config. + */ + def withModel(model: Model): PiAgent + +trait GeminiAgent extends Agent[BackendTag.Gemini.type]: + /** Pin the cheap-and-fast Gemini Flash model for subsequent calls, overriding + * `AgentConfig.model`. Bare `gemini` runs on Gemini Pro (pinned in the + * runtime wiring); `gemini.flash` opts down for cheap one-shots. + */ + def flash: GeminiAgent + + /** Pin any Gemini model id beyond `flash`, e.g. + * `gemini.withModel(Model("gemini-2.5-pro"))`. + */ + def withModel(model: Model): GeminiAgent + + override protected def defaultCheap: GeminiAgent = flash diff --git a/tools/src/main/scala/orca/llm/LlmCall.scala b/tools/src/main/scala/orca/agents/AgentCall.scala similarity index 72% rename from tools/src/main/scala/orca/llm/LlmCall.scala rename to tools/src/main/scala/orca/agents/AgentCall.scala index 60251d02..5e4515bc 100644 --- a/tools/src/main/scala/orca/llm/LlmCall.scala +++ b/tools/src/main/scala/orca/agents/AgentCall.scala @@ -1,7 +1,7 @@ -package orca.llm +package orca.agents import orca.AgentTurnFailed -import orca.backend.{Interaction, LlmBackend} +import orca.backend.{Interaction, AgentBackend} import orca.events.{OrcaEvent, OrcaListener} import orca.util.JsonSchemaGen import ox.resilience.{ResultPolicy, RetryConfig, retry} @@ -10,16 +10,16 @@ import ox.resilience.{ResultPolicy, RetryConfig, retry} * autonomous-vs-interactive choice into two sibling objects so the call site * always shows which mode it picked. */ -trait LlmCall[B <: BackendTag, O]: - def autonomous: AutonomousLlmCall[B, O] - def interactive: InteractiveLlmCall[B, O] +trait AgentCall[B <: BackendTag, O]: + def autonomous: AutonomousAgentCall[B, O] + def interactive: InteractiveAgentCall[B, O] /** Autonomous structured calls — single agentic turn, no human in the loop. - * Single method: pass a [[SessionId]] (typically from [[LlmTool.newSession]] - * or the default fresh one); the library starts on the first call, resumes on - * subsequent calls. Returns the (stable) session id. + * Single method: pass a [[SessionId]] (typically from `agent.session(seed)` + * (in a flow) or the default fresh one); the library starts on the first call, + * resumes on subsequent calls. Returns the (stable) session id. */ -trait AutonomousLlmCall[B <: BackendTag, O]: +trait AutonomousAgentCall[B <: BackendTag, O]: /** Run the agent on `input`. When `emitPrompt` is true (the default), fires * an `OrcaEvent.UserPrompt` carrying the human-readable form of `input` (the * `AgentInput[I]` serialization) so listeners can surface what's being @@ -31,22 +31,44 @@ trait AutonomousLlmCall[B <: BackendTag, O]: def run[I: AgentInput]( input: I, session: SessionId[B] = SessionId.fresh[B], - config: LlmConfig = LlmConfig.default, + config: AgentConfig = AgentConfig.default, emitPrompt: Boolean = true - ): (SessionId[B], O) + )(using orca.InStage): (SessionId[B], O) /** Interactive structured calls — open a conversation the user can drive * (clarifying questions, refinements) before the agent produces the final * structured `O`. Same shape as the autonomous variant. */ -trait InteractiveLlmCall[B <: BackendTag, O]: +trait InteractiveAgentCall[B <: BackendTag, O]: def run[I: AgentInput]( input: I, session: SessionId[B] = SessionId.fresh[B], - config: LlmConfig = LlmConfig.default - ): (SessionId[B], O) + config: AgentConfig = AgentConfig.default + )(using orca.InStage): (SessionId[B], O) -/** Default implementation of [[LlmCall]] for any backend. +/** Free-form text autonomous calls — the `Agent.autonomous` shape (the + * non-structured sibling of [[AutonomousAgentCall]]). Single method: pass a + * [[SessionId]] (typically from `agent.session(seed)` (in a flow) or the + * default fresh one) and the library starts the session on the first call, + * resumes it on subsequent calls. Returns the (stable) session id so the + * caller can pass it back unchanged. + */ +trait AutonomousTextCall[B <: BackendTag]: + /** Run the agent on `prompt`. When `emitPrompt` is true (the default), fires + * an `OrcaEvent.UserPrompt` carrying `prompt` so listeners can surface + * what's being asked; framework-internal callers that produce many + * near-identical prompts in quick succession (e.g. the per-task reviewer + * fan-out) pass `false` to keep the event log focused. Other events + * (`ToolUse`, `AssistantMessage`, `TokensUsed`, etc.) fire regardless. + */ + def run( + prompt: String, + session: SessionId[B] = SessionId.fresh[B], + config: AgentConfig = AgentConfig.default, + emitPrompt: Boolean = true + )(using orca.InStage): (SessionId[B], String) + +/** Default implementation of [[AgentCall]] for any backend. * * The trait splits into `autonomous` and `interactive` sibling objects so the * call site shows which mode it picked. This class wires both: @@ -60,41 +82,42 @@ trait InteractiveLlmCall[B <: BackendTag, O]: * user steering. No retry: the user is steering, and a parse failure on * the final payload is more useful surfaced than silently relaunched. */ -class DefaultLlmCall[B <: BackendTag, O]( - backend: LlmBackend[B], - effectiveConfig: LlmConfig => LlmConfig, +class DefaultAgentCall[B <: BackendTag, O]( + backend: AgentBackend[B], + effectiveConfig: AgentConfig => AgentConfig, prompts: Prompts, workDir: os.Path, events: OrcaListener, interaction: Interaction, /** Used as the `agent` axis on `OrcaEvent.TokensUsed` — typically the - * owning `LlmTool.name`, which carries the reviewer identity for tools + * owning `Agent.name`, which carries the reviewer identity for tools * renamed via `withName`. The `model` axis is read from the response (or * the pinned config); this name is the always-present agent identifier. */ agentName: String )(using jd: JsonData[O], announce: Announce[O]) - extends LlmCall[B, O]: + extends AgentCall[B, O]: private given sttp.tapir.Schema[O] = jd.schema private given com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec[O] = jd.codec - val autonomous: AutonomousLlmCall[B, O] = new AutonomousLlmCall[B, O]: + val autonomous: AutonomousAgentCall[B, O] = new AutonomousAgentCall[B, O]: def run[I: AgentInput]( input: I, session: SessionId[B] = SessionId.fresh[B], - config: LlmConfig = LlmConfig.default, + config: AgentConfig = AgentConfig.default, emitPrompt: Boolean = true - ): (SessionId[B], O) = + )(using orca.InStage): (SessionId[B], O) = runAutonomousWithRetry(input, config, session, emitPrompt) - val interactive: InteractiveLlmCall[B, O] = new InteractiveLlmCall[B, O]: + val interactive: InteractiveAgentCall[B, O] = new InteractiveAgentCall[B, O]: def run[I: AgentInput]( input: I, session: SessionId[B] = SessionId.fresh[B], - config: LlmConfig = LlmConfig.default - ): (SessionId[B], O) = runInteractiveOnce(input, config, session) + config: AgentConfig = AgentConfig.default + )(using orca.InStage): (SessionId[B], O) = + runInteractiveOnce(input, config, session) /** Emit a `StructuredResult` event carrying the raw payload and the * `Announce[O]`-derived summary (if any). The terminal listener renders @@ -110,7 +133,7 @@ class DefaultLlmCall[B <: BackendTag, O]( */ private def runAutonomousWithRetry[I]( input: I, - config: LlmConfig, + config: AgentConfig, session: SessionId[B], emitPrompt: Boolean )(using ai: AgentInput[I]): (SessionId[B], O) = @@ -197,22 +220,29 @@ class DefaultLlmCall[B <: BackendTag, O]( */ private def runInteractiveOnce[I]( input: I, - config: LlmConfig, + config: AgentConfig, session: SessionId[B] )(using ai: AgentInput[I]): (SessionId[B], O) = val serialized = ai.serialize(input) val outputSchema = JsonSchemaGen[O] val prompt = prompts.interactive(serialized, outputSchema, config) val effective = effectiveConfig(config) - val conversation = backend.runInteractive( - prompt, - session, - displayPrompt = serialized, - effective, - workDir, - Some(outputSchema) - ) - val result = interaction.drive(conversation) + // Per-turn structured-concurrency scope: `runInteractive` forks its workers + // into this Ox, `drive` consumes them, and `cancel` (in the `finally`) tears + // the conversation down before the scope joins — so a cancelled turn never + // leaks the subprocess/forks. On cancel `drive` throws, skipping the + // registerSession / TokensUsed bookkeeping below, as before. + val result = ox.supervised: + val conversation = backend.runInteractive( + prompt, + session, + displayPrompt = serialized, + effective, + workDir, + Some(outputSchema) + )(using summon[ox.Ox]) + try interaction.drive(conversation) + finally conversation.cancel() // Codex mints its server thread id inside the drain (not at spawn); // surface it back to the backend so a follow-up call with the same // `session` can resume the right thread. No-op for backends whose diff --git a/tools/src/main/scala/orca/llm/LlmConfig.scala b/tools/src/main/scala/orca/agents/AgentConfig.scala similarity index 76% rename from tools/src/main/scala/orca/llm/LlmConfig.scala rename to tools/src/main/scala/orca/agents/AgentConfig.scala index db693e48..9b8f9406 100644 --- a/tools/src/main/scala/orca/llm/LlmConfig.scala +++ b/tools/src/main/scala/orca/agents/AgentConfig.scala @@ -1,11 +1,16 @@ -package orca.llm +package orca.agents import ox.scheduling.Schedule import scala.concurrent.duration.DurationInt -case class LlmConfig( +case class AgentConfig( model: Option[Model] = None, + /** Model used by [[orca.agents.Agent.cheap]] for incidental work (branch + * naming, commit-message summaries, reviewer selection). `None` uses the + * backend's built-in cheap tier; set it via `Agent.withCheapModel`. + */ + cheapModel: Option[Model] = None, systemPrompt: Option[String] = None, /** Which tools auto-approve without a permission prompt. Only meaningful * for **interactive** sessions — autonomous turns have no prompt to @@ -13,7 +18,7 @@ case class LlmConfig( * [[tools]] is [[ToolSet.Full]] (the read-only tiers are autonomous * planners/reviewers). `Only(set)` should list a subset of the tools * [[tools]] makes available; entries outside it are dead. Neither - * invariant is type-enforced (one `LlmConfig` feeds both the interactive + * invariant is type-enforced (one `AgentConfig` feeds both the interactive * and autonomous paths). */ autoApprove: AutoApprove = AutoApprove.All, @@ -34,7 +39,7 @@ case class LlmConfig( * agent to drive git. */ selfManagedGit: Boolean = false, - retrySchedule: Schedule = LlmConfig.defaultRetrySchedule + retrySchedule: Schedule = AgentConfig.defaultRetrySchedule ): /** Return a config whose `autoApprove` set also includes `tool`. Backends use * this to silently authorise their own host-side tools (e.g. the MCP @@ -42,35 +47,35 @@ case class LlmConfig( * refuse. No-op when `autoApprove = AutoApprove.All` — everything is already * covered. */ - def autoApproveAlso(tool: String): LlmConfig = + def autoApproveAlso(tool: String): AgentConfig = autoApprove match case AutoApprove.All => this case AutoApprove.Only(tools) => copy(autoApprove = AutoApprove.Only(tools + tool)) -object LlmConfig: +object AgentConfig: // Must be declared before `default` so the case-class default arg resolves. val defaultRetrySchedule: Schedule = Schedule.exponentialBackoff(1.second).maxRetries(3) - /** The default LlmConfig. Shared as a singleton so the framework can detect, - * via `eq LlmConfig.default`, that the caller omitted the per-call `config` - * argument; in that case the tool-level config (set via - * `LlmTool.withConfig`) is used instead. Any explicit `LlmConfig` passed at + /** The default AgentConfig. Shared as a singleton so the framework can + * detect, via `eq AgentConfig.default`, that the caller omitted the per-call + * `config` argument; in that case the tool-level config (set via + * `Agent.withConfig`) is used instead. Any explicit `AgentConfig` passed at * the call site wholly replaces the tool-level one — there is no per-field - * merge. Pass `LlmConfig.default` (or omit the arg) to inherit the tool's - * defaults; constructing a fresh `LlmConfig()` defeats the detection and + * merge. Pass `AgentConfig.default` (or omit the arg) to inherit the tool's + * defaults; constructing a fresh `AgentConfig()` defeats the detection and * wipes them. */ - val default: LlmConfig = LlmConfig() + val default: AgentConfig = AgentConfig() enum AutoApprove: case All case Only(tools: Set[String]) /** The set of tools available to the agent — the capability tier on - * [[LlmConfig.tools]]. Each backend maps the tiers onto its own permission + * [[AgentConfig.tools]]. Each backend maps the tiers onto its own permission * model: * * - **ReadOnly** — reads only; no shell, no edits. The hard no-edit gate @@ -86,7 +91,7 @@ enum AutoApprove: * workspace-write`), so there the no-edit guarantee is **prompt-only** — * the planner prompts forbid edits. * - **Full** — every tool, write-capable; prompting then follows - * [[LlmConfig.autoApprove]]. + * [[AgentConfig.autoApprove]]. */ enum ToolSet: case ReadOnly diff --git a/tools/src/main/scala/orca/llm/AgentInput.scala b/tools/src/main/scala/orca/agents/AgentInput.scala similarity index 89% rename from tools/src/main/scala/orca/llm/AgentInput.scala rename to tools/src/main/scala/orca/agents/AgentInput.scala index b9ac00c5..064e3b82 100644 --- a/tools/src/main/scala/orca/llm/AgentInput.scala +++ b/tools/src/main/scala/orca/agents/AgentInput.scala @@ -1,10 +1,10 @@ -package orca.llm +package orca.agents import com.github.plokhotnyuk.jsoniter_scala.core.writeToString /** Typeclass that serializes an arbitrary value into the string that gets * embedded in the prompt sent to the LLM. Every `run` method on - * `AutonomousLlmCall` / `InteractiveLlmCall` that accepts an `input: I` + * `AutonomousAgentCall` / `InteractiveAgentCall` that accepts an `input: I` * requires an `AgentInput[I]` so callers don't have to pre-stringify their * arguments. * diff --git a/tools/src/main/scala/orca/llm/Announce.scala b/tools/src/main/scala/orca/agents/Announce.scala similarity index 92% rename from tools/src/main/scala/orca/llm/Announce.scala rename to tools/src/main/scala/orca/agents/Announce.scala index 7a82472d..273ec150 100644 --- a/tools/src/main/scala/orca/llm/Announce.scala +++ b/tools/src/main/scala/orca/agents/Announce.scala @@ -1,7 +1,7 @@ -package orca.llm +package orca.agents /** A human-readable summary for a domain value. The library calls - * `message(parsed)` after `LlmCall.resultAs[O]` succeeds and emits a `Step` + * `message(parsed)` after `AgentCall.resultAs[O]` succeeds and emits a `Step` * event with the result; `None` means "nothing to say" and is dropped * silently. Provide a specific given in the type's companion to opt into a * friendlier rendering. diff --git a/tools/src/main/scala/orca/llm/BackendTag.scala b/tools/src/main/scala/orca/agents/BackendTag.scala similarity index 52% rename from tools/src/main/scala/orca/llm/BackendTag.scala rename to tools/src/main/scala/orca/agents/BackendTag.scala index d1b28e2b..401a1753 100644 --- a/tools/src/main/scala/orca/llm/BackendTag.scala +++ b/tools/src/main/scala/orca/agents/BackendTag.scala @@ -1,10 +1,10 @@ -package orca.llm +package orca.agents /** Type tag for a concrete LLM backend. Carried as the `B` parameter on - * [[SessionId]], [[orca.backend.LlmResult]], [[orca.backend.Conversation]], - * [[LlmTool]], and [[orca.backend.LlmBackend]] so a session id from one + * [[SessionId]], [[orca.backend.AgentResult]], [[orca.backend.Conversation]], + * [[Agent]], and [[orca.backend.AgentBackend]] so a session id from one * backend can't accidentally flow into another. Distinct from - * [[orca.backend.LlmBackend]], which is the runtime SPI; this enum is the + * [[orca.backend.AgentBackend]], which is the runtime SPI; this enum is the * compile-time discriminator. */ enum BackendTag: @@ -14,6 +14,18 @@ enum BackendTag: case Pi case Gemini +/** Returns `true` iff `id` is a well-formed session id that is safe to embed in + * a file path, regex, or URL without further escaping. Accepted: 1–200 + * characters from `[A-Za-z0-9_-]`, covering all legitimate ids (UUIDs for + * claude/codex/gemini; `ses_…` for opencode). Rejects `.`, `/`, `*`, `[`, `?`, + * `#`, `..` and every other character that could enable path traversal, regex + * injection, or URL injection. + * + * Every `sessionExists` override calls this before using the id. + */ +def isSafeSessionId(id: String): Boolean = + id.nonEmpty && id.length <= 200 && id.matches("[A-Za-z0-9_-]+") + opaque type SessionId[B <: BackendTag] = String object SessionId: @@ -40,3 +52,14 @@ object SessionId: object Untyped: def from[B <: BackendTag](sid: SessionId[B]): Untyped = sid extension (u: Untyped) def as[B <: BackendTag]: SessionId[B] = u + + /** `SessionId[B]` is an opaque alias over `String`; within this file the + * alias is transparent, so we can delegate to the `JsonData[String]` + * instance directly. Referencing it by its synthesized name avoids the + * infinite-loop the compiler detects when `summon[JsonData[String]]` sees + * this given as a candidate (since `SessionId[B] = String` inside the + * opaque-alias file). A session id is a plain JSON string on the wire — no + * wrapping, lossless round-trip. + */ + given [B <: BackendTag]: JsonData[SessionId[B]] = + JsonData.given_JsonData_String diff --git a/tools/src/main/scala/orca/llm/BaseLlmTool.scala b/tools/src/main/scala/orca/agents/BaseAgent.scala similarity index 52% rename from tools/src/main/scala/orca/llm/BaseLlmTool.scala rename to tools/src/main/scala/orca/agents/BaseAgent.scala index 2fe9cc97..be48fc16 100644 --- a/tools/src/main/scala/orca/llm/BaseLlmTool.scala +++ b/tools/src/main/scala/orca/agents/BaseAgent.scala @@ -1,31 +1,31 @@ -package orca.llm +package orca.agents -import orca.backend.{Interaction, LlmBackend, LlmResult} +import orca.backend.{Interaction, AgentBackend, AgentResult} import orca.events.{OrcaEvent, OrcaListener} /** Skeleton shared by Claude and Codex's default tools — and by any future - * backend that follows the same `LlmBackend` contract. Centralises the + * backend that follows the same `AgentBackend` contract. Centralises the * autonomous-text path (which is otherwise pure delegation to * `backend.runAutonomous` plus `TokensUsed` emission), the `resultAs[O]` * factory, and the `withConfig` / `withSystemPrompt` / `withName` builders. * * Concrete subclasses provide: - * - the `Self` type bound (their own `LlmTool` subtype) so the builders - * return the concrete type; + * - the `Self` type bound (their own `Agent` subtype) so the builders return + * the concrete type; * - a `copyTool` factory that knows the subclass-specific extra params (e.g. * claude has no extra params; a hypothetical backend that needed more * config would override `copyTool` to thread them through); * - the model accessors (`haiku`/`sonnet`/`opus`, `mini`, …) — these are * backend-specific and stay on the subclass. */ -abstract class BaseLlmTool[B <: BackendTag, Self <: LlmTool[B]]( - backend: LlmBackend[B], - config: LlmConfig, +abstract class BaseAgent[B <: BackendTag, Self <: Agent[B]]( + backend: AgentBackend[B], + config: AgentConfig, prompts: Prompts, workDir: os.Path, events: OrcaListener, interaction: Interaction -) extends LlmTool[B]: +) extends Agent[B]: /** Build a sibling instance with the supplied overrides. Concrete subclasses * call their own constructor with subclass-specific extra parameters @@ -33,11 +33,11 @@ abstract class BaseLlmTool[B <: BackendTag, Self <: LlmTool[B]]( * model-pinning accessors. */ protected def copyTool( - config: LlmConfig = config, + config: AgentConfig = config, name: String = name ): Self - def withConfig(newConfig: LlmConfig): Self = copyTool(config = newConfig) + def withConfig(newConfig: AgentConfig): Self = copyTool(config = newConfig) def withSystemPrompt(prompt: String): Self = copyTool(config = config.copy(systemPrompt = Some(prompt))) def withName(newName: String): Self = copyTool(name = newName) @@ -48,20 +48,55 @@ abstract class BaseLlmTool[B <: BackendTag, Self <: LlmTool[B]]( override def withSelfManagedGit: Self = copyTool(config = config.copy(selfManagedGit = true)) - /** Pin the underlying CLI's `--model` flag for subsequent calls. Subclasses - * expose backend-specific accessors (`haiku`/`sonnet`/`opus`, `mini`) on top - * of this. + /** Pin the underlying CLI's `--model` flag for subsequent calls. Public so + * each backend trait can surface it (`claude.withModel(...)`, + * `codex.withModel(...)`); the named accessors (`haiku`/`sonnet`/`opus`, + * `mini`) are conveniences over it. Returns `Self` so they keep the concrete + * type. */ - protected def withModel(model: Model): Self = + def withModel(model: Model): Self = copyTool(config = config.copy(model = Some(model))) + /** The cheap variant: a `withCheapModel` override if the caller pinned one, + * otherwise the backend's built-in [[defaultCheap]] tier. + */ + override def cheap: Agent[B] = + config.cheapModel.map(withModel).getOrElse(defaultCheap) + + override def withCheapModel(model: Model): Self = + copyTool(config = config.copy(cheapModel = Some(model))) + + /** Delegates to the backend's best-effort probe. Overrides the trait default + * so that any tool built on a real [[orca.backend.AgentBackend]] reflects + * actual session state rather than always returning `false`. + */ + override def sessionExists(session: SessionId[B]): Boolean = + backend.sessionExists(session) + + /** Delegates to the backend's registry read so the flow runtime can persist + * the resume wire id after a run. + */ + override def resumeWireId( + client: SessionId[B] + ): Option[SessionId[B]] = + backend.resumeWireId(client) + + /** Delegates to the backend's `registerSession` so the flow runtime can + * rehydrate the resume wire id from the persisted log on resume. + */ + override def registerResumeWireId( + client: SessionId[B], + wireId: SessionId[B] + ): Unit = + backend.registerSession(client, wireId) + val autonomous: AutonomousTextCall[B] = new AutonomousTextCall[B]: def run( prompt: String, session: SessionId[B] = SessionId.fresh[B], - callConfig: LlmConfig = LlmConfig.default, + callConfig: AgentConfig = AgentConfig.default, emitPrompt: Boolean = true - ): (SessionId[B], String) = + )(using orca.InStage): (SessionId[B], String) = val effective = effectiveConfig(callConfig) if emitPrompt then events.onEvent(OrcaEvent.UserPrompt(prompt)) val result = @@ -69,8 +104,8 @@ abstract class BaseLlmTool[B <: BackendTag, Self <: LlmTool[B]]( emitTokens(effective, result) (result.sessionId, result.output) - def resultAs[O: JsonData: Announce]: LlmCall[B, O] = - new DefaultLlmCall[B, O]( + def resultAs[O: JsonData: Announce]: AgentCall[B, O] = + new DefaultAgentCall[B, O]( backend, effectiveConfig, prompts, @@ -84,14 +119,14 @@ abstract class BaseLlmTool[B <: BackendTag, Self <: LlmTool[B]]( * response-reported model (most precise) and falls back to whatever the * caller pinned in config. Stays None when neither is known. */ - private def emitTokens(effective: LlmConfig, result: LlmResult[B]): Unit = + private def emitTokens(effective: AgentConfig, result: AgentResult[B]): Unit = val model = result.model.orElse(effective.model) events.onEvent(OrcaEvent.TokensUsed(name, model, result.usage)) /** If the caller omitted the per-call `config` arg they get the shared - * `LlmConfig.default` singleton; in that case fall back to the tool-level - * config. Any explicit `LlmConfig` from the call site wholly replaces the + * `AgentConfig.default` singleton; in that case fall back to the tool-level + * config. Any explicit `AgentConfig` from the call site wholly replaces the * tool-level one — no per-field merge. */ - private def effectiveConfig(callConfig: LlmConfig): LlmConfig = - if callConfig eq LlmConfig.default then config else callConfig + private def effectiveConfig(callConfig: AgentConfig): AgentConfig = + if callConfig eq AgentConfig.default then config else callConfig diff --git a/tools/src/main/scala/orca/llm/CanAskUser.scala b/tools/src/main/scala/orca/agents/CanAskUser.scala similarity index 96% rename from tools/src/main/scala/orca/llm/CanAskUser.scala rename to tools/src/main/scala/orca/agents/CanAskUser.scala index d293f0e9..6b9f2e71 100644 --- a/tools/src/main/scala/orca/llm/CanAskUser.scala +++ b/tools/src/main/scala/orca/agents/CanAskUser.scala @@ -1,4 +1,4 @@ -package orca.llm +package orca.agents /** Compile-time capability typeclass: an instance exists iff the backend tagged * `B` exposes a host-side `ask_user` tool — i.e. interactive sessions on that @@ -8,7 +8,7 @@ package orca.llm * Used as a constraint on flow helpers that depend on the capability: * * {{{ - * def from[B <: BackendTag: CanAskUser](llm: LlmTool[B], ...): T + * def from[B <: BackendTag: CanAskUser](agent: Agent[B], ...): T * }}} * * Calling with a backend that lacks an instance is a compile error. diff --git a/tools/src/main/scala/orca/agents/JsonData.scala b/tools/src/main/scala/orca/agents/JsonData.scala new file mode 100644 index 00000000..6657e02f --- /dev/null +++ b/tools/src/main/scala/orca/agents/JsonData.scala @@ -0,0 +1,137 @@ +package orca.agents + +import com.github.plokhotnyuk.jsoniter_scala.core.{ + JsonReader, + JsonValueCodec, + JsonWriter +} +import com.github.plokhotnyuk.jsoniter_scala.macros.{ + CodecMakerConfig, + ConfiguredJsonValueCodec, + JsonCodecMaker +} +import sttp.tapir.Schema + +import scala.deriving.Mirror + +/** Bundles a tapir `Schema` and a jsoniter-scala `ConfiguredJsonValueCodec` for + * a type. Flow scripts use `derives JsonData` on case classes that travel in + * and out of LLM calls as structured JSON. + * + * Scripts must import via `import orca.{*, given}` — Scala 3's plain wildcard + * imports exclude givens, and `derives JsonData` on a case class with nested + * case-class fields needs the forwarder givens below in scope. + */ +trait JsonData[A]: + def schema: Schema[A] + def codec: ConfiguredJsonValueCodec[A] + +object JsonData: + + /** Stricter-than-default jsoniter config: missing `List` / `Set` / `Map` + * fields fail to parse rather than defaulting to empty, so an agent reply + * with the right overall shape but the wrong fields can't masquerade as a + * "success with no content". + */ + inline def strictCodecConfig: CodecMakerConfig = + CodecMakerConfig + .withRequireCollectionFields(true) + .withTransientEmpty(false) + + def apply[A]( + schemaInstance: Schema[A], + codecInstance: ConfiguredJsonValueCodec[A] + ): JsonData[A] = + new JsonData[A]: + val schema: Schema[A] = schemaInstance + val codec: ConfiguredJsonValueCodec[A] = codecInstance + + inline def derived[A](using Mirror.Of[A]): JsonData[A] = + apply( + Schema.derived[A], + ConfiguredJsonValueCodec.derived[A](using strictCodecConfig) + ) + + /** Wraps a plain `JsonValueCodec` as a `ConfiguredJsonValueCodec`. + * `ConfiguredJsonValueCodec` is a marker interface that extends + * `JsonValueCodec` without adding methods, so we just delegate all calls. + * Used by the hand-written primitive/generic givens below. + */ + private def wrap[A](c: JsonValueCodec[A]): ConfiguredJsonValueCodec[A] = + new ConfiguredJsonValueCodec[A]: + def decodeValue(in: JsonReader, default: A): A = + c.decodeValue(in, default) + def encodeValue(x: A, out: JsonWriter): Unit = c.encodeValue(x, out) + def nullValue: A = c.nullValue + + // ── Primitive givens ─────────────────────────────────────────────────────── + // Use the tapir Schema companion methods directly (not `summon`) to avoid + // triggering the package-level `schemaFromJsonData` given, which would + // reference the very instance being initialised (causing an infinite loop). + + given JsonData[String] = + apply(Schema.schemaForString, wrap(JsonCodecMaker.make)) + given JsonData[Int] = apply(Schema.schemaForInt, wrap(JsonCodecMaker.make)) + given JsonData[Long] = apply(Schema.schemaForLong, wrap(JsonCodecMaker.make)) + given JsonData[Boolean] = + apply(Schema.schemaForBoolean, wrap(JsonCodecMaker.make)) + given JsonData[Double] = + apply(Schema.schemaForDouble, wrap(JsonCodecMaker.make)) + + /** Unit serialises as `{}` (an empty JSON object) — a valid, round-trippable + * JSON value that conveys "no meaningful payload". `JsonCodecMaker` does not + * support `Unit`, so we write the codec by hand. + * + * Decode is strict: it requires exactly the empty object `{}` the encoder + * produces, rejecting any other token (including `null`). So `Option[Unit]` + * round-trips correctly — `None`/`Some(())` map to `null`/`{}`. + */ + given JsonData[Unit] = apply( + Schema.schemaForUnit, + new ConfiguredJsonValueCodec[Unit]: + def decodeValue(in: JsonReader, default: Unit): Unit = + if in.isNextToken('{') then + if !in.isNextToken('}') then in.objectEndOrCommaError() + else in.decodeError("expected '{'") + def encodeValue(x: Unit, out: JsonWriter): Unit = + out.writeObjectStart() + out.writeObjectEnd() + def nullValue: Unit = () + ) + + // ── Generic givens ───────────────────────────────────────────────────────── + + given [A](using jd: JsonData[A]): JsonData[Option[A]] = + given JsonValueCodec[A] = jd.codec + apply(Schema.schemaForOption(jd.schema), wrap(JsonCodecMaker.make)) + + given [A](using jd: JsonData[A]): JsonData[List[A]] = + given JsonValueCodec[A] = jd.codec + // schemaForIterable returns Schema[Iterable[A]]; the cast to Schema[List[A]] + // is safe because at runtime both are the same array schema with A elements. + apply( + Schema + .schemaForIterable[A, List](jd.schema) + .asInstanceOf[Schema[List[A]]], + wrap(JsonCodecMaker.make) + ) + + given [A, B](using jdA: JsonData[A], jdB: JsonData[B]): JsonData[(A, B)] = + given JsonValueCodec[A] = jdA.codec + given JsonValueCodec[B] = jdB.codec + apply(Schema.derived[(A, B)], wrap(JsonCodecMaker.make)) + + given [A, B, C](using + jdA: JsonData[A], + jdB: JsonData[B], + jdC: JsonData[C] + ): JsonData[(A, B, C)] = + given JsonValueCodec[A] = jdA.codec + given JsonValueCodec[B] = jdB.codec + given JsonValueCodec[C] = jdC.codec + apply(Schema.derived[(A, B, C)], wrap(JsonCodecMaker.make)) + +given schemaFromJsonData[A](using jd: JsonData[A]): Schema[A] = jd.schema + +given codecFromJsonData[A](using jd: JsonData[A]): ConfiguredJsonValueCodec[A] = + jd.codec diff --git a/tools/src/main/scala/orca/llm/Model.scala b/tools/src/main/scala/orca/agents/Model.scala similarity index 98% rename from tools/src/main/scala/orca/llm/Model.scala rename to tools/src/main/scala/orca/agents/Model.scala index 1c523159..8ed63812 100644 --- a/tools/src/main/scala/orca/llm/Model.scala +++ b/tools/src/main/scala/orca/agents/Model.scala @@ -1,4 +1,4 @@ -package orca.llm +package orca.agents import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonReader, diff --git a/tools/src/main/scala/orca/llm/Prompts.scala b/tools/src/main/scala/orca/agents/Prompts.scala similarity index 84% rename from tools/src/main/scala/orca/llm/Prompts.scala rename to tools/src/main/scala/orca/agents/Prompts.scala index dc9cca2f..5e4c9c54 100644 --- a/tools/src/main/scala/orca/llm/Prompts.scala +++ b/tools/src/main/scala/orca/agents/Prompts.scala @@ -1,10 +1,10 @@ -package orca.llm +package orca.agents import orca.util.PromptResource /** Builds the literal prompt strings Orca sends to the LLM for each invocation * mode. Each method takes the serialized user input, the generated JSON Schema - * for the expected output, and the active `LlmConfig`, and returns the final + * for the expected output, and the active `AgentConfig`, and returns the final * prompt text. Swap the default ([[DefaultPrompts]]) by passing `prompts = * ...` to `flow(...)` when you want to customise phrasing, add guardrails, or * use a different structured-output convention. @@ -13,7 +13,11 @@ trait Prompts: /** Prompt for a non-interactive call: the model is expected to emit the * structured JSON response directly, with no user turn in between. */ - def autonomous(input: String, outputSchema: String, config: LlmConfig): String + def autonomous( + input: String, + outputSchema: String, + config: AgentConfig + ): String /** Prompt for an interactive call: the model converses with the user on * intermediate turns, then produces a single JSON value matching the output @@ -24,7 +28,7 @@ trait Prompts: def interactive( input: String, outputSchema: String, - config: LlmConfig + config: AgentConfig ): String /** Builds a prompt asking the model to retry after a JSON parse failure. @@ -34,7 +38,8 @@ trait Prompts: def retry(failedResponse: String, parseError: String): String /** Default [[Prompts]] implementation. Templates live as `.md` resources under - * `src/main/resources/orca/llm/prompts/` and are loaded once at object init. + * `src/main/resources/orca/agents/prompts/` and are loaded once at object + * init. * * Autonomous calls ship the JSON Schema inline in the prompt and rely on * `ResponseParser` + the retry loop for structural validation — they don't @@ -47,30 +52,30 @@ trait Prompts: object DefaultPrompts extends Prompts: private val RawJsonRules: String = - PromptResource.load("/orca/llm/prompts/raw-json-rules.md") + PromptResource.load("/orca/agents/prompts/raw-json-rules.md") // Substitute the shared rules fragment once at init so each call only pays // for the dynamic `{{input}}` / `{{outputSchema}}` / `{{failedResponse}}` / // `{{parseError}}` replacements. private val AutonomousTemplate: String = PromptResource - .load("/orca/llm/prompts/autonomous.md") + .load("/orca/agents/prompts/autonomous.md") .replace("{{rawJsonRules}}", RawJsonRules) private val InteractiveTemplate: String = PromptResource - .load("/orca/llm/prompts/interactive.md") + .load("/orca/agents/prompts/interactive.md") .replace("{{rawJsonRules}}", RawJsonRules) private val RetryTemplate: String = PromptResource - .load("/orca/llm/prompts/retry.md") + .load("/orca/agents/prompts/retry.md") .replace("{{rawJsonRules}}", RawJsonRules) def autonomous( input: String, outputSchema: String, - config: LlmConfig + config: AgentConfig ): String = PromptResource.render( AutonomousTemplate, @@ -81,7 +86,7 @@ object DefaultPrompts extends Prompts: def interactive( input: String, outputSchema: String, - config: LlmConfig + config: AgentConfig ): String = PromptResource.render( InteractiveTemplate, diff --git a/tools/src/main/scala/orca/llm/ResponseParser.scala b/tools/src/main/scala/orca/agents/ResponseParser.scala similarity index 99% rename from tools/src/main/scala/orca/llm/ResponseParser.scala rename to tools/src/main/scala/orca/agents/ResponseParser.scala index 0aaa97a7..c3d11209 100644 --- a/tools/src/main/scala/orca/llm/ResponseParser.scala +++ b/tools/src/main/scala/orca/agents/ResponseParser.scala @@ -1,4 +1,4 @@ -package orca.llm +package orca.agents import com.github.plokhotnyuk.jsoniter_scala.core.{ JsonReaderException, diff --git a/tools/src/main/scala/orca/backend/LlmBackend.scala b/tools/src/main/scala/orca/backend/AgentBackend.scala similarity index 62% rename from tools/src/main/scala/orca/backend/LlmBackend.scala rename to tools/src/main/scala/orca/backend/AgentBackend.scala index e29cb22c..e12d30d3 100644 --- a/tools/src/main/scala/orca/backend/LlmBackend.scala +++ b/tools/src/main/scala/orca/backend/AgentBackend.scala @@ -1,11 +1,15 @@ package orca.backend import orca.events.OrcaListener -import orca.llm.{BackendTag, LlmConfig, SessionId} +import orca.agents.{BackendTag, AgentConfig, SessionId, isSafeSessionId} + +import ox.Ox + +import scala.util.control.NonFatal /** SPI implemented per backend (Claude, Codex, …). The framework calls these * methods from the autonomous-text and structured-output paths - * ([[AutonomousTextCall]], [[LlmCall]]). + * ([[AutonomousTextCall]], [[AgentCall]]). * * Each method takes a `session: SessionId[B]` — the framework hands the same * value across calls; the backend decides internally whether this is the first @@ -22,7 +26,7 @@ import orca.llm.{BackendTag, LlmConfig, SessionId} * * `workDir` is the working directory the agent subprocess sees. */ -trait LlmBackend[B <: BackendTag]: +trait AgentBackend[B <: BackendTag]: /** Run one autonomous turn against `session` and return its result. The * backend decides whether to create the session (first call with this id) or * resume it (subsequent calls). @@ -43,11 +47,11 @@ trait LlmBackend[B <: BackendTag]: def runAutonomous( prompt: String, session: SessionId[B], - config: LlmConfig, + config: AgentConfig, workDir: os.Path, events: OrcaListener = OrcaListener.noop, outputSchema: Option[String] = None - ): LlmResult[B] + ): AgentResult[B] /** Launch an interactive session against `session` and return a live * [[Conversation]] the caller hands to [[Interaction.drive]] for rendering @@ -63,10 +67,10 @@ trait LlmBackend[B <: BackendTag]: prompt: String, session: SessionId[B], displayPrompt: String, - config: LlmConfig, + config: AgentConfig, workDir: os.Path, outputSchema: Option[String] - ): Conversation[B] + )(using Ox): Conversation[B] /** Hook for backends that mint server-side session ids during a conversation * drain: after the interactive `Conversation` returned by [[runInteractive]] @@ -78,3 +82,43 @@ trait LlmBackend[B <: BackendTag]: * thread. */ def registerSession(client: SessionId[B], server: SessionId[B]): Unit = () + + /** Non-destructive check: does a live, resumable backend conversation exist + * for `session`? Best-effort — returns `false` when the backend store/CLI is + * absent or the answer can't be determined (the caller then re-seeds, which + * is always safe). Must NOT create, mutate, or resume the session. + */ + def sessionExists(session: SessionId[B]): Boolean = false + + /** Read the wire id to resume `client` against, or `None` if no live mapping + * is known (or the backend's sessions aren't durably resumable). Pure, + * thread-safe, side-effect-free. + * + * Backends with a durable [[SessionRegistry]] delegate to its + * `resumeWireId`; the default returns `None` (pi, whose temp-dir sessions + * don't survive a restart, keeps the default). Used by the flow runtime to + * persist the resume wire id into the progress log (so a resumed run can + * rehydrate it via [[registerSession]]) and to probe it for existence. + */ + def resumeWireId(client: SessionId[B]): Option[SessionId[B]] = None + + /** Run `probe` on `id` only if `id` is a safe session id, treating ANY + * non-fatal failure (and an unsafe id) as "not found". The non-destructive, + * best-effort contract every `sessionExists` probe shares. + */ + protected def probeGuarded(id: String)(probe: String => Boolean): Boolean = + if !isSafeSessionId(id) then false + else + try probe(id) + catch case NonFatal(_) => false + + /** For server-id backends: resolve the recorded resume wire id via `registry` + * (None ⇒ not found, no probe) and `probeGuarded` it. + */ + protected def probeServerSession( + session: SessionId[B], + registry: SessionRegistry[B] + )(probe: String => Boolean): Boolean = + registry.resumeWireId(session) match + case None => false + case Some(srv) => probeGuarded(SessionId.value(srv))(probe) diff --git a/tools/src/main/scala/orca/backend/LlmResult.scala b/tools/src/main/scala/orca/backend/AgentResult.scala similarity index 69% rename from tools/src/main/scala/orca/backend/LlmResult.scala rename to tools/src/main/scala/orca/backend/AgentResult.scala index 2c7fb16b..b45f875a 100644 --- a/tools/src/main/scala/orca/backend/LlmResult.scala +++ b/tools/src/main/scala/orca/backend/AgentResult.scala @@ -1,14 +1,14 @@ package orca.backend -import orca.llm.{BackendTag, Model, SessionId} +import orca.agents.{BackendTag, Model, SessionId} import orca.events.{Usage} -/** Outcome of a single LLM call. Returned by [[LlmBackend.runAutonomous]] / - * [[LlmBackend.continueAutonomous]] for the autonomous path, and by +/** Outcome of a single LLM call. Returned by [[AgentBackend.runAutonomous]] / + * [[AgentBackend.continueAutonomous]] for the autonomous path, and by * [[Conversation.awaitResult]] / [[Interaction.drive]] for the interactive * path. */ -case class LlmResult[B <: BackendTag]( +case class AgentResult[B <: BackendTag]( sessionId: SessionId[B], output: String, usage: Usage, diff --git a/tools/src/main/scala/orca/backend/AskUserEchoes.scala b/tools/src/main/scala/orca/backend/AskUserEchoes.scala new file mode 100644 index 00000000..cb949eaa --- /dev/null +++ b/tools/src/main/scala/orca/backend/AskUserEchoes.scala @@ -0,0 +1,37 @@ +package orca.backend + +/** Tracks the tool-call ids of `ask_user` invocations whose wire echo must be + * dropped from the conversation event stream. + * + * Every stream backend that bridges `ask_user` through the host-side MCP + * surfaces the question as a [[ConversationEvent.UserQuestion]], so + * re-emitting the agent's own tool-call block AND the paired tool-result would + * render the exchange twice — the user's typed answer reappearing as `⎿ + * <answer>` right after the prompt already showed it. Each driver therefore + * suppresses the `ask_user` tool-call and drops its matching result. + * + * What is shared is exactly this id bookkeeping. What is NOT shared — and + * stays at each call site — is the matcher for "is this an `ask_user` call", + * because the backends name the tool differently: claude + * `mcp__<server>__<tool>`, codex a `(server, tool)` pair, gemini + * `<server>__<tool>` or the bare slug. (Gemini notably must NOT match any name + * merely *containing* the slug.) + * + * Single-threaded: like the rest of the `ForkedConversation` subclass state, + * it is touched only from the reader thread. + */ +private[orca] final class AskUserEchoes: + private var ids: Set[String] = Set.empty + + /** Remember `id` so the paired tool-result echo is dropped when it arrives. + */ + def suppress(id: String): Unit = ids = ids + id + + /** True iff `id` was suppressed — and forgets it, since the echo arrives + * once. Returns false (and does nothing) for an id that was never + * suppressed, so it is safe to call on every tool-result. + */ + def consume(id: String): Boolean = + val hit = ids.contains(id) + if hit then ids = ids - id + hit diff --git a/tools/src/main/scala/orca/backend/BufferedStderrDiagnostics.scala b/tools/src/main/scala/orca/backend/BufferedStderrDiagnostics.scala index a76e15e4..682b8e0a 100644 --- a/tools/src/main/scala/orca/backend/BufferedStderrDiagnostics.scala +++ b/tools/src/main/scala/orca/backend/BufferedStderrDiagnostics.scala @@ -1,24 +1,24 @@ package orca.backend -import orca.llm.BackendTag +import orca.agents.BackendTag import java.util.concurrent.atomic.AtomicReference -/** Bounded-stderr diagnostics shared by the subprocess [[StreamConversation]]s - * (codex, pi). Keeps the last few trimmed stderr lines (capped on count and - * bytes) so a non-zero exit / clean-exit-without-result carries the real - * failure context in the thrown exception — listener subscribers saw each line - * as a `ConversationEvent.Error`, but a noop listener (tests, simple scripts) - * would otherwise lose it. Also joins the stderr drain at finalize so trailing - * lines reach the queue before it closes. +/** Bounded-stderr diagnostics shared by the subprocess [[ForkedConversation]]s + * (codex, gemini, pi). Keeps the last few trimmed stderr lines (capped on + * count and bytes) so a non-zero exit / clean-exit-without-result carries the + * real failure context in the thrown exception — listener subscribers saw each + * line as a `ConversationEvent.Error`, but a noop listener (tests, simple + * scripts) would otherwise lose it. Also joins the stderr drain at finalize so + * trailing lines reach the queue before it closes. * - * The driver keeps its own [[StreamConversation.handleStderr]] (the noise + * The driver keeps its own [[ForkedConversation.handleStderr]] (the noise * filter and prefix genuinely differ per backend) and calls [[recordStderr]] * for lines worth keeping. A driver needing extra teardown overrides * `onFinalize` and calls `super.onFinalize()`. */ private[orca] trait BufferedStderrDiagnostics[B <: BackendTag] - extends StreamConversation[B]: + extends ForkedConversation[B]: import BufferedStderrDiagnostics.* @@ -28,20 +28,22 @@ private[orca] trait BufferedStderrDiagnostics[B <: BackendTag] protected def recordStderr(line: String): Unit = val _ = stderrBuffer.updateAndGet(appendBounded(_, line)) - /** Bounded wait for the stderr drain so trailing lines reach the queue. */ + /** Wait for the stderr drain so trailing lines reach the queue before the + * failure outcome is computed. No timeout is needed: `cancel()`'s + * `destroyForcibly` (and a real process's exit) always EOFs the stderr + * stream, so the drain fork terminates. + */ override protected def onFinalize(): Unit = - try stderrDrainThread.join(DrainTimeoutMs) - catch case _: InterruptedException => Thread.currentThread().interrupt() + stderrDrainFork.join() - /** Recent stderr lines as a `stderr:` block; the base owns the outer framing. */ + /** Recent stderr lines as a `stderr:` block; the base owns the outer framing. + */ override protected def diagnosticContext: Option[String] = val lines = stderrBuffer.get() if lines.isEmpty then None else Some(lines.mkString("stderr:\n ", "\n ", "")) private[orca] object BufferedStderrDiagnostics: - /** Bounded wait for the stderr drain thread at finalize. */ - val DrainTimeoutMs: Long = 500L /** Cap on lines kept — sized for a typical stack trace plus a brief * explanation, bounded so a chatty subprocess can't grow memory. @@ -52,8 +54,8 @@ private[orca] object BufferedStderrDiagnostics: val MaxBytes: Int = 4096 /** Append `line` while respecting both caps, dropping oldest first. A single - * over-cap line is kept anyway (better than empty diagnostics). Pure, so it's - * a safe `AtomicReference.updateAndGet` callback. + * over-cap line is kept anyway (better than empty diagnostics). Pure, so + * it's a safe `AtomicReference.updateAndGet` callback. */ def appendBounded(buf: Vector[String], line: String): Vector[String] = var result = buf :+ line diff --git a/tools/src/main/scala/orca/backend/CliArgs.scala b/tools/src/main/scala/orca/backend/CliArgs.scala index 4b273a89..a1ca78dc 100644 --- a/tools/src/main/scala/orca/backend/CliArgs.scala +++ b/tools/src/main/scala/orca/backend/CliArgs.scala @@ -1,9 +1,9 @@ package orca.backend -import orca.llm.LlmConfig +import orca.agents.AgentConfig /** CLI-flag helpers shared between backend arg builders (`ClaudeArgs`, - * `CodexArgs`). Each helper renders a single `LlmConfig` field as a `Seq` + * `CodexArgs`). Each helper renders a single `AgentConfig` field as a `Seq` * suitable for concatenation into a backend's argv. Empty `Seq` when the field * is absent, so callers don't have to special-case `None`. */ @@ -13,5 +13,5 @@ private[orca] object CliArgs: * supported backends spell the flag the same way; if a future backend * differs, render the model name elsewhere and don't use this helper. */ - def modelArgs(config: LlmConfig): Seq[String] = + def modelArgs(config: AgentConfig): Seq[String] = config.model.toSeq.flatMap(m => Seq("--model", m.name)) diff --git a/tools/src/main/scala/orca/backend/Conversation.scala b/tools/src/main/scala/orca/backend/Conversation.scala index 9f93103b..f4f52769 100644 --- a/tools/src/main/scala/orca/backend/Conversation.scala +++ b/tools/src/main/scala/orca/backend/Conversation.scala @@ -1,6 +1,6 @@ package orca.backend -import orca.llm.{BackendTag} +import orca.agents.{BackendTag} import orca.{OrcaInteractiveCancelled} /** One live interactive session with a backend. Owned by the driver, read and @@ -13,8 +13,7 @@ import orca.{OrcaInteractiveCancelled} * * Tool-approval decisions are delivered via the closure carried on * [[ConversationEvent.ApproveTool]] — the channel does not track request-ids. - * `sendUserMessage` injects an unsolicited user turn; it and `cancel` are safe - * to call from any thread. + * `cancel` is safe to call from any thread. */ trait Conversation[B <: BackendTag]: @@ -37,7 +36,7 @@ trait Conversation[B <: BackendTag]: /** Block until the session finishes, then return its outcome. * - * - `Right(result)` — the session produced an [[LlmResult]] cleanly. + * - `Right(result)` — the session produced an [[AgentResult]] cleanly. * - `Left(cancelled)` — the user (or some peer) called [[cancel]], or the * subprocess died in a way the driver classified as a cancellation. * Recoverable: the caller can render a "cancelled" message, fail the @@ -47,15 +46,7 @@ trait Conversation[B <: BackendTag]: * abnormal exit codes) keep throwing [[OrcaFlowException]] — those aren't * recoverable signals; they're "the backend is broken, panic" cases. */ - def awaitResult(): Either[OrcaInteractiveCancelled, LlmResult[B]] - - /** Inject a user turn mid-conversation by writing to the subprocess's stdin. - * Only meaningful when the backend keeps stdin open for the life of the - * conversation — claude's stream-json does, codex's `exec` does not (it - * consumes stdin once at startup; ADR 0007). Implementations whose stdin - * channel isn't writable mid-session treat this as a no-op. - */ - def sendUserMessage(text: String): Unit + def awaitResult(): Either[OrcaInteractiveCancelled, AgentResult[B]] /** Whether the agent can pause to ask the host user a clarifying question * (and have the answer routed back into its turn). When `true`, the driver @@ -65,11 +56,6 @@ trait Conversation[B <: BackendTag]: * `AskUserMcpServer` registered with `-c mcp_servers.orca.url=…`) return * `true` on interactive sessions; autonomous sessions and backends that * don't wire the bridge return `false`. - * - * Independent of [[sendUserMessage]] — codex satisfies `canAskUser` while - * still having a no-op `sendUserMessage`. Flows that depend on being able to - * push a turn into stdin must check that channel separately rather than - * treating `canAskUser` as a proxy. */ def canAskUser: Boolean diff --git a/tools/src/main/scala/orca/backend/Conversations.scala b/tools/src/main/scala/orca/backend/Conversations.scala index 273b4476..580c6613 100644 --- a/tools/src/main/scala/orca/backend/Conversations.scala +++ b/tools/src/main/scala/orca/backend/Conversations.scala @@ -1,11 +1,11 @@ package orca.backend -import orca.OrcaInteractiveCancelled +import orca.{AgentTurnFailed, OrcaFlowException, OrcaInteractiveCancelled} import orca.events.{OrcaEvent, OrcaListener} -import orca.llm.BackendTag +import orca.agents.{BackendTag, SessionId} /** Drains a [[Conversation]] for the autonomous path, mapping conversation - * events to [[OrcaEvent]]s and returning the awaited `LlmResult`. + * events to [[OrcaEvent]]s and returning the awaited `AgentResult`. * * Structured mode (`conv.outputSchema.isDefined`) withholds the last assistant * turn so the closing JSON payload doesn't surface as an `AssistantMessage` — @@ -27,7 +27,7 @@ private[orca] object Conversations: def drainAutonomous[B <: BackendTag]( conv: Conversation[B], events: OrcaListener = OrcaListener.noop - ): LlmResult[B] = + ): AgentResult[B] = val structuredMode = conv.outputSchema.isDefined val textBuf = new StringBuilder // Previously-closed turn's text, kept around in structured mode while we @@ -85,7 +85,7 @@ private[orca] object Conversations: ) case ConversationEvent.UserMessage(_) => // Wire-level echo of input we already sent. Surfaced upstream as - // `OrcaEvent.UserPrompt` from the LlmTool layer; nothing to do + // `OrcaEvent.UserPrompt` from the Agent layer; nothing to do // here. () case ConversationEvent.ToolResult(_, _, _) => @@ -104,5 +104,46 @@ private[orca] object Conversations: conv.awaitResult() match case Right(result) => result // Autonomous callers can't produce a Left; surface as a throw so the - // LlmResult call shape is honoured. + // AgentResult call shape is honoured. case Left(cancelled) => throw cancelled + + /** The shared autonomous-turn finalize for the subprocess backends + * (claude/codex/gemini/pi): drain the conversation, then — on success only — + * commit the session as resumable. Returns the drained result verbatim; + * codex/gemini/pi re-stamp the caller's stable handle with `.copy(sessionId + * \= session)` at the call site. Only claude leaves the result's session id + * untouched — it surfaces the id claude reported, which in production + * already equals the client id. + * + * Two invariants are centralised here because each backend got them subtly + * wrong at some point: + * + * - [[AgentTurnFailed]] is re-thrown verbatim (a turn that ran and failed + * must NOT be retried — a retry would reopen the now-registered session + * id). Any other [[OrcaFlowException]] is rewrapped under `backendName` + * so the user sees which backend failed; the corrective-retry loop in + * `DefaultAgentCall` still recognises it as retryable. + * - `commitSuccess` runs only after a clean drain, so a subprocess that + * crashed before registering its session doesn't wedge the registry into + * resuming a session that was never created. + * + * `registry.commitSuccess(session, result.sessionId)` is uniform across both + * registry shapes: [[SessionRegistry.ClientToServer]] records the learned + * server id, while [[SessionRegistry.ClaimedOnce]] ignores the server arg + * and just marks the client id claimed. + */ + def drainAndCommit[B <: BackendTag]( + backendName: String, + conv: Conversation[B], + session: SessionId[B], + registry: SessionRegistry[B], + events: OrcaListener = OrcaListener.noop + ): AgentResult[B] = + val result = + try drainAutonomous(conv, events) + catch + case e: AgentTurnFailed => throw e + case e: OrcaFlowException => + throw OrcaFlowException(s"$backendName CLI failed: ${e.getMessage}") + registry.commitSuccess(session, result.sessionId) + result diff --git a/tools/src/main/scala/orca/backend/ForkedConversation.scala b/tools/src/main/scala/orca/backend/ForkedConversation.scala new file mode 100644 index 00000000..66fbb11e --- /dev/null +++ b/tools/src/main/scala/orca/backend/ForkedConversation.scala @@ -0,0 +1,381 @@ +package orca.backend + +import orca.backend.mcp.{AskUserBridge, AskUserSession} +import orca.agents.BackendTag +import orca.util.OrcaDebug +import orca.{AgentTurnFailed, OrcaFlowException, OrcaInteractiveCancelled} + +import ox.{Ox, discard, fork, forkUnsupervised, Fork, UnsupervisedFork} +import ox.channels.{Channel, ChannelClosed} + +import java.util.concurrent.atomic.{AtomicBoolean, AtomicReference} +import scala.util.control.NonFatal + +/** Structured-concurrency base for stream-driven [[Conversation]] drivers — the + * forked successor to `StreamConversation`. Workers are Ox forks bound to the + * caller's per-turn scope (hence the `using Ox`), the session outcome is the + * reader fork's return value, and teardown is a lexical `cancel()` the caller + * runs in a `finally` (which subsumes the old `finalizeLoop` resource close). + * + * Same hook surface as `StreamConversation` so subclasses port with minimal + * edits: [[handleLine]], [[handleStderr]], [[onFinalize]], + * [[cleanExitWithoutResult]], [[diagnosticContext]], [[appendContext]], + * [[askUser]], the `eventQueue.enqueue` shim, and [[succeedWith]] / + * [[failWith]]. The differences from `StreamConversation`: + * + * - no `start()` / `ensureStarted`: the workers are spawned lazily on first + * touch of the conversation surface ([[events]] / [[awaitResult]]), never + * from this constructor — spawning here would race the subclass's own + * field initializers (which run after this super-constructor) and let the + * reader fork call [[handleLine]] against half-built subclass state. By + * the time a consumer touches the surface, construction has finished. The + * forks bind to the `Ox` captured at construction (the per-turn scope the + * backend opened), not to whatever scope is active at first touch — in + * every call site construction and consumption share that one scope. + * - no `outcomeRef` CAS read by `awaitResult`: the reader fork RETURNS the + * `Outcome`; `awaitResult` is `readerFork.join()` mapped as before. The + * in-stream settle ([[succeedWith]] / [[failWith]], run on the reader + * thread inside [[handleLine]]) stashes its result for the reader's + * end-of-stream to pick up. + */ +private[orca] abstract class ForkedConversation[B <: BackendTag]( + source: StreamSource, + /** Used in fork-internal debug traces, parse-error messages, and the + * default stderr error prefix. Should match the user-facing backend name. + */ + backendName: String, + initialPrompt: String = "", + /** Set by backends that answer `ask_user` through their own protocol + * (OpenCode, via a native HTTP `question` event). It only affects what + * [[canAskUser]] reports — the MCP drainer is gated on [[askUser]] alone. + */ + nativeAskUser: Boolean = false +)(using Ox) + extends Conversation[B]: + + import ForkedConversation.* + + /** Bounded event channel: a slow consumer applies backpressure to the + * producer (the reader's `send` blocks once full) so it flows back into the + * subprocess pipe. Created with the explicit capacity so no `BufferCapacity` + * needs threading through the backends. + */ + private val channel: Channel[ConversationEvent] = + Channel.buffered(EventQueueCapacity) + + /** `enqueue` / `close` facade over the channel for subclass drivers. It + * centralises the swallow-on-closed decision in one place: both use the + * `*OrClosed` variants so a late enqueue (e.g. the ask-user drainer racing + * the reader's close) is a no-op rather than a thrown `ChannelClosed` that + * would tear the scope. + */ + protected final class EventQueue: + def enqueue(event: ConversationEvent): Unit = + channel.sendOrClosed(event).discard + def close(): Unit = channel.doneOrClosed().discard + + protected val eventQueue: EventQueue = new EventQueue + + /** In-stream settled outcome, written once by [[succeedWith]] / [[failWith]] + * (on the reader thread, inside [[handleLine]]) and read by the reader at + * end-of-stream. Not read by [[awaitResult]] — the outcome flows out as the + * reader fork's return value. + */ + private val settledOutcome: AtomicReference[Option[Outcome[B]]] = + AtomicReference(None) + + private val cancelled: AtomicBoolean = new AtomicBoolean(false) + + /** Guards the one-shot finalize (subclass [[onFinalize]] + [[askUser]] + * close): whichever of the reader (happy path) or [[cancel]] (teardown) + * reaches it first runs it; the other is a no-op, so cancel never + * double-emits. + */ + private val finalized: AtomicBoolean = new AtomicBoolean(false) + + /** Optional `ask_user` MCP resource bundle for this conversation. Interactive + * subclasses override (via `override val askUser` on the ctor param) to + * point the base at the bundle; the base spawns the drainer fork when the + * workers start and closes the bundle in the finalize. Autonomous calls + * leave the default `None`. + */ + protected def askUser: Option[AskUserSession] = None + + /** True iff an `ask_user` MCP bundle is wired (claude/codex) or the backend + * declared its own ask-user channel via [[nativeAskUser]] (OpenCode). + */ + final def canAskUser: Boolean = nativeAskUser || askUser.isDefined + + // Surface the opening prompt before any agent output, so the channel has + // something to anchor the eventual response against instead of sitting silent + // while the agent warms up. Lands in the buffer now; the consumer reads it + // first once the reader starts. + if initialPrompt.nonEmpty then + eventQueue.enqueue(ConversationEvent.UserMessage(initialPrompt)) + + // --- Workers (Ox forks) --- + // + // Spawned lazily, never from this constructor (see the class scaladoc). The + // reader is `forkUnsupervised` so its (by-design impossible) stray exception + // surfaces via `join` rather than tearing the per-turn scope; it returns an + // `Outcome[B]` and never throws. The stderr-drain and ask-user forks are + // ordinary daemon `fork`s the scope cancels at its end. + + private lazy val stderrFork: Fork[Unit] = fork(stderrLoop()) + + private lazy val askUserFork: Unit = + askUser.foreach(r => fork(askUserDrain(r.bridge)).discard) + + private lazy val readerFork: UnsupervisedFork[Outcome[B]] = + // Force the auxiliary workers first so stderr/ask-user are running before + // the reader starts consuming stdout. + stderrFork.discard + askUserFork + forkUnsupervised(runReader()) + + private def ensureStarted(): Unit = readerFork.discard + + // --- Conversation surface --- + + def events: Iterator[ConversationEvent] = + ensureStarted() + channelIterator + + def awaitResult(): Either[OrcaInteractiveCancelled, AgentResult[B]] = + readerFork.join() match + case Outcome.Success(r) => Right(r) + case Outcome.Cancelled() => Left(new OrcaInteractiveCancelled()) + // A failure here means the turn ran (the session is registered) and then + // errored — tag it non-retryable so the autonomous retry loop doesn't + // reopen the locked session id. See [[AgentTurnFailed]]. + case Outcome.Failed(e: AgentTurnFailed) => throw e + case Outcome.Failed(e) => + throw new AgentTurnFailed( + Option(e.getMessage).filter(_.nonEmpty).getOrElse(e.toString) + ) + + def cancel(): Unit = + if cancelled.compareAndSet(false, true) then + // Graceful SIGINT first, then the guaranteed forcible backstop, so the + // non-interruptible reader's `source.lines` always reaches EOF and the + // scope join never hangs. On the happy path both are no-ops (the source + // already ended). Then run the full finalize (subsuming the old + // `finalizeLoop` resource close); the `finalized` guard means the reader, + // if it got there first, isn't re-run. + source.interrupt() + source.destroyForcibly() + runFinalize() + + // --- Settle helpers (called from handleLine, on the reader thread) --- + + /** Settle the turn with `result`, then interrupt the source so the reader + * loop reaches EOF. For drivers whose stream stays open after the turn + * (SSE), this is how the turn terminates; for subprocess drivers the source + * closes on its own and the interrupt is a harmless early teardown. + */ + protected def succeedWith(result: AgentResult[B]): Unit = + settledOutcome.compareAndSet(None, Some(Outcome.Success(result))).discard + source.interrupt() + + /** Settle the turn as a failure, then interrupt the source (see + * [[succeedWith]] for the ordering rationale). + */ + protected def failWith(error: Throwable): Unit = + settledOutcome.compareAndSet(None, Some(Outcome.failed(error))).discard + source.interrupt() + + // --- Hooks for backend implementations --- + + /** Process a single line of stdout. Implementations parse the protocol + * message and translate to [[ConversationEvent]] enqueues (and/or + * [[succeedWith]] / [[failWith]] settles). Exceptions thrown here are caught + * by the reader and surfaced as a generic parse-error Error event. + */ + protected def handleLine(line: String): Unit + + /** Process one line of stderr. Default: enqueue as an Error event with the + * backend name as a prefix. Override to filter known-noise lines. + */ + protected def handleStderr(line: String): Unit = + if line.trim.nonEmpty then + eventQueue.enqueue(ConversationEvent.Error(s"$backendName: $line")) + + /** Hook called inside the finalize **before** the failure outcome is + * computed, so subclasses can drain background streams whose buffered state + * [[diagnosticContext]] / [[cleanExitWithoutResult]] depend on (join the + * [[stderrDrainFork]] here). Also where backends release session-scoped + * resources. Runs exactly once (reader happy-path OR [[cancel]]). Default: + * no-op. + */ + protected def onFinalize(): Unit = () + + /** The exception used when the subprocess exits with code 0 without having + * sent a terminal protocol message. Default folds [[diagnosticContext]] into + * the message; backends may override. + */ + protected def cleanExitWithoutResult(): Throwable = + new OrcaFlowException( + appendContext( + s"$backendName exited cleanly but never sent a terminal message" + ) + ) + + /** Optional context the base folds into the non-zero-exit / clean-exit + * failure messages, so noop-listener callers still get something useful in + * the thrown exception. Default `None`; backends override to attach buffered + * stderr. Overrides return just the payload, never the separator. + */ + protected def diagnosticContext: Option[String] = None + + /** Fold the [[diagnosticContext]] payload (if any) onto a failure-message + * base — newline + two-space-indented payload. Centralised so every consumer + * gets the same framing. + */ + protected def appendContext(base: String): String = + diagnosticContext.fold(base)(ctx => s"$base\n $ctx") + + /** The stderr-drain fork, for backends whose [[onFinalize]] needs to wait for + * stderr to flush before computing a diagnostic-bearing failure. + */ + protected def stderrDrainFork: Fork[Unit] = stderrFork + + protected def debugLog(channel: String, line: String): Unit = + if OrcaDebug.streamTrace then + System.err.println(s"[orca-debug $backendName-$channel] $line") + + // --- Internals --- + + /** Sole outcome producer. Catches everything and RETURNS an `Outcome[B]` — + * never throws (H1), so `readerFork.join()` always yields an outcome. + */ + private def runReader(): Outcome[B] = + try + val readException: Option[Throwable] = + try + for line <- source.lines do + debugLog("stdout", line) + if !cancelled.get() then + try handleLine(line) + catch + case e: Exception => + eventQueue.enqueue( + ConversationEvent.Error( + s"Failed to parse $backendName line: ${e.getMessage}" + ) + ) + None + catch + case NonFatal(e) => + debugLog("stdout-error", e.toString) + Some(e) + // Finalize (drain background streams, close session resources) BEFORE + // computing a failure outcome so `diagnosticContext` is populated. + runFinalize() + val outcome: Outcome[B] = + // A user cancel (`destroyForcibly`) can make the in-flight read throw + // rather than EOF cleanly, so `cancelled` is checked BEFORE + // `readException` — a Ctrl-C must surface as `Cancelled`, never as a + // spurious turn failure. + settledOutcome + .get() + .orElse(Option.when(cancelled.get())(Outcome.cancelled[B])) + .orElse(readException.map(Outcome.failed[B])) + .getOrElse(outcomeFromExit(source.tryExitCode)) + eventQueue.close() + outcome + catch + case NonFatal(t) => + debugLog("reader-error", t.toString) + eventQueue.close() + Outcome.failed[B](t) + + private def stderrLoop(): Unit = + try + for line <- source.errorLines do + debugLog("stderr", line) + handleStderr(line) + catch + case NonFatal(t) => + // Best-effort — the reader doesn't depend on it. Surface the swallowed + // throwable under ORCA_DEBUG so a real bug isn't masked. + debugLog("stderr-error", s"${t.getClass.getName}: ${t.getMessage}") + + /** Bridge an [[AskUserBridge]] into the event stream: each pending question + * becomes a `UserQuestion` whose `respond` closure delivers the user's typed + * answer back to the blocked MCP handler. Exits cleanly when + * `bridge.close()` (driven by the finalize) raises `ChannelClosedException` + * from `nextQuestion()`. + */ + private def askUserDrain(bridge: AskUserBridge): Unit = + try + while true do + val q = bridge.nextQuestion() + eventQueue.enqueue( + ConversationEvent.UserQuestion(q.question, q.respond) + ) + catch case NonFatal(_) => () + + private def runFinalize(): Unit = + if finalized.compareAndSet(false, true) then + // 1. Subclass hook — typically joins the stderr-drain fork so trailing + // lines reach the queue / `diagnosticContext` before the outcome. + onFinalize() + // 2. Close the ask_user bundle (bridge → server → extras) if wired, after + // `onFinalize` so any cleanup depending on it runs first. Idempotent. + askUser.foreach(_.close()) + + private def outcomeFromExit(exitCode: Option[Int]): Outcome[B] = + exitCode match + case Some(0) => Outcome.failed[B](cleanExitWithoutResult()) + case Some(code) => + Outcome.failed[B]( + new OrcaFlowException( + appendContext(s"$backendName exited with code $code") + ) + ) + case None => Outcome.cancelled[B] + + /** Single-consumer iterator over the event channel; `done` ends it. */ + private val channelIterator: Iterator[ConversationEvent] = + new Iterator[ConversationEvent]: + // null — nothing peeked yet (will block on next `hasNext`); Some(e) — + // peeked event; None — stream closed, `hasNext` stays false forever. + private var peeked: Option[ConversationEvent] = null + + def hasNext: Boolean = + if peeked == null then + peeked = channel.receiveOrClosed() match + case _: ChannelClosed => None + case e: ConversationEvent => Some(e) + peeked.isDefined + + def next(): ConversationEvent = + if !hasNext then throw new NoSuchElementException("event stream closed") + val value = peeked.get + peeked = null + value + +private[orca] object ForkedConversation: + + /** Cap on in-flight unread `ConversationEvent`s (today's `StreamConversation` + * value). The producer blocks on `send` once full so backpressure flows back + * into the subprocess pipe. + */ + val EventQueueCapacity: Int = 1024 + + /** Internal outcome of the session as the reader sees it. Modelled as a + * sealed trait + cases so `Cancelled` / `Failed` are backend-agnostic + * (`Nothing` for their `B` keeps them assignable to any `Outcome[B]` while + * the match still narrows `Success`'s `B`). `Outcome` is invariant in `B` + * because `AgentResult[B]` is. + */ + sealed trait Outcome[B <: BackendTag] + object Outcome: + final case class Success[B <: BackendTag](result: AgentResult[B]) + extends Outcome[B] + final case class Cancelled[B <: BackendTag]() extends Outcome[B] + final case class Failed[B <: BackendTag](error: Throwable) + extends Outcome[B] + + def cancelled[B <: BackendTag]: Outcome[B] = Cancelled[B]() + def failed[B <: BackendTag](error: Throwable): Outcome[B] = Failed[B](error) diff --git a/tools/src/main/scala/orca/backend/Interaction.scala b/tools/src/main/scala/orca/backend/Interaction.scala index 0beb6f90..8e292e1b 100644 --- a/tools/src/main/scala/orca/backend/Interaction.scala +++ b/tools/src/main/scala/orca/backend/Interaction.scala @@ -1,6 +1,6 @@ package orca.backend -import orca.llm.{BackendTag} +import orca.agents.{BackendTag} import orca.events.{OrcaListener} /** The channel that connects a flow to its user. `listeners` are registered on @@ -15,11 +15,11 @@ trait Interaction: def listeners: List[OrcaListener] /** Drive a live interactive session to completion. Returns the final - * [[LlmResult]] on success, throws [[OrcaInteractiveCancelled]] if the user - * cancelled mid-session, or any [[OrcaFlowException]] subtype for other + * [[AgentResult]] on success, throws [[OrcaInteractiveCancelled]] if the + * user cancelled mid-session, or any [[OrcaFlowException]] subtype for other * failures. */ - def drive[B <: BackendTag](conversation: Conversation[B]): LlmResult[B] + def drive[B <: BackendTag](conversation: Conversation[B]): AgentResult[B] /** Release any background resources (worker threads, channels, etc.). The * runtime calls this once after the flow body completes, regardless of diff --git a/tools/src/main/scala/orca/backend/SessionRegistry.scala b/tools/src/main/scala/orca/backend/SessionRegistry.scala index b44e39a8..1f2d3fd0 100644 --- a/tools/src/main/scala/orca/backend/SessionRegistry.scala +++ b/tools/src/main/scala/orca/backend/SessionRegistry.scala @@ -1,6 +1,6 @@ package orca.backend -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} /** Whether a backend call against a given caller-supplied session id should * start a fresh session or resume an existing one. Each backend has its own @@ -27,12 +27,23 @@ enum Dispatch[B <: BackendTag]: * flows fan reviewers out via `mapParUnordered`. The `dispatchFor` → spawn → * `commitSuccess` sequence is NOT atomic, so callers must not share a session * id across concurrent calls — each reviewer mints its own via - * `LlmTool.newSession`. + * `Agent.newSession`. */ trait SessionRegistry[B <: BackendTag]: def dispatchFor(client: SessionId[B]): Dispatch[B] def commitSuccess(client: SessionId[B], server: SessionId[B]): Unit + /** Pure, thread-safe read of the wire id to resume `client` against, or + * `None` if no live mapping is known. For [[ClientToServer]] this is the + * recorded server thread id; for [[ClaimedOnce]] the client IS the wire id, + * so it returns `Some(client)` once claimed and `None` before. + * + * Used by the flow runtime to persist the resume wire id into the progress + * log and to drive the existence probe. Never creates, mutates, or resumes a + * session. + */ + def resumeWireId(client: SessionId[B]): Option[SessionId[B]] + object SessionRegistry: /** For backends whose client-supplied session id IS the canonical id on the @@ -56,6 +67,12 @@ object SessionRegistry: def commitSuccess(client: SessionId[B], server: SessionId[B]): Unit = val _ = claimed.add(SessionId.value(client)) + /** The client id IS the wire id, so a claimed client resumes against + * itself. + */ + def resumeWireId(client: SessionId[B]): Option[SessionId[B]] = + Option.when(claimed.contains(SessionId.value(client)))(client) + /** For backends whose session id is server-minted at first use, learned from * the protocol response. The framework hands the caller a stable client id; * the backend maps it to whatever the wire id turns out to be and resumes @@ -83,3 +100,6 @@ object SessionRegistry: SessionId.value(client), SessionId.value(server) ) + + def resumeWireId(client: SessionId[B]): Option[SessionId[B]] = + Option(map.get(SessionId.value(client))).map(SessionId[B](_)) diff --git a/tools/src/main/scala/orca/backend/StreamConversation.scala b/tools/src/main/scala/orca/backend/StreamConversation.scala deleted file mode 100644 index c145a226..00000000 --- a/tools/src/main/scala/orca/backend/StreamConversation.scala +++ /dev/null @@ -1,391 +0,0 @@ -package orca.backend - -import orca.backend.mcp.{AskUserBridge, AskUserSession} -import orca.llm.BackendTag -import orca.util.OrcaDebug -import orca.{AgentTurnFailed, OrcaFlowException, OrcaInteractiveCancelled} - -import java.util.concurrent.LinkedBlockingQueue -import java.util.concurrent.atomic.{AtomicBoolean, AtomicReference} -import scala.util.control.NonFatal - -/** Base implementation for stream-driven [[Conversation]] drivers. - * - * Owns the boilerplate every backend needs to translate a [[StreamSource]] (a - * subprocess's stdout, or an SSE connection) into a [[Conversation]]: - * - * - A daemon reader thread that calls [[handleLine]] on each inbound line - * and, on EOF or error, finalises the [[Outcome]]. - * - A daemon diagnostic drain thread that calls [[handleStderr]] on each - * [[StreamSource.errorLines]] line. - * - A single-consumer event queue surfaced via [[events]]. - * - Lifecycle: `awaitResult` joins the reader, returns `Right(LlmResult)` / - * `Left(OrcaInteractiveCancelled)` / throws for anything else; `cancel` - * interrupts the source and lets the reader observe EOF. - * - * Backends supply only the protocol-specific bits: [[handleLine]] (parse + - * translate to events / outcome), the optional [[handleStderr]] override - * (filter noise, drop the prefix), and the optional [[onFinalize]] hook (drain - * the diagnostic stream before the queue closes, etc.). - * - * The driver's role is strictly protocol translation: bytes in → - * [[ConversationEvent]]s out. Rendering and approval-policy live elsewhere. - * Outbound writes (user turns, tool-approval responses) happen on the - * channel's thread; the reader thread only produces events. - */ -private[orca] abstract class StreamConversation[B <: BackendTag]( - source: StreamSource, - /** Used in thread names ("claude-conversation-reader"), debug traces, - * parse-error messages, and the default stderr error prefix. Should match - * the user-facing backend name. - */ - backendName: String, - initialPrompt: String = "", - /** Set by backends that answer `ask_user` through their own protocol - * (OpenCode, via a native HTTP `question` event). It only affects what - * [[canAskUser]] reports — the MCP drainer is gated on [[askUser]] alone. - */ - nativeAskUser: Boolean = false -) extends Conversation[B]: - - import StreamConversation.* - - protected val eventQueue: EventQueue = new EventQueue - protected val outcomeRef: AtomicReference[Option[Outcome[B]]] = - AtomicReference(None) - protected val cancelled: AtomicBoolean = new AtomicBoolean(false) - - /** Optional `ask_user` MCP resource bundle for this conversation. Subclasses - * on interactive calls override (via `override val askUser` on the ctor - * param) to point the base at the bundle; the base spawns the drainer thread - * inside [[start]] and closes the bundle in [[finalizeLoop]] - * post-`onFinalize`. Autonomous calls leave the default `None` and the - * wiring is a no-op. - */ - protected def askUser: Option[AskUserSession] = None - - /** True iff an `ask_user` MCP bundle is wired (claude/codex) or the backend - * declared its own ask-user channel via [[nativeAskUser]] (OpenCode). - */ - final def canAskUser: Boolean = nativeAskUser || askUser.isDefined - - // Surface the opening prompt to the channel before any agent - // output. Without this, the channel sits silent while the agent - // warms up — giving the user nothing to anchor the eventual - // response against. - if initialPrompt.nonEmpty then - eventQueue.enqueue(ConversationEvent.UserMessage(initialPrompt)) - - // Lazy so the worker threads only spin up once the subclass has - // finished initialising its own fields — otherwise a pre-populated - // stdout fake (think tests) can let the reader race past EOF and - // touch a `null` subclass val before its `val` initializer runs. - // Concrete drivers call `start()` at the end of their constructor. - private lazy val readerThread: Thread = - val t = new Thread(() => readLoop(), s"$backendName-conversation-reader") - t.setDaemon(true) - t - - private lazy val stderrThread: Thread = - val t = new Thread(() => stderrLoop(), s"$backendName-conversation-stderr") - t.setDaemon(true) - t - - /** Spin up the stdout + stderr workers. Subclasses **must** call this at the - * end of their constructor, after their own fields are assigned — forgetting - * it leaves `awaitResult` returning immediately with "interactive session - * ended without producing a result" because `Thread.join` on a never-started - * thread is a no-op. The [[ensureStarted]] guard at every public entry point - * shouts loudly when a subclass forgets. - */ - protected def start(): Unit = - // Stderr first so a synchronous-finishing reader (pre-populated fake - // queues, etc.) still sees a non-null `stderrThread` if it reaches - // `onFinalize` immediately. - stderrThread.start() - readerThread.start() - // Drainer spawn lands here so the subclass's `askUser` override has - // already initialized by the time we read it. - askUser.foreach(r => startAskUserDrainer(r.bridge)) - - /** Spin up a daemon thread that bridges an [[AskUserBridge]] into the - * conversation's event stream: each pending question becomes a - * `ConversationEvent.UserQuestion` whose `respond` closure delivers the - * user's typed answer back to the blocked MCP handler. Called from [[start]] - * for every conversation whose [[askUser]] is `Some`; the thread exits - * cleanly when `bridge.close()` raises `ChannelClosedException` from - * `nextQuestion()` — driven by [[finalizeLoop]]. - */ - private def startAskUserDrainer(bridge: AskUserBridge): Unit = - val t = new Thread( - () => - try - while !Thread.currentThread().isInterrupted do - val q = bridge.nextQuestion() - eventQueue.enqueue( - ConversationEvent.UserQuestion(q.question, q.respond) - ) - catch - // Bridge closure surfaces as ChannelClosedException; exit - // quietly. NonFatal so an InterruptedException still propagates - // if something else interrupts the thread. - case NonFatal(_) => (), - s"$backendName-conversation-ask-user" - ) - t.setDaemon(true) - t.start() - - /** Fail loudly if a subclass constructor reached one of the public methods - * without calling [[start]] — the symptom would otherwise be a silent - * "session ended without producing a result". Cheap NEW-state probe. - */ - private def ensureStarted(label: String): Unit = - if readerThread.getState == Thread.State.NEW then - throw new IllegalStateException( - s"$backendName conversation: $label called before start() — " + - "subclass constructor likely forgot to call start() at the end." - ) - - // --- Conversation surface --- - - def events: Iterator[ConversationEvent] = - ensureStarted("events") - eventQueue.iterator - - def awaitResult(): Either[OrcaInteractiveCancelled, LlmResult[B]] = - ensureStarted("awaitResult") - readerThread.join() - outcomeRef.get() match - case Some(Outcome.Success(r)) => Right(r) - case Some(Outcome.Cancelled()) => Left(new OrcaInteractiveCancelled()) - // A failure here means the turn ran (the session is registered) and then - // errored — tag it non-retryable so the autonomous retry loop doesn't - // reopen the locked session id. See [[AgentTurnFailed]]. - case Some(Outcome.Failed(e: AgentTurnFailed)) => throw e - case Some(Outcome.Failed(e)) => - throw new AgentTurnFailed( - Option(e.getMessage).filter(_.nonEmpty).getOrElse(e.toString) - ) - case None => - throw new OrcaFlowException( - s"$backendName interactive session ended without producing a result" - ) - - def cancel(): Unit = - if cancelled.compareAndSet(false, true) then - source.interrupt() - // The reader loop sees EOF on `source.lines` and finalises the - // outcome on its own — no need to touch the queue from here. - - /** Settle the turn with `result`, then interrupt the source so the reader - * loop reaches EOF. For drivers whose stream stays open after the turn - * (SSE), this is how the turn terminates. Setting the outcome **before** the - * interrupt is load-bearing: closing the source makes the blocked read in - * [[readLoop]] throw, which would otherwise CAS a failure outcome — harmless - * only because the success is already in place. - */ - protected def succeedWith(result: LlmResult[B]): Unit = - val _ = outcomeRef.compareAndSet(None, Some(Outcome.Success(result))) - source.interrupt() - - /** Settle the turn as a failure, then interrupt the source (see - * [[succeedWith]] for the ordering rationale). - */ - protected def failWith(error: Throwable): Unit = - val _ = outcomeRef.compareAndSet(None, Some(Outcome.failed(error))) - source.interrupt() - - // --- Hooks for backend implementations --- - - /** Process a single line of stdout. Implementations parse the protocol - * message and translate to [[ConversationEvent]] enqueues (and/or - * [[outcomeRef]] updates). Exceptions thrown here are caught by the base - * loop and surfaced as a generic parse-error Error event — backends don't - * need their own try/catch. - */ - protected def handleLine(line: String): Unit - - /** Process one line of stderr. Default: enqueue as an Error event with the - * backend name as a prefix. Override to filter known- noise lines or to - * apply different formatting. - */ - protected def handleStderr(line: String): Unit = - if line.trim.nonEmpty then - eventQueue.enqueue(ConversationEvent.Error(s"$backendName: $line")) - - /** Hook called inside `finalizeLoop` **before** the failure outcome is - * computed, so subclasses can drain any background streams whose buffered - * state [[diagnosticContext]] / [[cleanExitWithoutResult]] depend on (codex - * joins its stderr-drain thread here so the buffered lines reach the - * exception message). Also where backends release session-scoped resources - * (claude closes its MCP bridge + Netty server). Default: no-op. - */ - protected def onFinalize(): Unit = () - - /** The exception used when the subprocess exits with code 0 without having - * sent a terminal protocol message. Default: a generic `OrcaFlowException` - * whose message [[appendContext]]-folds [[diagnosticContext]]. Backends may - * override outright if they prefer a different framing. - */ - protected def cleanExitWithoutResult(): Throwable = - new OrcaFlowException( - appendContext( - s"$backendName exited cleanly but never sent a terminal message" - ) - ) - - /** Optional context the base layer folds into the non-zero-exit / - * clean-exit-without-result failure messages, so noop-listener callers - * (programmatic invocations, tests) still get something useful in the thrown - * exception even when no live listener observed the in-stream - * `ConversationEvent.Error` events. Default `None`; backends override to - * attach buffered stderr or similar. The base layer owns the formatting — - * overrides return just the payload, never the separator. - */ - protected def diagnosticContext: Option[String] = None - - /** Fold the [[diagnosticContext]] payload (if any) onto a failure-message - * base. Centralised so every consumer gets the same framing — newline + - * two-space-indented payload — and overrides don't have to remember to - * include a leading newline. Subclasses that override - * [[cleanExitWithoutResult]] to set their own message body should call this - * so the diagnostic context still flows through. - */ - protected def appendContext(base: String): String = - diagnosticContext.fold(base)(ctx => s"$base\n $ctx") - - // --- Internals --- - - private def readLoop(): Unit = - try - for line <- source.lines do - debugLog("stdout", line) - if !cancelled.get() then - try handleLine(line) - catch - case e: Exception => - eventQueue.enqueue( - ConversationEvent.Error( - s"Failed to parse $backendName line: ${e.getMessage}" - ) - ) - catch - case NonFatal(e) => - debugLog("stdout-error", e.toString) - val _ = outcomeRef.compareAndSet(None, Some(Outcome.failed[B](e))) - finally finalizeLoop() - - private def stderrLoop(): Unit = - try - for line <- source.errorLines do - debugLog("stderr", line) - handleStderr(line) - catch - case NonFatal(t) => - // stderr draining is best-effort — the main thread doesn't - // depend on it. Surface the swallowed throwable under - // ORCA_DEBUG so a real bug isn't masked. - debugLog("stderr-error", s"${t.getClass.getName}: ${t.getMessage}") - - protected def debugLog(channel: String, line: String): Unit = - if OrcaDebug.streamTrace then - System.err.println(s"[orca-debug $backendName-$channel] $line") - - /** Access to the stderr-drain thread for backends whose [[onFinalize]] needs - * to wait for stderr to flush. - */ - protected def stderrDrainThread: Thread = stderrThread - - private def finalizeLoop(): Unit = - // 1. Subclass hook — typically drains background streams whose buffered - // state `diagnosticContext` / `cleanExitWithoutResult` depend on - // (codex joins its stderr-drain thread here). - onFinalize() - // 2. Close the ask_user resource bundle if one was wired. Happens after - // `onFinalize` so any subclass cleanup that might depend on the - // bridge / MCP server runs first; in practice neither backend does. - // `AskUserSession.close` handles ordering (bridge → server → - // extras) and swallows close-time failures. - askUser.foreach(_.close()) - val finalOutcome: Outcome[B] = outcomeRef.get() match - case Some(existing) => existing - case None if cancelled.get() => Outcome.cancelled[B] - case None => outcomeFromExit(source.tryExitCode) - val _ = outcomeRef.compareAndSet(None, Some(finalOutcome)) - eventQueue.close() - - private def outcomeFromExit(exitCode: Option[Int]): Outcome[B] = - exitCode match - case Some(0) => Outcome.failed[B](cleanExitWithoutResult()) - case Some(code) => - Outcome.failed[B]( - new OrcaFlowException( - appendContext(s"$backendName exited with code $code") - ) - ) - case None => Outcome.cancelled[B] - -private[orca] object StreamConversation: - - /** Internal outcome of the session as the reader sees it. Modelled as a - * sealed trait + cases (rather than an `enum`) because `Cancelled` and - * `Failed` are backend-agnostic — using `Nothing` as their `B` makes them - * assignable to any `Outcome[B]` while the pattern match still narrows - * `Success`'s `B` correctly. - * - * `Outcome` is invariant in `B` because `LlmResult[B]` is invariant (the - * phantom `B` in `SessionId[B]` etc. is meant to be exact); `Cancelled` and - * `Failed` get a wide-bounded `Outcome[B]` via the `Outcome.cancelled` / - * `Outcome.failed` smart constructors below. - */ - sealed trait Outcome[B <: BackendTag] - object Outcome: - final case class Success[B <: BackendTag](result: LlmResult[B]) - extends Outcome[B] - final case class Cancelled[B <: BackendTag]() extends Outcome[B] - final case class Failed[B <: BackendTag](error: Throwable) - extends Outcome[B] - - def cancelled[B <: BackendTag]: Outcome[B] = Cancelled[B]() - def failed[B <: BackendTag](error: Throwable): Outcome[B] = Failed[B](error) - - /** Default cap on in-flight unread `ConversationEvent`s. Producers block on - * `put` once full so backpressure flows back into the subprocess pipe - * (Claude pauses, OS pipe buffer fills, claude blocks). Picked empirically: - * large enough that a chatty turn doesn't stall, small enough that a slow - * consumer (user on a long readline) doesn't accumulate unbounded memory. - */ - val DefaultEventQueueCapacity: Int = 1024 - - /** Blocking queue + single-consumer iterator. `close()` signals end-of-stream - * to whichever thread is iterating. - * - * Bounded by `capacity` (default [[DefaultEventQueueCapacity]]) so a slow - * consumer applies backpressure to the producer — `enqueue` uses `put`, - * which blocks once the queue is full. - */ - final class EventQueue(capacity: Int = DefaultEventQueueCapacity): - private val queue = - new LinkedBlockingQueue[Option[ConversationEvent]](capacity) - - def enqueue(event: ConversationEvent): Unit = queue.put(Some(event)) - - def close(): Unit = queue.put(None) - - val iterator: Iterator[ConversationEvent] = new Iterator[ConversationEvent]: - // Single-consumer per the Conversation contract; a plain `var` - // with a null sentinel is enough. The three states are: - // null — nothing peeked yet (will block on next `hasNext`) - // Some(e) — peeked event, returned by next `next()` - // None — stream closed, `hasNext` stays false forever - private var peeked: Option[ConversationEvent] = null - - def hasNext: Boolean = - if peeked == null then peeked = queue.take() - peeked.isDefined - - def next(): ConversationEvent = - if !hasNext then throw new NoSuchElementException("event stream closed") - val value = peeked.get - peeked = null - value diff --git a/tools/src/main/scala/orca/backend/StreamSource.scala b/tools/src/main/scala/orca/backend/StreamSource.scala index 47e2b2df..a380cb96 100644 --- a/tools/src/main/scala/orca/backend/StreamSource.scala +++ b/tools/src/main/scala/orca/backend/StreamSource.scala @@ -2,7 +2,7 @@ package orca.backend import orca.subprocess.PipedCliProcess -/** The line-oriented source a [[StreamConversation]] drives. +/** The line-oriented source a [[orca.backend.ForkedConversation]] drives. * * Abstracts the four things the driver needs from its producer — a primary * line stream, an optional secondary diagnostic stream, a way to stop it, and @@ -33,6 +33,14 @@ private[orca] trait StreamSource: */ def interrupt(): Unit + /** Guaranteed backstop after [[interrupt]]: SIGKILL the subprocess / close + * the connection so [[lines]] always terminates even if the graceful + * interrupt didn't take. Default delegates to [[interrupt]] (sufficient for + * sources whose interrupt already hard-closes); the subprocess source + * overrides it. Must tolerate calls from any thread and more than once. + */ + def destroyForcibly(): Unit = interrupt() + /** Terminal status once [[lines]] has ended: `Some(0)` clean, `Some(n)` * non-zero failure, `None` unknown/aborted. A subprocess reports its exit * code; a stream that merely closed reports `Some(0)` (a clean end with no @@ -47,4 +55,5 @@ private[orca] object StreamSource: def lines: Iterator[String] = process.stdoutLines def errorLines: Iterator[String] = process.stderrLines def interrupt(): Unit = process.sendSigInt() + override def destroyForcibly(): Unit = process.destroyForcibly() def tryExitCode: Option[Int] = process.tryExitCode diff --git a/tools/src/main/scala/orca/backend/SubprocessSpawn.scala b/tools/src/main/scala/orca/backend/SubprocessSpawn.scala new file mode 100644 index 00000000..187f40c8 --- /dev/null +++ b/tools/src/main/scala/orca/backend/SubprocessSpawn.scala @@ -0,0 +1,53 @@ +package orca.backend + +import orca.OrcaFlowException +import orca.subprocess.PipedCliProcess + +import scala.util.control.NonFatal + +/** The two-level spawn-and-build teardown every subprocess stream backend + * (claude/codex/gemini/pi) needs, factored out so the resource-leak handling — + * the part most dangerous to get subtly wrong — lives in one place. + * + * The lifecycle: + * + * - `spawn` builds the argv (system-prompt files, MCP config, the + * fresh-vs-resume dispatch lookup) and launches the process. If it throws, + * the process never came up, so only `resources` are released. + * - `build` wraps the live process in a [[Conversation]] (writing the + * opening turn, closing stdin, etc.). If it throws after the spawn, the + * child is SIGINT-ed (so it doesn't linger) and the failure is rethrown as + * "Failed to open <sessionLabel> session". + * - `resources` are the session-scoped `AutoCloseable`s (the ask_user + * bundle, pi's temp files) the conversation takes ownership of on success + * — on the happy path they ride through the conversation's `onFinalize`, + * so this closes them (in reverse) only when spawn or build fails. + * By-name, because pi accumulates them while `spawn` builds the argv; it + * is evaluated only on the failure path. + * + * `sessionLabel` is the backend's own descriptor for the failure message + * ("claude stream-json", "codex", "gemini", "pi RPC") — deliberately not the + * bare backend name, which is pinned by tests. + */ +private[orca] object SubprocessSpawn: + + def open[C]( + sessionLabel: String, + resources: => List[AutoCloseable] + )(spawn: => PipedCliProcess)(build: PipedCliProcess => C): C = + try + val process = spawn + try build(process) + catch + case e: Exception => + // SIGINT the process; the outer catch releases `resources`. + process.sendSigInt() + throw OrcaFlowException( + s"Failed to open $sessionLabel session: ${e.getMessage}" + ) + catch + case e: Throwable => + resources.reverseIterator.foreach: r => + try r.close() + catch case NonFatal(_) => () + throw e diff --git a/tools/src/main/scala/orca/backend/SystemPromptComposer.scala b/tools/src/main/scala/orca/backend/SystemPromptComposer.scala index 23fb2fe7..b5fc92ed 100644 --- a/tools/src/main/scala/orca/backend/SystemPromptComposer.scala +++ b/tools/src/main/scala/orca/backend/SystemPromptComposer.scala @@ -1,9 +1,9 @@ package orca.backend -import orca.llm.{LlmConfig, ToolSet} +import orca.agents.{AgentConfig, ToolSet} /** Shared helper for assembling a backend-agnostic "system prompt body" from - * the configured [[LlmConfig.systemPrompt]], an optional caller-supplied + * the configured [[AgentConfig.systemPrompt]], an optional caller-supplied * `extraHint` (typically the shared `ask_user` MCP hint on interactive calls), * and the standing [[RuntimeOwnsGit]] rule. Concatenates non-empty pieces with * a blank line between them. @@ -24,8 +24,8 @@ private[orca] object SystemPromptComposer: * `git.diff()` empty so `reviewAndFixLoop`'s reviewer selection sees no * changed files and runs no reviewers. Omitted on read-only turns (planning, * triage, reviewer selection — they can't write anyway) and on - * [[LlmConfig.selfManagedGit]] turns (the caller's explicit escape hatch via - * `llm.withSelfManagedGit`). Otherwise an invariant of orca's + * [[AgentConfig.selfManagedGit]] turns (the caller's explicit escape hatch + * via `agent.withSelfManagedGit`). Otherwise an invariant of orca's * runtime-owns-git model, applied on top of any `withSystemPrompt`. */ val RuntimeOwnsGit: String = @@ -35,7 +35,7 @@ private[orca] object SystemPromptComposer: "the right points." def combine( - config: LlmConfig, + config: AgentConfig, extraHint: Option[String] = None ): Option[String] = val gitRule = @@ -51,7 +51,7 @@ private[orca] object SystemPromptComposer: * unchanged when nothing applies. */ def foldIntoPrompt( - config: LlmConfig, + config: AgentConfig, userPrompt: String, extraHint: Option[String] = None ): String = diff --git a/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala b/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala index f4053c23..23824212 100644 --- a/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala +++ b/tools/src/main/scala/orca/backend/mcp/AskUserMcpServer.scala @@ -9,9 +9,12 @@ import sttp.tapir.server.netty.sync.NettySyncServer import scala.concurrent.duration.{DurationInt, FiniteDuration} /** Input shape of the `ask_user` MCP tool. The agent fills in `question`; we - * hand the typed answer back as the tool result. + * hand the typed answer back as the tool result. `private[mcp]` — only the + * handler in this file references it (the server's `ServerName`/`ToolSlug`/ + * `ToolTimeout`/`Hint` stay `private[orca]` because the backends consult + * them). */ -private[orca] case class AskUserInput(question: String) derives Codec, Schema +private[mcp] case class AskUserInput(question: String) derives Codec, Schema /** Boots a tiny MCP HTTP server exposing the `ask_user` tool. The handler * closes over an [[AskUserBridge]] — each tool invocation enqueues the diff --git a/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala b/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala index 9a1b72f7..29db62c9 100644 --- a/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala +++ b/tools/src/main/scala/orca/backend/mcp/AskUserSession.scala @@ -21,7 +21,7 @@ private[orca] case class AskUserSession( bridge: AskUserBridge, server: AskUserMcpServer, extras: List[AutoCloseable] -): +) extends AutoCloseable: import AskUserSession.swallow def close(): Unit = diff --git a/tools/src/main/scala/orca/events/OrcaEvent.scala b/tools/src/main/scala/orca/events/OrcaEvent.scala index 26e619dc..03e13c11 100644 --- a/tools/src/main/scala/orca/events/OrcaEvent.scala +++ b/tools/src/main/scala/orca/events/OrcaEvent.scala @@ -1,6 +1,6 @@ package orca.events -import orca.llm.Model +import orca.agents.Model /** Flow-level event fanned out to every registered [[OrcaListener]]. Covers * stage transitions, tool invocations, token usage, structured results, and @@ -26,13 +26,13 @@ enum OrcaEvent: /** Token usage for a single LLM call, attributed along two independent axes: * - * - `agent` is the [[LlmTool.name]] that issued the call. For reviewer + * - `agent` is the [[Agent.name]] that issued the call. For reviewer * agents this carries the reviewer identity (`abstraction`, * `performance`, …); for the main coding agent it's `claude` / `codex` * (or whatever the script renamed it to via `withName`). * - `model` is the concrete model the backend reports it actually served * the call with. `None` when the response didn't carry it and no model - * was pinned via `LlmConfig.model`. + * was pinned via `AgentConfig.model`. * * `CostTracker` summarises usage along both axes — by-agent shows where the * tokens were spent, by-model shows which models cost what. diff --git a/tools/src/main/scala/orca/events/Usage.scala b/tools/src/main/scala/orca/events/Usage.scala index b4636607..40966856 100644 --- a/tools/src/main/scala/orca/events/Usage.scala +++ b/tools/src/main/scala/orca/events/Usage.scala @@ -2,14 +2,21 @@ package orca.events /** Token + cost accounting for one or more LLM calls. * - * `inputTokens` / `outputTokens` are the totals as billed by the backend. - * `cachedInputTokens` is the sub-portion of `inputTokens` served from prompt - * cache (cache_creation + cache_read for Claude, `cached_input_tokens` for - * codex); `reasoningOutputTokens` is the sub-portion of `outputTokens` that - * the model spent on internal reasoning (codex / o-series). Both sub-counts - * are non-cumulative breakdowns and are surfaced separately so callers can - * report cache hit ratios and reasoning overhead without doing the maths - * themselves. + * **Normalisation contract.** All backends map onto the same axes so that + * summing `Usage` across calls and across backends is apples-to-apples: + * + * - `inputTokens` is the TOTAL prompt tokens, **inclusive** of any served + * from prompt cache. `outputTokens` is the total completion tokens. + * - `cachedInputTokens` is the cache-served sub-portion (`cachedInputTokens + * <= inputTokens`); `reasoningOutputTokens` is the internal-reasoning + * sub-portion of `outputTokens` (codex / o-series). Both are + * non-cumulative breakdowns, surfaced so callers can report cache-hit and + * reasoning ratios without re-deriving them. + * + * Backends reach this contract from different wire shapes, so a NEW backend + * must fold any cache-served tokens INTO `inputTokens` rather than report them + * alongside it. The per-backend arithmetic (and why it differs) is documented + * at each driver's `Usage(...)` construction site. */ case class Usage( inputTokens: Long, diff --git a/tools/src/main/scala/orca/llm/JsonData.scala b/tools/src/main/scala/orca/llm/JsonData.scala deleted file mode 100644 index 85920167..00000000 --- a/tools/src/main/scala/orca/llm/JsonData.scala +++ /dev/null @@ -1,52 +0,0 @@ -package orca.llm - -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - CodecMakerConfig, - ConfiguredJsonValueCodec -} -import sttp.tapir.Schema - -import scala.deriving.Mirror - -/** Bundles a tapir `Schema` and a jsoniter-scala `ConfiguredJsonValueCodec` for - * a type. Flow scripts use `derives JsonData` on case classes that travel in - * and out of LLM calls as structured JSON. - * - * Scripts must import via `import orca.{*, given}` — Scala 3's plain wildcard - * imports exclude givens, and `derives JsonData` on a case class with nested - * case-class fields needs the forwarder givens below in scope. - */ -trait JsonData[A]: - def schema: Schema[A] - def codec: ConfiguredJsonValueCodec[A] - -object JsonData: - - /** Stricter-than-default jsoniter config: missing `List` / `Set` / `Map` - * fields fail to parse rather than defaulting to empty, so an agent reply - * with the right overall shape but the wrong fields can't masquerade as a - * "success with no content". - */ - inline def strictCodecConfig: CodecMakerConfig = - CodecMakerConfig - .withRequireCollectionFields(true) - .withTransientEmpty(false) - - def apply[A]( - schemaInstance: Schema[A], - codecInstance: ConfiguredJsonValueCodec[A] - ): JsonData[A] = - new JsonData[A]: - val schema: Schema[A] = schemaInstance - val codec: ConfiguredJsonValueCodec[A] = codecInstance - - inline def derived[A](using Mirror.Of[A]): JsonData[A] = - apply( - Schema.derived[A], - ConfiguredJsonValueCodec.derived[A](using strictCodecConfig) - ) - -given schemaFromJsonData[A](using jd: JsonData[A]): Schema[A] = jd.schema - -given codecFromJsonData[A](using jd: JsonData[A]): ConfiguredJsonValueCodec[A] = - jd.codec diff --git a/tools/src/main/scala/orca/llm/LlmTool.scala b/tools/src/main/scala/orca/llm/LlmTool.scala deleted file mode 100644 index c5c47cd1..00000000 --- a/tools/src/main/scala/orca/llm/LlmTool.scala +++ /dev/null @@ -1,159 +0,0 @@ -package orca.llm - -/** An LLM adapter usable from flow scripts — the handle you call from a - * `flow(...)` block (`claude`, `codex`, etc.). Two paths to invoke the model: - * - * - **`autonomous`** — free-form text, no structured output, no JSON schema - * wrapping. The agent's reply is returned verbatim. - * - **`resultAs[O]`** — fix the output type and obtain a call object that - * exposes both `autonomous` and `interactive` modes. - * - * Each mode has a single `run(input, session = …, config = …)` method that - * always returns `(SessionId[B], output)`. Pre-allocate a session with - * [[newSession]] and pass it across calls to keep one conversation alive; omit - * the argument to get a fresh one-shot session per call. - * - * The API never hides the autonomous-vs-interactive choice behind a default — - * it's always visible at the call site as the leftmost segment after the tool - * / call gateway. - * - * Parameterized by the concrete `BackendTag` so session ids and results carry - * the backend identity at the type level. - */ -trait LlmTool[B <: BackendTag]: - def name: String - - /** Free-form text autonomous calls. Use this when the agent's reply is prose - * / code / anything that doesn't need to parse as a structured `O`. For - * structured output (and the interactive-conversation path), use - * [[resultAs]]. - */ - def autonomous: AutonomousTextCall[B] - - /** Fix the output type of a structured call and obtain a gateway with both - * `autonomous` and `interactive` modes. `O` needs a `JsonData[O]` — `derives - * JsonData` on a case class is the normal way to provide one. - * - * An `Announce[O]` is also required; the library's default given returns - * `None` (no auto-announce), so callers don't need to do anything unless - * they want a friendly summary on the channel. See [[Announce]]. - */ - def resultAs[O: JsonData: Announce]: LlmCall[B, O] - - def withConfig(config: LlmConfig): LlmTool[B] - def withSystemPrompt(prompt: String): LlmTool[B] - def withName(name: String): LlmTool[B] - - /** Return a sibling tool whose config pins [[LlmConfig.tools]] to `tools` — - * the capability tier (see [[ToolSet]]). The primitive behind - * [[withReadOnly]] and [[withNetworkOnly]]; preserves the rest of the tool's - * config (model, system prompt, autoApprove). - */ - def withTools(tools: ToolSet): LlmTool[B] - - /** Sibling tool restricted to read-only tools ([[ToolSet.ReadOnly]]): no - * edits, no shell. Used by planning and review helpers so e.g. - * `claude.opus.withReadOnly` keeps the opus pin while gating writes. - */ - def withReadOnly: LlmTool[B] = withTools(ToolSet.ReadOnly) - - /** Sibling tool restricted to reads plus network ([[ToolSet.NetworkOnly]]) — - * for planner turns that must read an issue/PR. See [[ToolSet]] for the - * per-backend no-edit guarantee (hard on most; prompt-only on pi / codex). - */ - def withNetworkOnly: LlmTool[B] = withTools(ToolSet.NetworkOnly) - - /** Return a sibling tool that manages git itself — flips - * [[LlmConfig.selfManagedGit]] on, suppressing the standing "runtime owns - * git" rule the runtime otherwise injects (don't `git commit`/`push`/branch; - * leave edits in the working tree). Use only for a flow that genuinely wants - * the agent to drive git; the default keeps git runtime-owned. - * - * Unlike [[withReadOnly]] this carries a no-op default (returns `this`), so - * a custom `LlmTool` that forgets to wire it simply keeps the safe - * runtime-owns-git behaviour rather than silently granting the escape hatch. - */ - def withSelfManagedGit: LlmTool[B] = this - - /** Mint a fresh session id you can pass to `.run(...)` across multiple calls. - * The first call with this id starts the session; subsequent calls resume - * it. Lets flow scripts hold a stable `val session = claude.newSession` - * instead of threading a `var Option[SessionId]` through the loop. - * - * Default implementation generates a UUID via [[SessionId.fresh]]; backends - * that need a different format (or eager server-side allocation) override. - */ - def newSession: SessionId[B] = SessionId.fresh[B] - -trait ClaudeTool extends LlmTool[BackendTag.ClaudeCode.type]: - /** Pin the Claude model for subsequent calls, overriding `LlmConfig.model`. - * Typical usage: `claude.haiku.autonomous.run("summarize this")._2` for a - * cheap fast one-shot call (discard the returned session id). - */ - def haiku: ClaudeTool - def sonnet: ClaudeTool - def opus: ClaudeTool - def fable: ClaudeTool - - /** Set the read-only network allowlist used on [[ToolSet.NetworkOnly]] turns - * (claude `--allowedTools` syntax, e.g. `Bash(gh api:*)`, `WebFetch`). - * Claude-specific, so it's here rather than on `LlmConfig`; defaults to - * `ClaudeBackend.DefaultNetworkTools`. Pass it before handing the tool to a - * planning helper: `claude.opus.withNetworkTools(Seq("WebFetch"))`. - */ - def withNetworkTools(tools: Seq[String]): ClaudeTool - -trait CodexTool extends LlmTool[BackendTag.Codex.type]: - def mini: CodexTool - -/** OpenCode spans providers, so its model accessors are provider-prefixed (the - * prefix keeps the vendor explicit at the call site). [[withModel]] takes any - * `provider/model` id — including self-hosted ones, e.g. `ollama/llama3.1`. - */ -trait OpencodeTool extends LlmTool[BackendTag.Opencode.type]: - def anthropicOpus: OpencodeTool - def anthropicSonnet: OpencodeTool - def anthropicHaiku: OpencodeTool - def openaiGpt5: OpencodeTool - def openaiGpt5Codex: OpencodeTool - def openaiGpt5Mini: OpencodeTool - - /** Pin any `provider/model` id (e.g. `ollama/llama3.1`, `myhost/qwen-coder`). - */ - def withModel(providerModel: String): OpencodeTool - - /** Two-arg form of [[withModel]], e.g. `withModel("ollama", "llama3.1")`. The - * default joins with `/`; the concrete tool validates the parts. - */ - def withModel(provider: String, modelId: String): OpencodeTool = - withModel(s"$provider/$modelId") - -trait PiTool extends LlmTool[BackendTag.Pi.type] - -trait GeminiTool extends LlmTool[BackendTag.Gemini.type]: - /** Pin the cheap-and-fast Gemini Flash model for subsequent calls, overriding - * `LlmConfig.model`. Bare `gemini` runs on Gemini Pro (pinned in the runtime - * wiring); `gemini.flash` opts down for cheap one-shots. - */ - def flash: GeminiTool - -/** Free-form text autonomous calls — the `LlmTool.autonomous` shape. Single - * method: pass a [[SessionId]] (typically from [[LlmTool.newSession]] or the - * default fresh one) and the library starts the session on the first call, - * resumes it on subsequent calls. Returns the (stable) session id so the - * caller can pass it back unchanged. - */ -trait AutonomousTextCall[B <: BackendTag]: - /** Run the agent on `prompt`. When `emitPrompt` is true (the default), fires - * an `OrcaEvent.UserPrompt` carrying `prompt` so listeners can surface - * what's being asked; framework-internal callers that produce many - * near-identical prompts in quick succession (e.g. the per-task reviewer - * fan-out) pass `false` to keep the event log focused. Other events - * (`ToolUse`, `AssistantMessage`, `TokensUsed`, etc.) fire regardless. - */ - def run( - prompt: String, - session: SessionId[B] = SessionId.fresh[B], - config: LlmConfig = LlmConfig.default, - emitPrompt: Boolean = true - ): (SessionId[B], String) diff --git a/tools/src/main/scala/orca/subprocess/CliProcess.scala b/tools/src/main/scala/orca/subprocess/CliProcess.scala index 09fbf4f0..da36e7c5 100644 --- a/tools/src/main/scala/orca/subprocess/CliProcess.scala +++ b/tools/src/main/scala/orca/subprocess/CliProcess.scala @@ -5,11 +5,22 @@ trait CliProcess: def isAlive: Boolean def waitForExit(): Int - /** SIGINT this process and any descendants. Default = just this process; - * override where a launch wrapper (e.g. `ollama launch opencode`) may fork - * the real process — a single-PID SIGINT would orphan it. + /** Forcibly terminate this process (SIGKILL for the OS-backed process; close + * the pipes for fakes), the guaranteed backstop a [[StreamSource]] uses + * after a graceful `sendSigInt` so a reader blocked on stdout always + * unblocks. Must tolerate calls from any thread and more than once; on the + * normal path the process has already exited, making this a no-op. */ - def sendSigIntTree(): Unit = sendSigInt() + def destroyForcibly(): Unit + + /** Forcibly terminate this process AND any descendants. Default = just this + * process; override where a launch wrapper (e.g. `ollama launch opencode`) + * forks the real process, which inherits the stdout/stderr pipe fds: a + * single-PID kill would orphan it and leave those write-ends open, so a + * reader blocked on the pipe would never see EOF. Same call-anywhere / + * idempotent contract as [[destroyForcibly]]. + */ + def destroyForciblyTree(): Unit = destroyForcibly() /** A spawned process whose stdin / stdout / stderr are connected to pipes the * caller controls. The backend writes the opening user turn (or any further diff --git a/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala b/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala index 43f72f68..6aa4c4ab 100644 --- a/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala +++ b/tools/src/main/scala/orca/subprocess/OsProcCliRunner.scala @@ -3,7 +3,6 @@ package orca.subprocess import org.slf4j.LoggerFactory import scala.jdk.CollectionConverters.given -import scala.jdk.StreamConverters.* /** Runs external commands via os-lib. `check = false` is intentional — callers * inspect `exitCode` and `stderr` rather than handling thrown exceptions, @@ -93,20 +92,24 @@ private final class OsPipedSubProcess( def sendSigInt(): Unit = val _ = QuietProc.call(Seq("kill", "-INT", sub.wrapped.pid.toString)) - override def sendSigIntTree(): Unit = - // Descendants first, then the root: if the spawned process is a launch - // wrapper that forked the real `opencode serve`, SIGINT-ing only the root - // PID would leave the server orphaned. Snapshot is best-effort. - val handle = sub.wrapped.toHandle - val pids = - (handle.descendants().toScala(List) :+ handle) - .filter(_.isAlive) - .map(_.pid.toString) - if pids.nonEmpty then - val _ = QuietProc.call(Seq("kill", "-INT") ++ pids) - def isAlive: Boolean = sub.isAlive() + def destroyForcibly(): Unit = + val _ = sub.wrapped.destroyForcibly() + + override def destroyForciblyTree(): Unit = + // Descendants first, then the root: a launch wrapper that forked the real + // `serve` child leaves it holding the inherited stdout/stderr pipe + // write-ends, so killing only the root PID would never EOF a blocked drain. + // `descendants()` is a best-effort snapshot of live parent→child linkage; it + // catches a wrapper that stays alive as the worker's parent (the `ollama + // launch` case). It would NOT catch a wrapper that double-forks and exits, + // reparenting the worker to init — that would need a process-group kill or a + // recorded worker PID. No current launcher does that. + val handle = sub.wrapped.toHandle + handle.descendants().forEach(h => { val _ = h.destroyForcibly() }) + val _ = handle.destroyForcibly() + def waitForExit(): Int = val _ = sub.join() sub.exitCode() diff --git a/tools/src/main/scala/orca/tools/FsTool.scala b/tools/src/main/scala/orca/tools/FsTool.scala index 38b56113..d03c65d2 100644 --- a/tools/src/main/scala/orca/tools/FsTool.scala +++ b/tools/src/main/scala/orca/tools/FsTool.scala @@ -1,5 +1,7 @@ package orca.tools +import orca.InStage + import java.nio.file.FileSystems /** Filesystem adapter usable from flow scripts — the handle behind the `fs` @@ -20,7 +22,7 @@ trait FsTool: */ def read(path: String): Option[String] - def write(path: String, content: String): Unit + def write(path: String, content: String)(using InStage): Unit def list(glob: String): List[String] /** `FsTool` implementation backed by os-lib. Path resolution and glob semantics @@ -34,7 +36,7 @@ private[orca] class OsFsTool(base: os.Path = os.pwd) extends FsTool: val p = resolve(path) if os.isFile(p) then Some(os.read(p)) else None - def write(path: String, content: String): Unit = + def write(path: String, content: String)(using InStage): Unit = os.write.over(resolve(path), content, createFolders = true) def list(glob: String): List[String] = diff --git a/tools/src/main/scala/orca/tools/GhJson.scala b/tools/src/main/scala/orca/tools/GhJson.scala new file mode 100644 index 00000000..a52c22b5 --- /dev/null +++ b/tools/src/main/scala/orca/tools/GhJson.scala @@ -0,0 +1,71 @@ +package orca.tools + +import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec +import com.github.plokhotnyuk.jsoniter_scala.macros.{ + ConfiguredJsonValueCodec, + JsonCodecMaker +} + +// Wire-format DTOs for the `gh`/`git` JSON output that `OsGitHubTool` parses. +// Kept separate from the public GitHubTool contract: these mirror the GitHub +// CLI/API shape and change for a different reason than the tool's own API. +// All `private[orca]` — internal to the tools layer, never part of the public +// surface (callers see `Comment`/`Issue`/`PrHandle`, not these). + +private[orca] case class GhCheck( + // CheckRun entries use `status`/`conclusion`; legacy commit-status entries + // use only `state`. Both are optional so a single codec handles both. + status: Option[String] = None, + conclusion: Option[String] = None, + state: Option[String] = None, + name: Option[String] = None +) derives ConfiguredJsonValueCodec + +private[orca] case class GhCheckRollup( + statusCheckRollup: List[GhCheck] +) derives ConfiguredJsonValueCodec + +private[orca] case class GhCommentJson( + body: String, + user: GhUserJson +) derives ConfiguredJsonValueCodec + +/** Comment JSON with its numeric id, used internally by [[OsGitHubTool]] to + * support the PATCH path in `upsertComment`. The id is a GitHub-issued int. + * Not exposed publicly — callers see the id-free [[Comment]] type. + */ +private[orca] case class GhIdentifiedCommentJson( + id: Long, + body: String, + user: GhUserJson +) derives ConfiguredJsonValueCodec + +/** Minimal PR fields returned by `gh pr list --json number,url`. Used by + * [[OsGitHubTool.findOpenPr]] to map a head branch to an open PR. Only the URL + * is needed: the owner/repo/number are extracted from it via [[PrUrlPattern]] + * so no separate `headRefName` field is required. + */ +private[orca] case class GhPrListJson( + number: Int, + url: String +) derives ConfiguredJsonValueCodec + +private[orca] case class GhUserJson(login: String) + derives ConfiguredJsonValueCodec + +private[orca] case class GhIssueJson( + title: String, + // Issues without a body return `null` from the API; the codec + // treats a missing key as None and a null literal as None too. + body: Option[String] = None, + user: GhUserJson, + state: String +) derives ConfiguredJsonValueCodec + +private[orca] given ghCommentListCodec: JsonValueCodec[List[GhCommentJson]] = + JsonCodecMaker.make +private[orca] given ghIdentifiedCommentListCodec + : JsonValueCodec[List[GhIdentifiedCommentJson]] = + JsonCodecMaker.make +private[orca] given ghPrListCodec: JsonValueCodec[List[GhPrListJson]] = + JsonCodecMaker.make diff --git a/tools/src/main/scala/orca/tools/GitHubTool.scala b/tools/src/main/scala/orca/tools/GitHubTool.scala index 39460281..54b66397 100644 --- a/tools/src/main/scala/orca/tools/GitHubTool.scala +++ b/tools/src/main/scala/orca/tools/GitHubTool.scala @@ -1,23 +1,21 @@ package orca.tools -import com.github.plokhotnyuk.jsoniter_scala.core.{ - JsonValueCodec, - readFromString -} -import com.github.plokhotnyuk.jsoniter_scala.macros.{ - ConfiguredJsonValueCodec, - JsonCodecMaker -} -import orca.OrcaFlowException +import com.github.plokhotnyuk.jsoniter_scala.core.readFromString +import orca.{InStage, OrcaFlowException} import orca.events.{OrcaEvent, OrcaListener} -import orca.subprocess.CliRunner +import orca.agents.JsonData +import orca.subprocess.{CliResult, CliRunner} import ox.sleep import ox.resilience.{ResultPolicy, RetryConfig, retry} import ox.scheduling.Schedule import scala.concurrent.duration.{DurationInt, FiniteDuration} -case class PrHandle(owner: String, repo: String, number: Int): +/** A handle to an open pull request. `derives JsonData` so `stage` can record + * and replay a `PrHandle` result — e.g. when a push-and-open-PR stage is the + * checkpoint before a CI wait (ADR 0018 §3.2). + */ +case class PrHandle(owner: String, repo: String, number: Int) derives JsonData: /** `<owner>/<repo>#<number>` — the canonical GitHub short-form. Used in * commit messages, PR descriptions (`Closes …`), and log output. */ @@ -125,13 +123,15 @@ final class NoChecksConfigured(grace: FiniteDuration) * writes PR comments, and polls GitHub's check-run status. */ trait GitHubTool: - def createPr(title: String, body: String): Either[PrCreateFailed, PrHandle] + def createPr(title: String, body: String)(using + InStage + ): Either[PrCreateFailed, PrHandle] /** Replace an existing PR's title and body. Used to refresh a PR opened with * a tentative description (e.g. when only a failing test had landed) once * later work — the fix — is pushed. */ - def updatePr(pr: PrHandle, title: String, body: String): Unit + def updatePr(pr: PrHandle, title: String, body: String)(using InStage): Unit /** Fetch the issue's title, body, author, and state. */ def readIssue(issue: IssueHandle): Issue @@ -149,13 +149,32 @@ trait GitHubTool: /** Post a top-level issue-style comment on a pull request (the comments the * GitHub UI shows under the description, not line-level review comments). */ - def writeComment(pr: PrHandle, body: String): Unit + def writeComment(pr: PrHandle, body: String)(using InStage): Unit /** Post a top-level comment on an issue. Used by assess-then-act flows to * surface a follow-up question / critique / rebuff back to the reporter when * no PR will be opened. */ - def writeComment(issue: IssueHandle, body: String): Unit + def writeComment(issue: IssueHandle, body: String)(using InStage): Unit + + /** Idempotent comment on a PR. Finds the first existing comment whose body + * contains `marker`, then updates it via a REST PATCH; if none is found, + * creates a new comment with `body` followed by `marker` on a separate line. + * The `marker` is an HTML comment the caller embeds (e.g. `<!-- + * orca:<hash>:<purpose> -->`) so a re-run can locate and update its own + * comment instead of duplicating it. Plain [[writeComment]] stays + * append-only. + */ + def upsertComment(pr: PrHandle, marker: String, body: String)(using + InStage + ): Unit + + /** Idempotent comment on an issue. Same find/update/create semantics as + * [[upsertComment(PrHandle, String, String)]]. + */ + def upsertComment(issue: IssueHandle, marker: String, body: String)(using + InStage + ): Unit /** Aggregate status of the checks attached to `pr`. * @@ -185,38 +204,6 @@ trait GitHubTool: noChecksGrace: FiniteDuration = 90.seconds ): Either[BuildWaitFailed, BuildStatus] -private[orca] case class GhCheck( - // CheckRun entries use `status`/`conclusion`; legacy commit-status entries - // use only `state`. Both are optional so a single codec handles both. - status: Option[String] = None, - conclusion: Option[String] = None, - state: Option[String] = None, - name: Option[String] = None -) derives ConfiguredJsonValueCodec - -private[orca] case class GhCheckRollup( - statusCheckRollup: List[GhCheck] -) derives ConfiguredJsonValueCodec - -private[orca] case class GhCommentJson( - body: String, - user: GhUserJson -) derives ConfiguredJsonValueCodec - -private[orca] case class GhUserJson(login: String) - derives ConfiguredJsonValueCodec - -private[orca] case class GhIssueJson( - title: String, - // Issues without a body return `null` from the API; the codec - // treats a missing key as None and a null literal as None too. - body: Option[String] = None, - user: GhUserJson, - state: String -) derives ConfiguredJsonValueCodec - -private[orca] given JsonValueCodec[List[GhCommentJson]] = JsonCodecMaker.make - /** GitHubTool implementation that shells out to the `gh` CLI via a `CliRunner`. * `waitForBuild` polls `buildStatus` every `pollInterval` until a terminal * outcome or the caller-supplied timeout expires. @@ -252,14 +239,14 @@ private[orca] class OsGitHubTool( private val PrUrlPattern = """https://github\.com/([^/]+)/([^/]+)/pull/(\d+)""".r - def createPr(title: String, body: String): Either[PrCreateFailed, PrHandle] = + def createPr(title: String, body: String)(using + InStage + ): Either[PrCreateFailed, PrHandle] = // Inspect exit code + stderr ourselves so we can split the recoverable // "branch already has a PR" / "no commits to push" cases out from - // genuine system failures. - val result = cli.run( - Seq("gh", "pr", "create", "--title", title, "--body", body), - cwd = workDir - ) + // genuine system failures — so this goes through `runGhResult` (which + // returns the raw result) rather than `ghRead`/`ghMutate` (which abort). + val result = runGhResult("pr", "create", "--title", title, "--body", body) if result.exitCode == 0 then val output = result.stdout.trim PrUrlPattern.findFirstMatchIn(output) match @@ -272,15 +259,56 @@ private[orca] class OsGitHubTool( s"Unexpected output from gh pr create: $output" ) else - val combined = (result.stdout + "\n" + result.stderr).toLowerCase - if combined.contains("already exists") then Left(new PrAlreadyExists) - else if combined.contains("no commits") || - combined.contains("must first push") - then Left(new NoCommitsToPr) - else - throw OrcaFlowException( - s"gh pr create failed (exit ${result.exitCode}): ${result.stderr}" - ) + val combined = result.stdout + "\n" + result.stderr + if OsGitHubTool.isPrAlreadyExists(combined) then + // R24: look up the existing open PR rather than failing — crash-safe + // for flows that may re-enter a "push + open PR" stage. + findOpenPr(currentBranchGit()) match + case Some(pr) => + events.onEvent(OrcaEvent.Step(s"Reusing existing PR: ${pr.url}")) + Right(pr) + case None => Left(new PrAlreadyExists) + else if OsGitHubTool.isNoCommitsToPr(combined) then + Left(new NoCommitsToPr) + else fail("gh pr create", result) + + /** Resolve the current branch name via `git rev-parse --abbrev-ref HEAD`. + * Used by [[createPr]] to pass the head branch to [[findOpenPr]]. + */ + private def currentBranchGit(): String = + val result = cli.run( + Seq("git", "rev-parse", "--abbrev-ref", "HEAD"), + cwd = workDir + ) + if result.exitCode == 0 then result.stdout.trim + else fail("git rev-parse", result) + + /** Find an open PR whose head branch matches `head`, using `gh pr list --head + * <head> --state open --json number,url`. Returns the first match, or `None` + * when no open PR is found. Uses [[ghRead]] (with retry) because this is an + * idempotent read. + * + * Matching on head-only: this suffices in practice because a branch can only + * have one open PR targeting any given base at a time, and `createPr` is + * called from within an orca-managed branch where hijacking a stacked PR is + * not expected. + */ + private def findOpenPr(head: String): Option[PrHandle] = + val output = ghRead( + "pr", + "list", + "--head", + head, + "--state", + "open", + "--json", + "number,url" + ) + readFromString[List[GhPrListJson]](output).headOption.flatMap: entry => + PrUrlPattern + .findFirstMatchIn(entry.url) + .map: m => + PrHandle(m.group(1), m.group(2), m.group(3).toInt) def readIssue(issue: IssueHandle): Issue = val output = ghRead( @@ -318,12 +346,12 @@ private[orca] class OsGitHubTool( readFromString[List[GhCommentJson]](output).map: c => Comment(author = c.user.login, body = c.body) - def updatePr(pr: PrHandle, title: String, body: String): Unit = + def updatePr(pr: PrHandle, title: String, body: String)(using InStage): Unit = // Use the REST API directly rather than `gh pr edit`: the latter runs a // GraphQL metadata query that selects `projectCards` before applying any // edit, which fails outright on repos where GitHub has sunset Projects // (classic). The REST PATCH endpoint doesn't touch projects. - val _ = gh( + val _ = ghMutate( "api", "-X", "PATCH", @@ -335,8 +363,8 @@ private[orca] class OsGitHubTool( ) events.onEvent(OrcaEvent.Step(s"Updated PR: ${pr.url}")) - def writeComment(pr: PrHandle, body: String): Unit = - val _ = gh( + def writeComment(pr: PrHandle, body: String)(using InStage): Unit = + val _ = ghMutate( "pr", "comment", pr.number.toString, @@ -346,8 +374,8 @@ private[orca] class OsGitHubTool( body ) - def writeComment(issue: IssueHandle, body: String): Unit = - val _ = gh( + def writeComment(issue: IssueHandle, body: String)(using InStage): Unit = + val _ = ghMutate( "issue", "comment", issue.number.toString, @@ -357,6 +385,74 @@ private[orca] class OsGitHubTool( body ) + def upsertComment(pr: PrHandle, marker: String, body: String)(using + InStage + ): Unit = + upsertCommentAt(pr.owner, pr.repo, pr.number, marker, body): + writeComment(pr, _) + + def upsertComment(issue: IssueHandle, marker: String, body: String)(using + InStage + ): Unit = + upsertCommentAt(issue.owner, issue.repo, issue.number, marker, body): + writeComment(issue, _) + + /** Shared upsert logic for both PR and issue targets. Fetches comments with + * their ids, finds the first comment containing `marker`, then either + * PATCHes that comment or delegates to `createFn` to post a new one. The + * body stored (both on create and update) is `<body>\n\n<marker>` so future + * re-runs can locate and update the same comment. + */ + private def upsertCommentAt( + owner: String, + repo: String, + number: Int, + marker: String, + body: String + )(createFn: String => Unit): Unit = + val markedBody = s"$body\n\n$marker" + fetchIdentifiedComments(owner, repo, number).find( + _.body.contains(marker) + ) match + case Some(existing) => + patchComment(owner, repo, existing.id, markedBody) + case None => + createFn(markedBody) + + /** Fetch comments for a PR/issue with their numeric ids. Returns an internal + * list used only by [[upsertCommentAt]]; ids are GitHub-issued ints and + * never leak into the public API. + */ + private def fetchIdentifiedComments( + owner: String, + repo: String, + number: Int + ): List[GhIdentifiedCommentJson] = + val output = ghRead( + "api", + "--paginate", + s"repos/$owner/$repo/issues/$number/comments" + ) + readFromString[List[GhIdentifiedCommentJson]](output) + + /** PATCH an existing issue/PR comment body via the REST API. `id` is the + * GitHub-issued numeric comment id returned by the comments endpoint. + */ + private def patchComment( + owner: String, + repo: String, + id: Long, + body: String + ): Unit = + val _ = ghMutate( + "api", + "-X", + "PATCH", + s"repos/$owner/$repo/issues/comments/$id", + "-f", + s"body=$body" + ) + def buildStatus(pr: PrHandle): BuildStatus = val output = ghRead( "pr", @@ -408,21 +504,48 @@ private[orca] class OsGitHubTool( loop(sawAnyCheck = false) - private def gh(args: String*): String = - val result = cli.run("gh" +: args, cwd = workDir) - if result.exitCode != 0 then - throw OrcaFlowException( - s"gh ${args.mkString(" ")} failed (exit ${result.exitCode}): ${result.stderr}" - ) + /** Run `gh` once and return the raw [[CliResult]]. The single point every gh + * invocation funnels through. Used directly only by [[createPr]], which + * inspects the exit code itself to split recoverable cases from failures; + * every other call goes through [[ghRead]] or [[ghMutate]]. + */ + private def runGhResult(args: String*): CliResult = + cli.run("gh" +: args, cwd = workDir) + + /** Run `gh` once, returning stdout or aborting on a non-zero exit. The shared + * abort-on-failure logic behind [[ghRead]] and [[ghMutate]] — NOT called + * directly, so the read-vs-mutate (and thus retry-vs-no-retry) choice is + * always explicit at the call site via one of those two wrappers. + */ + private def runGh(args: String*): String = + val result = runGhResult(args*) + if result.exitCode != 0 then fail(s"gh ${args.mkString(" ")}", result) result.stdout - /** Like [[gh]] but retries a transient failure under [[readRetryConfig]]. - * ONLY for idempotent reads (`api`/`pr view`); never wrap a mutating call - * (`pr create`, `pr comment`, `pr edit`) — a retry after a lost response - * would double the side effect (duplicate PR / comment). + /** Abort with a uniform `"<label> failed (exit N): <stderr>"` message for an + * unrecoverable CLI failure. Callers handle the EXPECTED non-zero exits (PR + * already exists, no commits) as `Left`s before reaching here. + */ + private def fail(label: String, result: CliResult): Nothing = + throw OrcaFlowException( + s"$label failed (exit ${result.exitCode}): ${result.stderr}" + ) + + /** Run an **idempotent read** (`api` GET, `pr view`, `pr list`), retrying a + * transient failure under [[readRetryConfig]]. Safe to retry precisely + * because it has no side effect. */ private def ghRead(args: String*): String = - retry(readRetryConfig)(gh(args*)) + retry(readRetryConfig)(runGh(args*)) + + /** Run a **mutating** `gh` call (`pr comment`, `pr edit`, …) exactly once — + * deliberately NOT retried: a retry after a lost response would double the + * side effect (duplicate comment / PR edit). Use [[ghRead]] for reads. The + * one-PR-per-branch idempotency for `pr create` is handled separately in + * [[createPr]] (it inspects the exit code itself), which is why it doesn't + * go through this wrapper. + */ + private def ghMutate(args: String*): String = runGh(args*) private[orca] object OsGitHubTool: @@ -434,6 +557,31 @@ private[orca] object OsGitHubTool: val defaultReadRetry: Schedule = Schedule.exponentialBackoff(1.second).maxRetries(4) + // --- Recoverable `gh pr create` stderr/stdout predicates --- + // + // gh has no machine-readable signal for "an open PR already exists" or "the + // branch has no commits", so `createPr` splits these recoverable cases from a + // system failure by matching gh's human-readable output (combined + // stdout+stderr). gh's messages are UI text, not a contract, so the matchers + // are centralised here — named, documented, unit-tested — and kept lenient so + // a wording tweak doesn't reclassify a recoverable failure as fatal. Each + // case-folds its input internally, so callers pass gh's output verbatim. + + /** True when `gh pr create` reported that an open PR already exists for the + * head branch — the case `createPr` resolves by reusing the existing PR + * (R24). Takes gh's combined stdout+stderr verbatim and case-folds itself. + */ + private[tools] def isPrAlreadyExists(combined: String): Boolean = + combined.toLowerCase.contains("already exists") + + /** True when `gh pr create` reported there is nothing to open a PR from (the + * branch has no commits ahead of base, or hasn't been pushed yet). + * Case-folds internally (see [[isPrAlreadyExists]]). + */ + private[tools] def isNoCommitsToPr(combined: String): Boolean = + val lower = combined.toLowerCase + lower.contains("no commits") || lower.contains("must first push") + private val StatusCompleted = "COMPLETED" private val SuccessfulConclusions = Set("SUCCESS", "NEUTRAL", "SKIPPED") private val LegacyStateSuccess = "SUCCESS" diff --git a/tools/src/main/scala/orca/tools/GitTool.scala b/tools/src/main/scala/orca/tools/GitTool.scala index a1799108..353fb48e 100644 --- a/tools/src/main/scala/orca/tools/GitTool.scala +++ b/tools/src/main/scala/orca/tools/GitTool.scala @@ -1,6 +1,6 @@ package orca.tools -import orca.OrcaFlowException +import orca.{InStage, OrcaFlowException} import orca.events.{OrcaEvent, OrcaListener} import orca.subprocess.QuietProc import ox.either.orThrow @@ -80,36 +80,63 @@ trait GitTool: * the working tree is unchanged in that case. Throws `OrcaFlowException` for * system-level failures (git binary, IO). */ - def createBranch(name: String): Either[BranchAlreadyExists, Unit] + def createBranch(name: String)(using + InStage + ): Either[BranchAlreadyExists, Unit] /** Switch to an existing branch `name` (`git checkout`). Returns * `Left(BranchNotFound)` when no such branch exists — the working tree is * unchanged. Throws `OrcaFlowException` for system-level failures. */ - def checkout(name: String): Either[BranchNotFound, Unit] + def checkout(name: String)(using InStage): Either[BranchNotFound, Unit] /** Switch to `name`, creating it from `HEAD` if it doesn't exist yet. * Idempotent: calling on the current branch is a no-op (no `Step` event * emitted in that case). Useful for resumable flows that may run against a * repo where the branch was already created on a previous attempt. */ - def checkoutOrCreate(name: String): Unit + def checkoutOrCreate(name: String)(using InStage): Unit /** Stage all tracked + untracked changes, then commit them with `message`. * Flow scripts rarely want to manage the index separately, so staging is * part of the commit contract. Returns `Left(NothingToCommit)` when the tree * is already clean. */ - def commit(message: String): Either[NothingToCommit, Unit] + def commit(message: String)(using InStage): Either[NothingToCommit, Unit] + + /** Force-stage `path` (`git add -f`), bypassing `.gitignore`. The stage + * runtime uses this to stage its progress-log file even when the project + * gitignores `.orca/`, so the log travels with the branch (ADR 0018 §2.1). + * Always a single explicit path — never a glob or directory — so nothing + * else gitignored is swept in. + */ + def forceAdd(path: os.Path)(using InStage): Unit /** Push the current branch, setting upstream on first push. Returns * `Left(PushRejected)` when the remote rejected as non-fast-forward (caller * can fetch + rebase). Other failures (auth, network) throw. */ - def push(): Either[PushRejected, Unit] + def push()(using InStage): Either[PushRejected, Unit] def currentBranch(): String + /** Best-effort name of the repository's default branch, read from the + * remote's recorded `origin/HEAD` (`refs/remotes/origin/HEAD` → + * `origin/<name>` → `<name>`). READ-ONLY; no [[InStage]] needed. Returns + * `None` when there is no remote, `origin/HEAD` is unset, or any error + * occurs — callers treat that as "no extra protected branch beyond + * main/master" (ADR 0018). + */ + def defaultBranch(): Option[String] + + /** Discard all uncommitted changes, resetting the working tree and index to + * `HEAD` (`git reset --hard`). Used by the flow failure teardown to drop a + * failed stage's partial edits while keeping the committed history (and the + * committed progress log) intact, so a re-run resumes cleanly (ADR 0018 + * §2.5). + */ + def resetHard()(using InStage): Unit + /** All changes since the last commit (staged and unstaged). */ def diff(): String @@ -149,7 +176,7 @@ trait GitTool: * Returns `true` if a stash was created, `false` if the tree was already * clean. */ - def ensureClean(stashMessage: String): Boolean + def ensureClean(stashMessage: String)(using InStage): Boolean /** Create a linked worktree at `path` on `branch`. If the branch already * exists it is checked out in the new worktree; otherwise it is created from @@ -161,19 +188,37 @@ trait GitTool: def addWorktree( path: os.Path, branch: String - ): Either[WorktreeAddFailed, Worktree] + )(using InStage): Either[WorktreeAddFailed, Worktree] /** Remove the linked worktree rooted at `path`, also deleting the working * directory. Returns `Left(WorktreeNotFound)` when no worktree is registered * at that path. */ - def removeWorktree(path: os.Path): Either[WorktreeNotFound, Unit] + def removeWorktree(path: os.Path)(using + InStage + ): Either[WorktreeNotFound, Unit] /** All linked worktrees attached to the repository, including the main one. * Detached-HEAD worktrees (no branch) are skipped. */ def listWorktrees(): List[Worktree] + /** Force-delete a local branch (`git branch -D <name>`). Best-effort — does + * not throw; failures are silently swallowed so callers can use this in + * teardown without risking an error cascade. Never deletes the current + * branch. + */ + def deleteBranch(name: String)(using InStage): Unit + + /** Diff of `featureBranch` vs `startBranch`, excluding the `.orca/` + * directory. Used by the throwaway-branch check: an empty result means the + * feature branch has no substantive changes beyond orca bookkeeping. + */ + def diffBranchExcludingOrca( + startBranch: String, + featureBranch: String + ): String + /** `GitTool` implementation that shells out to the `git` CLI via os-lib. * Contract semantics (commit auto-staging, push upstream setup, diff vs HEAD, * worktree branch-exists handling) are specified on the trait; this class @@ -189,21 +234,23 @@ private[orca] class OsGitTool( events: OrcaListener = OrcaListener.noop ) extends GitTool: - def createBranch(name: String): Either[BranchAlreadyExists, Unit] = + def createBranch(name: String)(using + InStage + ): Either[BranchAlreadyExists, Unit] = if branchExists(name) then Left(new BranchAlreadyExists(name)) else val _ = git("checkout", "-b", name) events.onEvent(OrcaEvent.Step(s"Switched to a new branch '$name'")) Right(()) - def checkout(name: String): Either[BranchNotFound, Unit] = + def checkout(name: String)(using InStage): Either[BranchNotFound, Unit] = if !branchExists(name) then Left(new BranchNotFound(name)) else val _ = git("checkout", name) events.onEvent(OrcaEvent.Step(s"Switched to branch '$name'")) Right(()) - def checkoutOrCreate(name: String): Unit = + def checkoutOrCreate(name: String)(using InStage): Unit = if currentBranch() == name then // Already on the target — no work to do, no event to emit. () @@ -213,7 +260,7 @@ private[orca] class OsGitTool( private def branchExists(name: String): Boolean = git("branch", "--list", name).trim.nonEmpty - def ensureClean(stashMessage: String): Boolean = + def ensureClean(stashMessage: String)(using InStage): Boolean = val dirty = git("status", "--porcelain").trim.nonEmpty if dirty then val _ = git("stash", "push", "-u", "-m", stashMessage) @@ -225,7 +272,7 @@ private[orca] class OsGitTool( true else false - def commit(message: String): Either[NothingToCommit, Unit] = + def commit(message: String)(using InStage): Either[NothingToCommit, Unit] = val _ = gitWithDiagnostics("add", "-A") // `git status --porcelain` after staging is the cheapest "are there // changes?" check that doesn't depend on parsing localised git output. @@ -235,6 +282,9 @@ private[orca] class OsGitTool( events.onEvent(OrcaEvent.Step(s"Committed: $message")) Right(()) + def forceAdd(path: os.Path)(using InStage): Unit = + val _ = git("add", "-f", path.toString) + /** Like [[git]] but on non-zero exit throws an `OrcaFlowException` enriched * with a `git status --porcelain` + `git fsck --no-progress` snapshot. Used * by the commit path where a bare stderr line ("unable to read tree X") is @@ -268,7 +318,7 @@ private[orca] class OsGitTool( fsck = tryRun("fsck", "--no-progress") ) - def push(): Either[PushRejected, Unit] = + def push()(using InStage): Either[PushRejected, Unit] = // `-u origin HEAD` sets upstream on first push and is a no-op afterwards. // We need to inspect stderr on failure to distinguish the recoverable // "non-fast-forward" case from auth/network errors, so use `gitProc` @@ -286,16 +336,31 @@ private[orca] class OsGitTool( Right(()) else val stderr = result.err.text() - if stderr.contains("non-fast-forward") || stderr.contains("rejected") then + if OsGitTool.isPushRejection(stderr) then Left(new PushRejected(stderr.trim)) - else - throw OrcaFlowException( - s"git push failed (exit ${result.exitCode}): $stderr" - ) + else fail("git push", result) def currentBranch(): String = git("rev-parse", "--abbrev-ref", "HEAD").trim + def defaultBranch(): Option[String] = + try + val result = gitProc( + Seq("git", "symbolic-ref", "--short", "refs/remotes/origin/HEAD") + ) + if result.exitCode == 0 then + // Output is the short ref, e.g. "origin/main"; strip the remote prefix + // to get the bare branch name callers compare against. + Some(result.out.text().trim.stripPrefix("origin/")).filter(_.nonEmpty) + else None + catch case NonFatal(_) => None + + def resetHard()(using InStage): Unit = + val _ = git("reset", "--hard") + events.onEvent( + OrcaEvent.Step("Discarded uncommitted changes (reset --hard)") + ) + def diff(): String = // vs HEAD: show both staged and unstaged changes since the last commit. git("diff", "HEAD") @@ -353,7 +418,7 @@ private[orca] class OsGitTool( def addWorktree( path: os.Path, branch: String - ): Either[WorktreeAddFailed, Worktree] = + )(using InStage): Either[WorktreeAddFailed, Worktree] = // Check out existing branch if it already exists; otherwise branch off // HEAD. `git branch --list <name>` prints the branch when it exists, // empty when not. @@ -368,17 +433,13 @@ private[orca] class OsGitTool( Right(Worktree(path, branch)) else val stderr = result.err.text().trim - // git surfaces both expected cases ("already exists", "is already - // checked out") via stderr. Anything else is a system-level failure. - if stderr.contains("already exists") || - stderr.contains("already checked out") - then Left(new WorktreeAddFailed(path, stderr)) - else - throw OrcaFlowException( - s"git worktree add failed (exit ${result.exitCode}): $stderr" - ) + if OsGitTool.isWorktreeAlreadyPresent(stderr) then + Left(new WorktreeAddFailed(path, stderr)) + else fail("git worktree add", result) - def removeWorktree(path: os.Path): Either[WorktreeNotFound, Unit] = + def removeWorktree( + path: os.Path + )(using InStage): Either[WorktreeNotFound, Unit] = if !listWorktrees().exists(w => samePath(w.path, path)) then Left(new WorktreeNotFound(path)) else @@ -389,6 +450,25 @@ private[orca] class OsGitTool( def listWorktrees(): List[Worktree] = OsGitTool.parseWorktreeList(git("worktree", "list", "--porcelain")) + def deleteBranch(name: String)(using InStage): Unit = + // Best-effort: swallow all failures so teardown is never blocked by a + // cosmetic cleanup step. Never attempt to delete the current branch. + try + if currentBranch() != name then + val result = gitProc(Seq("git", "branch", "-D", name)) + if result.exitCode == 0 then + events.onEvent(OrcaEvent.Step(s"Deleted branch '$name'")) + catch case NonFatal(_) => () + + def diffBranchExcludingOrca( + startBranch: String, + featureBranch: String + ): String = + // Two-dot diff (direct) to see all changes the feature branch has vs the + // start branch. Pathspec `:(exclude).orca/*` strips the orca bookkeeping + // directory so only substantive code changes appear in the result. + git("diff", s"$startBranch..$featureBranch", "--", ".", ":(exclude).orca/*") + private def samePath(left: os.Path, right: os.Path): Boolean = def normalised(path: os.Path): java.nio.file.Path = try path.toNIO.toRealPath() @@ -402,6 +482,15 @@ private[orca] class OsGitTool( private def gitProc(args: Seq[String]): os.CommandResult = QuietProc.call(args, cwd = workDir, env = OsGitTool.nonInteractiveEnv) + /** Abort with a uniform `"<label> failed (exit N): <stderr>"` message for an + * unrecoverable git failure. Callers handle the EXPECTED non-zero exits + * (rejected push, "already exists") as `Left`s before reaching here. + */ + private def fail(label: String, result: os.CommandResult): Nothing = + throw OrcaFlowException( + s"$label failed (exit ${result.exitCode}): ${result.err.text().trim}" + ) + /** Read a single git config value (`git config --get`), `None` when unset. */ private def gitConfigGet(key: String): Option[String] = val r = gitProc(Seq("git", "config", "--get", key)) @@ -416,14 +505,37 @@ private[orca] class OsGitTool( // the event log via the OrcaEvent.Step calls in the public methods // above; we don't need git's verbose stderr for that. val result = gitProc("git" +: args) - if result.exitCode != 0 then - throw OrcaFlowException( - s"git ${args.mkString(" ")} failed (exit ${result.exitCode}): ${result.err.text()}" - ) + if result.exitCode != 0 then fail(s"git ${args.mkString(" ")}", result) result.out.text() private[orca] object OsGitTool: + // --- Recoverable-failure stderr predicates --- + // + // git exits non-zero with a uniform code for many distinct failures, so the + // ONLY way to split a recoverable case (caller gets a `Left`) from a system + // failure (we throw) is to match git's human-readable stderr. These strings + // are git porcelain, not a stable contract, so the matchers are centralised + // here — named, documented, and unit-tested — rather than inlined at the call + // sites, so the fragile coupling lives in one place. Each is intentionally + // lenient (substring, any matching phrase) so a wording tweak across git + // versions doesn't silently reclassify a recoverable failure as fatal. + + /** True when `git push` stderr indicates the push was rejected because the + * remote moved on or a hook refused it (`! [rejected] … (non-fast-forward)`) + * — the recoverable case the caller can resolve by pulling/rebasing. Any + * other non-zero push (auth, network, bad refspec) is a system failure. + */ + private[tools] def isPushRejection(stderr: String): Boolean = + stderr.contains("non-fast-forward") || stderr.contains("rejected") + + /** True when `git worktree add` stderr indicates the target path or branch is + * already a worktree (`… already exists` / `… is already checked out`) — the + * recoverable case. Anything else is a system failure. + */ + private[tools] def isWorktreeAlreadyPresent(stderr: String): Boolean = + stderr.contains("already exists") || stderr.contains("already checked out") + /** Environment that forces git — and any ssh it spawns — to run * non-interactively. A flow subprocess has no usable TTY, so an HTTPS * username/password prompt or an SSH key-passphrase prompt would block the diff --git a/tools/src/main/scala/orca/util/TerminalControl.scala b/tools/src/main/scala/orca/util/TerminalControl.scala index a3736500..7cb84e91 100644 --- a/tools/src/main/scala/orca/util/TerminalControl.scala +++ b/tools/src/main/scala/orca/util/TerminalControl.scala @@ -5,7 +5,8 @@ package orca.util * subprocess stderr, model or tool output — before it is rendered or surfaced * as a diagnostic. Left in, these bytes make `fansi` abort on unsupported * sequences (cursor/status controls like `ESC[?25l`, OSC hyperlinks) and can - * corrupt the terminal. Newlines and tabs are kept; other controls are dropped. + * corrupt the terminal. Newlines and tabs are kept; other controls are + * dropped. */ private[orca] object TerminalControl: diff --git a/tools/src/main/scala/orca/util/TextWrap.scala b/tools/src/main/scala/orca/util/TextWrap.scala index 416f73d7..1d5b8312 100644 --- a/tools/src/main/scala/orca/util/TextWrap.scala +++ b/tools/src/main/scala/orca/util/TextWrap.scala @@ -10,7 +10,7 @@ package orca.util * 76 columns leaves room for the `▶ ` glyph in the terminal at typical * stage-depth indents. */ -object TextWrap: +private[orca] object TextWrap: /** Wrap `s` to `maxWidth` characters, breaking at whitespace. Continuation * lines are prefixed with `continuation` so they sit under the first diff --git a/tools/src/test/scala/orca/AgentInputTest.scala b/tools/src/test/scala/orca/AgentInputTest.scala index da4d444c..28974f9d 100644 --- a/tools/src/test/scala/orca/AgentInputTest.scala +++ b/tools/src/test/scala/orca/AgentInputTest.scala @@ -1,6 +1,6 @@ package orca -import orca.llm.{AgentInput, JsonData} +import orca.agents.{AgentInput, JsonData} case class User(name: String, age: Int) derives JsonData diff --git a/tools/src/test/scala/orca/InStageTest.scala b/tools/src/test/scala/orca/InStageTest.scala new file mode 100644 index 00000000..177eab29 --- /dev/null +++ b/tools/src/test/scala/orca/InStageTest.scala @@ -0,0 +1,17 @@ +package orca + +/** Tests for the InStage capability type (ADR 0018 §2.2). + * + * Positive test: verifies that runtime code (in package `orca`) can mint and + * use an InStage token. + */ +class InStageTest extends munit.FunSuite: + + test( + "runtime code (private[orca]) can mint an InStage and pass it to a block" + ): + val token: InStage = InStage.unsafe + def needsStage(using InStage): Boolean = true + assert(needsStage(using token)) + +end InStageTest diff --git a/tools/src/test/scala/orca/agents/AgentCheapTest.scala b/tools/src/test/scala/orca/agents/AgentCheapTest.scala new file mode 100644 index 00000000..246d2635 --- /dev/null +++ b/tools/src/test/scala/orca/agents/AgentCheapTest.scala @@ -0,0 +1,187 @@ +package orca.agents + +/** Verifies that `Agent.cheap` returns the expected cheap variant per backend, + * and that the default implementation returns `this` (for backends with no + * cheaper tier, e.g. `PiAgent`). + */ +class AgentCheapTest extends munit.FunSuite: + + // ── per-backend cheap assertions ──────────────────────────────────────── + + test("ClaudeAgent.cheap returns haiku"): + val tool = new StubClaudeAgent + val c = tool.cheap + assertEquals(c.name, "haiku") + + test("CodexAgent.cheap returns mini"): + val tool = new StubCodexAgent + val c = tool.cheap + assertEquals(c.name, "mini") + + test("GeminiAgent.cheap returns flash"): + val tool = new StubGeminiAgent + val c = tool.cheap + assertEquals(c.name, "flash") + + test("OpencodeAgent.cheap returns anthropicHaiku"): + val tool = new StubOpencodeAgent + val c = tool.cheap + assertEquals(c.name, "anthropicHaiku") + + test("PiAgent.cheap returns this (no cheaper tier)"): + val tool = new StubPiAgent + assertSameInstance(tool.cheap, tool) + + // ── Agent base default: cheap returns this ───────────────────────────── + + test("Agent default cheap returns this"): + val tool = new StubBaseAgent + assertSameInstance(tool.cheap, tool) + + private def assertSameInstance(a: Any, b: Any): Unit = + assert( + a.asInstanceOf[AnyRef] eq b.asInstanceOf[AnyRef], + s"expected same instance but got $a vs $b" + ) + + // ── minimal stubs ──────────────────────────────────────────────────────── + + private class StubBaseAgent extends Agent[BackendTag.Pi.type]: + val name: String = "base" + def autonomous: AutonomousTextCall[BackendTag.Pi.type] = ??? + def resultAs[O: JsonData: Announce]: AgentCall[BackendTag.Pi.type, O] = ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Pi.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Pi.type] = this + def withName(n: String): Agent[BackendTag.Pi.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Pi.type] = this + + private class StubClaudeAgent extends ClaudeAgent: + val name: String = "claude" + def haiku: ClaudeAgent = namedClaude("haiku") + def sonnet: ClaudeAgent = namedClaude("sonnet") + def opus: ClaudeAgent = namedClaude("opus") + def fable: ClaudeAgent = namedClaude("fable") + def withModel(model: Model): ClaudeAgent = this + def withNetworkTools(t: Seq[String]): ClaudeAgent = this + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = + ??? + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = this + def withName(n: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + private def namedClaude(n: String): ClaudeAgent = + val self = this + new ClaudeAgent: + val name: String = n + def haiku: ClaudeAgent = self.haiku + def sonnet: ClaudeAgent = self.sonnet + def opus: ClaudeAgent = self.opus + def fable: ClaudeAgent = self.fable + def withModel(model: Model): ClaudeAgent = self + def withNetworkTools(t: Seq[String]): ClaudeAgent = self + def autonomous: AutonomousTextCall[BackendTag.ClaudeCode.type] = ??? + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.ClaudeCode.type, O] = ??? + def withConfig(c: AgentConfig): Agent[BackendTag.ClaudeCode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.ClaudeCode.type] = + this + def withName(n2: String): Agent[BackendTag.ClaudeCode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.ClaudeCode.type] = this + + private class StubCodexAgent extends CodexAgent: + val name: String = "codex" + def mini: CodexAgent = namedCodex("mini") + def withModel(model: Model): CodexAgent = this + def autonomous: AutonomousTextCall[BackendTag.Codex.type] = ??? + def resultAs[O: JsonData: Announce]: AgentCall[BackendTag.Codex.type, O] = + ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Codex.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Codex.type] = this + def withName(n: String): Agent[BackendTag.Codex.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Codex.type] = this + private def namedCodex(n: String): CodexAgent = + new CodexAgent: + val name: String = n + def mini: CodexAgent = this + def withModel(model: Model): CodexAgent = this + def autonomous: AutonomousTextCall[BackendTag.Codex.type] = ??? + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.Codex.type, O] = + ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Codex.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Codex.type] = this + def withName(n2: String): Agent[BackendTag.Codex.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Codex.type] = this + + private class StubGeminiAgent extends GeminiAgent: + val name: String = "gemini" + def flash: GeminiAgent = namedGemini("flash") + def withModel(model: Model): GeminiAgent = this + def autonomous: AutonomousTextCall[BackendTag.Gemini.type] = ??? + def resultAs[O: JsonData: Announce]: AgentCall[BackendTag.Gemini.type, O] = + ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Gemini.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Gemini.type] = this + def withName(n: String): Agent[BackendTag.Gemini.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Gemini.type] = this + private def namedGemini(n: String): GeminiAgent = + new GeminiAgent: + val name: String = n + def flash: GeminiAgent = this + def withModel(model: Model): GeminiAgent = this + def autonomous: AutonomousTextCall[BackendTag.Gemini.type] = ??? + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.Gemini.type, O] = + ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Gemini.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Gemini.type] = this + def withName(n2: String): Agent[BackendTag.Gemini.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Gemini.type] = this + + private class StubOpencodeAgent extends OpencodeAgent: + val name: String = "opencode" + def anthropicOpus: OpencodeAgent = namedOpencode("anthropicOpus") + def anthropicSonnet: OpencodeAgent = namedOpencode("anthropicSonnet") + def anthropicHaiku: OpencodeAgent = namedOpencode("anthropicHaiku") + def openaiGpt5: OpencodeAgent = namedOpencode("openaiGpt5") + def openaiGpt5Codex: OpencodeAgent = namedOpencode("openaiGpt5Codex") + def openaiGpt5Mini: OpencodeAgent = namedOpencode("openaiGpt5Mini") + def withModel(providerModel: String): OpencodeAgent = this + def autonomous: AutonomousTextCall[BackendTag.Opencode.type] = ??? + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.Opencode.type, O] = + ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Opencode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Opencode.type] = this + def withName(n: String): Agent[BackendTag.Opencode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Opencode.type] = this + private def namedOpencode(n: String): OpencodeAgent = + new OpencodeAgent: + val name: String = n + def anthropicOpus: OpencodeAgent = this + def anthropicSonnet: OpencodeAgent = this + def anthropicHaiku: OpencodeAgent = this + def openaiGpt5: OpencodeAgent = this + def openaiGpt5Codex: OpencodeAgent = this + def openaiGpt5Mini: OpencodeAgent = this + def withModel(pm: String): OpencodeAgent = this + def autonomous: AutonomousTextCall[BackendTag.Opencode.type] = ??? + def resultAs[O: JsonData: Announce] + : AgentCall[BackendTag.Opencode.type, O] = ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Opencode.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Opencode.type] = + this + def withName(n2: String): Agent[BackendTag.Opencode.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Opencode.type] = this + + private class StubPiAgent extends PiAgent: + val name: String = "pi" + def withModel(model: Model): PiAgent = this + def autonomous: AutonomousTextCall[BackendTag.Pi.type] = ??? + def resultAs[O: JsonData: Announce]: AgentCall[BackendTag.Pi.type, O] = ??? + def withConfig(c: AgentConfig): Agent[BackendTag.Pi.type] = this + def withSystemPrompt(p: String): Agent[BackendTag.Pi.type] = this + def withName(n: String): Agent[BackendTag.Pi.type] = this + def withTools(t: ToolSet): Agent[BackendTag.Pi.type] = this diff --git a/tools/src/test/scala/orca/llm/DefaultPromptsTest.scala b/tools/src/test/scala/orca/agents/DefaultPromptsTest.scala similarity index 94% rename from tools/src/test/scala/orca/llm/DefaultPromptsTest.scala rename to tools/src/test/scala/orca/agents/DefaultPromptsTest.scala index df8d318d..1a43f374 100644 --- a/tools/src/test/scala/orca/llm/DefaultPromptsTest.scala +++ b/tools/src/test/scala/orca/agents/DefaultPromptsTest.scala @@ -1,9 +1,9 @@ -package orca.llm +package orca.agents class DefaultPromptsTest extends munit.FunSuite: private val input = """{"task":"refactor"}""" private val schema = """{"type":"object"}""" - private val config = LlmConfig.default + private val config = AgentConfig.default test("autonomous prompt embeds input and schema and forbids code fences"): val prompt = DefaultPrompts.autonomous(input, schema, config) diff --git a/tools/src/test/scala/orca/agents/JsonDataGivensTest.scala b/tools/src/test/scala/orca/agents/JsonDataGivensTest.scala new file mode 100644 index 00000000..a9cb0784 --- /dev/null +++ b/tools/src/test/scala/orca/agents/JsonDataGivensTest.scala @@ -0,0 +1,69 @@ +package orca.agents + +import com.github.plokhotnyuk.jsoniter_scala.core.{ + readFromString, + writeToString +} +import munit.FunSuite + +class JsonDataGivensTest extends FunSuite: + + private def roundTrip[A](value: A)(using jd: JsonData[A]): A = + readFromString[A](writeToString(value)(using jd.codec))(using jd.codec) + + test("String round-trips"): + assertEquals(roundTrip("hello world"), "hello world") + + test("Int round-trips"): + assertEquals(roundTrip(42), 42) + + test("Long round-trips"): + assertEquals(roundTrip(9876543210L), 9876543210L) + + test("Boolean round-trips true"): + assertEquals(roundTrip(true), true) + + test("Boolean round-trips false"): + assertEquals(roundTrip(false), false) + + test("Double round-trips"): + assertEquals(roundTrip(3.14), 3.14) + + test("Unit round-trips"): + assertEquals(roundTrip(()), ()) + + test("Option[Unit] Some round-trips"): + assertEquals(roundTrip(Some(()): Option[Unit]), Some(())) + + test("Option[Unit] None round-trips"): + // The strict Unit decoder rejects `null`, so the Option codec's own + // null→None handling is what produces None here (and Some(()) reads `{}`). + assertEquals(roundTrip(None: Option[Unit]), None) + + test("Option[Int] Some round-trips"): + assertEquals(roundTrip(Some(1): Option[Int]), Some(1)) + + test("Option[Int] None round-trips"): + assertEquals(roundTrip(None: Option[Int]), None) + + test("List[Int] round-trips"): + assertEquals(roundTrip(List(1, 2, 3)), List(1, 2, 3)) + + test("List[Int] empty round-trips"): + assertEquals(roundTrip(List.empty[Int]), List.empty[Int]) + + test("(String, Int) round-trips"): + assertEquals(roundTrip(("hello", 42)), ("hello", 42)) + + test("(String, Int, Boolean) round-trips"): + assertEquals(roundTrip(("x", 1, true)), ("x", 1, true)) + + test("Option[List[Int]] nested round-trips"): + assertEquals( + roundTrip(Some(List(1, 2)): Option[List[Int]]), + Some(List(1, 2)) + ) + + test("SessionId round-trips"): + val id = SessionId[BackendTag.ClaudeCode.type]("abc-123") + assertEquals(roundTrip(id), id) diff --git a/tools/src/test/scala/orca/llm/ResponseParserTest.scala b/tools/src/test/scala/orca/agents/ResponseParserTest.scala similarity index 98% rename from tools/src/test/scala/orca/llm/ResponseParserTest.scala rename to tools/src/test/scala/orca/agents/ResponseParserTest.scala index 068749b2..c8b8e788 100644 --- a/tools/src/test/scala/orca/llm/ResponseParserTest.scala +++ b/tools/src/test/scala/orca/agents/ResponseParserTest.scala @@ -1,4 +1,4 @@ -package orca.llm +package orca.agents import com.github.plokhotnyuk.jsoniter_scala.macros.ConfiguredJsonValueCodec diff --git a/tools/src/test/scala/orca/agents/WithCheapModelTest.scala b/tools/src/test/scala/orca/agents/WithCheapModelTest.scala new file mode 100644 index 00000000..d37f83c4 --- /dev/null +++ b/tools/src/test/scala/orca/agents/WithCheapModelTest.scala @@ -0,0 +1,86 @@ +package orca.agents + +import orca.backend.{Conversation, Interaction, AgentBackend, AgentResult} +import orca.events.OrcaListener + +/** `withCheapModel` pins the model that [[Agent.cheap]] resolves to, overriding + * the backend default, and the override rides on the config so it survives + * other builders. + */ +class WithCheapModelTest extends munit.FunSuite: + + test("no override: cheap falls back to defaultCheap"): + // The stub has no cheaper tier, so defaultCheap is `this`. + assertEquals(newTool().cheap.name, "model:none") + + test("withCheapModel: cheap pins the given model"): + assertEquals( + newTool().withCheapModel(Model("cheap-x")).cheap.name, + "model:cheap-x" + ) + + test("the override rides on the config through other builders"): + assertEquals( + newTool().withCheapModel(Model("cheap-x")).withReadOnly.cheap.name, + "model:cheap-x" + ) + + private def newTool(): Agent[BackendTag.Pi.type] = new StubTool( + AgentConfig.default + ) + + /** Minimal real `BaseAgent` whose `name` reflects the pinned model, so the + * model `cheap` lands on is observable. `copyTool` threads the config + * (unlike a degenerate `this`-returning stub), which is exactly what + * `withCheapModel` relies on. + */ + private class StubTool(cfg: AgentConfig) + extends BaseAgent[BackendTag.Pi.type, Agent[BackendTag.Pi.type]]( + StubBackend, + cfg, + StubPrompts, + os.temp.dir(), + OrcaListener.noop, + StubInteraction + ): + val name: String = "model:" + cfg.model.map(Model.name).getOrElse("none") + protected def copyTool( + config: AgentConfig = cfg, + name: String = name + ): Agent[BackendTag.Pi.type] = new StubTool(config) + + private object StubBackend extends AgentBackend[BackendTag.Pi.type]: + def runAutonomous( + prompt: String, + session: SessionId[BackendTag.Pi.type], + config: AgentConfig, + workDir: os.Path, + events: OrcaListener, + outputSchema: Option[String] + ): AgentResult[BackendTag.Pi.type] = ??? + def runInteractive( + prompt: String, + session: SessionId[BackendTag.Pi.type], + displayPrompt: String, + config: AgentConfig, + workDir: os.Path, + outputSchema: Option[String] + )(using ox.Ox): Conversation[BackendTag.Pi.type] = ??? + + private object StubPrompts extends Prompts: + def autonomous( + input: String, + outputSchema: String, + config: AgentConfig + ): String = ??? + def interactive( + input: String, + outputSchema: String, + config: AgentConfig + ): String = ??? + def retry(failedResponse: String, parseError: String): String = ??? + + private object StubInteraction extends Interaction: + def listeners: List[OrcaListener] = Nil + def drive[B <: BackendTag](conversation: Conversation[B]): AgentResult[B] = + ??? diff --git a/tools/src/test/scala/orca/backend/AskUserEchoesTest.scala b/tools/src/test/scala/orca/backend/AskUserEchoesTest.scala new file mode 100644 index 00000000..940b1ed6 --- /dev/null +++ b/tools/src/test/scala/orca/backend/AskUserEchoesTest.scala @@ -0,0 +1,20 @@ +package orca.backend + +class AskUserEchoesTest extends munit.FunSuite: + + test("consume returns true once for a suppressed id, then false"): + val echoes = new AskUserEchoes + echoes.suppress("a") + assert(echoes.consume("a"), "first consume of a suppressed id is true") + assert(!echoes.consume("a"), "the id is forgotten after one consume") + + test("consume is false for an id that was never suppressed"): + val echoes = new AskUserEchoes + assert(!echoes.consume("missing")) + + test("ids are tracked independently"): + val echoes = new AskUserEchoes + echoes.suppress("a") + echoes.suppress("b") + assert(echoes.consume("b")) + assert(echoes.consume("a")) diff --git a/tools/src/test/scala/orca/backend/ConversationsTest.scala b/tools/src/test/scala/orca/backend/ConversationsTest.scala index 823e227a..08529b9b 100644 --- a/tools/src/test/scala/orca/backend/ConversationsTest.scala +++ b/tools/src/test/scala/orca/backend/ConversationsTest.scala @@ -2,13 +2,15 @@ package orca.backend import orca.OrcaInteractiveCancelled import orca.events.{OrcaEvent, OrcaListener, Usage} -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} import java.util.concurrent.atomic.{AtomicInteger, AtomicReference} private class ScriptedConversation( eventList: List[ConversationEvent], - outcome: Either[OrcaInteractiveCancelled, LlmResult[BackendTag.Codex.type]], + outcome: Either[OrcaInteractiveCancelled, AgentResult[ + BackendTag.Codex.type + ]], val outputSchema: Option[String] = None ) extends Conversation[BackendTag.Codex.type]: val drained = new AtomicInteger(0) @@ -17,9 +19,8 @@ private class ScriptedConversation( e } def awaitResult() - : Either[OrcaInteractiveCancelled, LlmResult[BackendTag.Codex.type]] = + : Either[OrcaInteractiveCancelled, AgentResult[BackendTag.Codex.type]] = outcome - def sendUserMessage(text: String): Unit = () def canAskUser: Boolean = false def cancel(): Unit = () @@ -34,7 +35,7 @@ private class RecordingListener extends OrcaListener: class ConversationsTest extends munit.FunSuite: - private val sampleResult = LlmResult[BackendTag.Codex.type]( + private val sampleResult = AgentResult[BackendTag.Codex.type]( sessionId = SessionId[BackendTag.Codex.type]("sid"), output = "out", usage = Usage(0L, 0L, None) diff --git a/tools/src/test/scala/orca/backend/SessionRegistryTest.scala b/tools/src/test/scala/orca/backend/SessionRegistryTest.scala index 1d39047c..aa4cff44 100644 --- a/tools/src/test/scala/orca/backend/SessionRegistryTest.scala +++ b/tools/src/test/scala/orca/backend/SessionRegistryTest.scala @@ -1,6 +1,6 @@ package orca.backend -import orca.llm.{BackendTag, SessionId} +import orca.agents.{BackendTag, SessionId} class SessionRegistryTest extends munit.FunSuite: @@ -41,6 +41,25 @@ class SessionRegistryTest extends munit.FunSuite: reg.commitSuccess(client, server) assertEquals(reg.dispatchFor(client), Dispatch.Resume(server)) + test( + "ClientToServer: resumeWireId is None before commit, Some(server) after" + ): + val reg = new SessionRegistry.ClientToServer[BackendTag.Codex.type] + val client = serverSid("client-uuid") + val server = serverSid("server-thread-xyz") + assertEquals(reg.resumeWireId(client), None) + reg.commitSuccess(client, server) + assertEquals(reg.resumeWireId(client), Some(server)) + + test( + "ClaimedOnce: resumeWireId is None before commit, Some(client) after" + ): + val reg = new SessionRegistry.ClaimedOnce[BackendTag.ClaudeCode.type] + val client = sid("client-A") + assertEquals(reg.resumeWireId(client), None) + reg.commitSuccess(client, client) + assertEquals(reg.resumeWireId(client), Some(client)) + test( "ClientToServer: putIfAbsent semantics — second commit doesn't overwrite" ): diff --git a/tools/src/test/scala/orca/backend/StreamConversationTest.scala b/tools/src/test/scala/orca/backend/StreamConversationTest.scala deleted file mode 100644 index 414cadf0..00000000 --- a/tools/src/test/scala/orca/backend/StreamConversationTest.scala +++ /dev/null @@ -1,43 +0,0 @@ -package orca.backend - -import orca.llm.BackendTag -import orca.subprocess.FakePipedCliProcess - -/** Subclass that "forgets" to call `start()` at the end of its constructor — - * the exact mistake [[StreamConversation.ensureStarted]] is designed to catch. - * Public methods should fail loudly rather than silently return "session ended - * without producing a result". - */ -private class UnstartedConversation(process: FakePipedCliProcess) - extends StreamConversation[BackendTag.Codex.type]( - source = StreamSource.fromProcess(process), - backendName = "test" - ): - val outputSchema: Option[String] = None - def sendUserMessage(text: String): Unit = () - protected def handleLine(line: String): Unit = () - -class StreamConversationTest extends munit.FunSuite: - - // Each public entry point must report which one was called so a future - // change that skips the guard on one of them surfaces in the message, - // not just in a generic IllegalStateException. - private val guardedEntryPoints: List[(String, Conversation[?] => Unit)] = - List( - "awaitResult" -> { c => - val _ = c.awaitResult() - }, - "events" -> { c => - val _ = c.events.hasNext - } - ) - - for (label, action) <- guardedEntryPoints do - test(s"$label shouts when the subclass constructor didn't call start"): - val conv = new UnstartedConversation(new FakePipedCliProcess()) - val ex = intercept[IllegalStateException](action(conv)) - assert( - ex.getMessage.contains(s"$label called before start()"), - s"expected the message to name `$label` as the entry point; " + - s"got: ${ex.getMessage}" - ) diff --git a/tools/src/test/scala/orca/backend/StreamSourceConversationTest.scala b/tools/src/test/scala/orca/backend/StreamSourceConversationTest.scala deleted file mode 100644 index 406ab8cb..00000000 --- a/tools/src/test/scala/orca/backend/StreamSourceConversationTest.scala +++ /dev/null @@ -1,49 +0,0 @@ -package orca.backend - -import orca.events.Usage -import orca.llm.{BackendTag, SessionId} - -/** Drives the [[StreamConversation]] base class from a non-subprocess - * [[StreamSource]] — the property the OpenCode backend relies on (its source - * is an SSE connection, not a `PipedCliProcess`). - */ -class StreamSourceConversationTest extends munit.FunSuite: - - private class ListSource(items: List[String]) extends StreamSource: - def lines: Iterator[String] = items.iterator - def errorLines: Iterator[String] = Iterator.empty - def interrupt(): Unit = () - def tryExitCode: Option[Int] = Some(0) - - /** Lines become text deltas; the `DONE` line settles the result. */ - private class TestConversation(source: StreamSource) - extends StreamConversation[BackendTag.Codex.type](source, "test"): - import StreamConversation.Outcome - val outputSchema: Option[String] = None - def sendUserMessage(text: String): Unit = () - protected def handleLine(line: String): Unit = - if line == "DONE" then - val result = LlmResult( - SessionId[BackendTag.Codex.type]("s1"), - output = "hello", - usage = Usage(0L, 0L, None) - ) - val _ = outcomeRef.compareAndSet(None, Some(Outcome.Success(result))) - else eventQueue.enqueue(ConversationEvent.AssistantTextDelta(line)) - start() - - test("translates lines to events and settles the result"): - val conv = new TestConversation(new ListSource(List("a", "b", "DONE"))) - assertEquals( - conv.events.toList, - List( - ConversationEvent.AssistantTextDelta("a"), - ConversationEvent.AssistantTextDelta("b") - ) - ) - assertEquals(conv.awaitResult().map(_.output), Right("hello")) - - test("a source that ends without a result fails the turn"): - val conv = new TestConversation(new ListSource(List("a"))) - conv.events.foreach(_ => ()) - intercept[orca.AgentTurnFailed](conv.awaitResult()) diff --git a/tools/src/test/scala/orca/backend/SupervisedBackend.scala b/tools/src/test/scala/orca/backend/SupervisedBackend.scala index d6a528e6..71bf1bd0 100644 --- a/tools/src/test/scala/orca/backend/SupervisedBackend.scala +++ b/tools/src/test/scala/orca/backend/SupervisedBackend.scala @@ -22,7 +22,11 @@ private[orca] object SupervisedBackend: */ private val DefaultBufferCapacity: BufferCapacity = BufferCapacity(8) - def using[B, T](make: (Ox, BufferCapacity) ?=> B)(body: B => T): T = + /** `body` is a context function so the scope's `Ox` is visible inside it — + * interactive backends need it to call `runInteractive(...)(using Ox)`. + * Autonomous bodies simply ignore the given. + */ + def using[B, T](make: (Ox, BufferCapacity) ?=> B)(body: Ox ?=> B => T): T = supervised: given BufferCapacity = DefaultBufferCapacity body(make) diff --git a/tools/src/test/scala/orca/backend/SystemPromptComposerTest.scala b/tools/src/test/scala/orca/backend/SystemPromptComposerTest.scala index a7286b19..45f1b1c3 100644 --- a/tools/src/test/scala/orca/backend/SystemPromptComposerTest.scala +++ b/tools/src/test/scala/orca/backend/SystemPromptComposerTest.scala @@ -1,41 +1,41 @@ package orca.backend -import orca.llm.{LlmConfig, ToolSet} +import orca.agents.{AgentConfig, ToolSet} class SystemPromptComposerTest extends munit.FunSuite: private val gitRule = SystemPromptComposer.RuntimeOwnsGit test("write-capable turn with nothing else gets just the runtime-git rule"): - val out = SystemPromptComposer.combine(LlmConfig.default, None) + val out = SystemPromptComposer.combine(AgentConfig.default, None) assertEquals(out, Some(gitRule)) test("read-only turn with neither config nor hint returns None"): // Read-only turns can't commit, so the git rule is omitted — and with no // systemPrompt or hint there's nothing left to compose. val out = SystemPromptComposer.combine( - LlmConfig.default.copy(tools = ToolSet.ReadOnly), + AgentConfig.default.copy(tools = ToolSet.ReadOnly), None ) assertEquals(out, None) test("network-only turn also omits the git rule (not Full)"): val out = SystemPromptComposer.combine( - LlmConfig.default.copy(tools = ToolSet.NetworkOnly), + AgentConfig.default.copy(tools = ToolSet.NetworkOnly), None ) assertEquals(out, None) test("config systemPrompt precedes the appended runtime-git rule"): val out = SystemPromptComposer.combine( - LlmConfig.default.copy(systemPrompt = Some("be terse")), + AgentConfig.default.copy(systemPrompt = Some("be terse")), extraHint = None ) assertEquals(out, Some(s"be terse\n\n$gitRule")) test("read-only config keeps just its systemPrompt (no git rule)"): val out = SystemPromptComposer.combine( - LlmConfig.default.copy( + AgentConfig.default.copy( systemPrompt = Some("be terse"), tools = ToolSet.ReadOnly ), @@ -44,10 +44,10 @@ class SystemPromptComposerTest extends munit.FunSuite: assertEquals(out, Some("be terse")) test("selfManagedGit escape hatch omits the git rule"): - // `llm.withSelfManagedGit` flips this flag; the runtime then stays out of + // `agent.withSelfManagedGit` flips this flag; the runtime then stays out of // the agent's git so the agent may commit/push itself. val out = SystemPromptComposer.combine( - LlmConfig.default.copy(selfManagedGit = true), + AgentConfig.default.copy(selfManagedGit = true), extraHint = None ) assertEquals(out, None) @@ -56,7 +56,7 @@ class SystemPromptComposerTest extends munit.FunSuite: // Pins both the order (config, hint, rule) and the separator (\\n\\n) — // backends rely on blank lines so the agent reads distinct paragraphs. val out = SystemPromptComposer.combine( - LlmConfig.default.copy(systemPrompt = Some("be terse")), + AgentConfig.default.copy(systemPrompt = Some("be terse")), extraHint = Some("the hint") ) assertEquals(out, Some(s"be terse\n\nthe hint\n\n$gitRule")) diff --git a/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala b/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala index 86070aeb..133f42d3 100644 --- a/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala +++ b/tools/src/test/scala/orca/subprocess/FakePipedCliProcess.scala @@ -35,6 +35,17 @@ class FakePipedCliProcess( def isAlive: Boolean = alive.get() + /** Forcible kill: close both streams so a reader blocked on `stdoutLines` / + * `stderrLines` unblocks, mirroring the real process's pipes closing. Unlike + * `sendSigInt`, this does not bump `sigIntCount` — it is the backstop, not a + * SIGINT. Idempotent (closing an already-closed queue just offers another + * EOF sentinel, which the single-consumer iterator ignores). + */ + def destroyForcibly(): Unit = + alive.set(false) + closeStdout() + closeStderr() + def waitForExit(): Int = 0 def tryExitCode: Option[Int] = if alive.get() then None else Some(0) diff --git a/tools/src/test/scala/orca/subprocess/OsProcCliRunnerTest.scala b/tools/src/test/scala/orca/subprocess/OsProcCliRunnerTest.scala new file mode 100644 index 00000000..c0257bc8 --- /dev/null +++ b/tools/src/test/scala/orca/subprocess/OsProcCliRunnerTest.scala @@ -0,0 +1,39 @@ +package orca.subprocess + +class OsProcCliRunnerTest extends munit.FunSuite: + + private def alive(pid: Long): Boolean = + val h = ProcessHandle.of(pid) + h.isPresent && h.get.isAlive + + private def awaitDead(pid: Long): Boolean = + val deadline = System.currentTimeMillis + 3000 + while alive(pid) && System.currentTimeMillis < deadline do Thread.sleep(20) + !alive(pid) + + /** Regression guard for the opencode-serve teardown fix: a launch wrapper + * forks the real worker, which inherits the stdout/stderr pipes, so killing + * only the wrapper PID would orphan it and leave a drain reader blocked on a + * never-EOF'd pipe. `destroyForciblyTree` must reap the descendant too. This + * fails if the method ever regresses to the PID-only `destroyForcibly` + * default: the reparented `sleep` would survive and `awaitDead` would time + * out. + */ + test("destroyForciblyTree reaps a forked descendant"): + // `$!` is the backgrounded sleep's PID; `wait` keeps bash alive as its + // parent so it is reachable via `descendants()` at kill time. + val proc = OsProcCliRunner.spawnPiped( + Seq("bash", "-c", "sleep 30 & echo $!; wait"), + env = Map.empty, + cwd = os.pwd, + pipeStderr = true + ) + val childPid = proc.stdoutLines.next().trim.toLong + assert(alive(childPid), "the forked descendant should be running") + + proc.destroyForciblyTree() + + assert( + awaitDead(childPid), + "tree kill must terminate the forked descendant" + ) diff --git a/tools/src/test/scala/orca/testkit/GitRepo.scala b/tools/src/test/scala/orca/testkit/GitRepo.scala new file mode 100644 index 00000000..6b9a6249 --- /dev/null +++ b/tools/src/test/scala/orca/testkit/GitRepo.scala @@ -0,0 +1,23 @@ +package orca.testkit + +/** Test helper: a throwaway temp git repo. `empty` does `git init -b main` plus + * user config; `seeded` adds one `seed.txt` commit so a flow's stash/branch/ + * commit lifecycle (ADR 0018 §2.5) runs in isolation from the dev checkout. + */ +object GitRepo: + /** Fresh temp repo: `git init -b main` + test user config. No commits. */ + def empty(): os.Path = + val dir = os.temp.dir() + val _ = os.proc("git", "init", "-b", "main").call(cwd = dir) + val _ = + os.proc("git", "config", "user.email", "test@example.com").call(cwd = dir) + val _ = os.proc("git", "config", "user.name", "Test").call(cwd = dir) + dir + + /** `empty()` plus a single `seed.txt` commit (`seed`). */ + def seeded(): os.Path = + val dir = empty() + os.write(dir / "seed.txt", "seed") + val _ = os.proc("git", "add", "-A").call(cwd = dir) + val _ = os.proc("git", "commit", "-m", "seed").call(cwd = dir) + dir diff --git a/tools/src/test/scala/orca/tools/CliFailurePredicatesTest.scala b/tools/src/test/scala/orca/tools/CliFailurePredicatesTest.scala new file mode 100644 index 00000000..63560c93 --- /dev/null +++ b/tools/src/test/scala/orca/tools/CliFailurePredicatesTest.scala @@ -0,0 +1,67 @@ +package orca.tools + +/** Pins the recoverable-failure stderr/stdout predicates against realistic git + * and gh output. These match human-readable CLI text (the tools expose no + * machine-readable signal for these cases), so the samples here double as + * documentation of what output each predicate is expected to classify. + */ +class CliFailurePredicatesTest extends munit.FunSuite: + + test("isPushRejection matches a non-fast-forward rejection"): + val stderr = + """To github.com:owner/repo.git + | ! [rejected] feat -> feat (non-fast-forward) + |error: failed to push some refs to 'github.com:owner/repo.git'""".stripMargin + assert(OsGitTool.isPushRejection(stderr)) + + test("isPushRejection matches a hook rejection"): + assert(OsGitTool.isPushRejection("remote: error: GH006: rejected")) + + test("isPushRejection does not match an auth failure"): + val stderr = + "fatal: Authentication failed for 'https://github.com/owner/repo.git/'" + assert(!OsGitTool.isPushRejection(stderr)) + + test("isWorktreeAlreadyPresent matches an existing path"): + assert( + OsGitTool.isWorktreeAlreadyPresent( + "fatal: '/tmp/wt' already exists" + ) + ) + + test("isWorktreeAlreadyPresent matches an already-checked-out branch"): + assert( + OsGitTool.isWorktreeAlreadyPresent( + "fatal: 'feat' is already checked out at '/tmp/other'" + ) + ) + + test("isWorktreeAlreadyPresent does not match a generic failure"): + assert(!OsGitTool.isWorktreeAlreadyPresent("fatal: not a git repository")) + + test("isPrAlreadyExists matches gh's duplicate-PR message (case-folded)"): + // Verbatim gh output, mixed case — the predicate case-folds internally. + val combined = + "a pull request for branch \"feat\" into branch \"main\" already exists:\n" + + "https://github.com/owner/repo/pull/7" + assert(OsGitHubTool.isPrAlreadyExists(combined)) + assert(OsGitHubTool.isPrAlreadyExists("PR Already Exists")) + + test("isNoCommitsToPr matches the no-commits message"): + assert( + OsGitHubTool.isNoCommitsToPr( + "pull request create failed: No commits between main and feat" + ) + ) + + test("isNoCommitsToPr matches the must-first-push message"): + assert( + OsGitHubTool.isNoCommitsToPr( + "Must first push the current branch to a remote" + ) + ) + + test("the gh predicates do not match an unrelated failure"): + val combined = "error: could not resolve to a repository" + assert(!OsGitHubTool.isPrAlreadyExists(combined)) + assert(!OsGitHubTool.isNoCommitsToPr(combined)) diff --git a/tools/src/test/scala/orca/tools/OsFsToolTest.scala b/tools/src/test/scala/orca/tools/OsFsToolTest.scala index 19857bc6..2289cee8 100644 --- a/tools/src/test/scala/orca/tools/OsFsToolTest.scala +++ b/tools/src/test/scala/orca/tools/OsFsToolTest.scala @@ -1,7 +1,13 @@ package orca.tools +import orca.InStage + class OsFsToolTest extends munit.FunSuite: + // Tests exercise gated mutators directly; mint the in-stage token once for the + // whole suite (tests are package `orca.tools`, so `InStage.unsafe` is in reach). + private given InStage = InStage.unsafe + private def withFs(body: (OsFsTool, os.Path) => Unit): Unit = val tmp = os.temp.dir() body(new OsFsTool(base = tmp), tmp) diff --git a/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala b/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala index 8852fbb5..8c6fe4a0 100644 --- a/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala +++ b/tools/src/test/scala/orca/tools/OsGitHubToolTest.scala @@ -1,18 +1,28 @@ package orca.tools -import orca.OrcaFlowException +import orca.{InStage, OrcaFlowException} import orca.events.{OrcaEvent, OrcaListener} -import orca.subprocess.{CliResult, CliRunner, PipedCliProcess, StubCliRunner} +import orca.subprocess.{ + CliCall, + CliResult, + CliRunner, + PipedCliProcess, + StubCliRunner +} import ox.either.orThrow import ox.scheduling.Schedule import java.util.concurrent.ConcurrentLinkedQueue -import java.util.concurrent.atomic.AtomicInteger +import java.util.concurrent.atomic.{AtomicInteger, AtomicReference} import scala.concurrent.duration.DurationInt import scala.jdk.CollectionConverters.* class OsGitHubToolTest extends munit.FunSuite: + // Tests exercise gated gh mutators directly; mint the in-stage token once for + // the whole suite (package `orca.tools` can reach `InStage.unsafe`). + private given InStage = InStage.unsafe + private def stubGh(response: CliResult): (StubCliRunner, OsGitHubTool) = val cli = new StubCliRunner(response) (cli, new OsGitHubTool(cli, pollInterval = 10.millis)) @@ -24,20 +34,28 @@ class OsGitHubToolTest extends munit.FunSuite: /** A `CliRunner` that returns each response in turn (the last repeats), for * tests that need consecutive `run` calls to differ — e.g. fail then - * succeed. Only `run` is exercised here. + * succeed. Records every call so tests can assert on args. Only `run` is + * exercised here. */ private class SequencedCliRunner(responses: List[CliResult]) extends CliRunner: private val next = new AtomicInteger(0) - private val calls = new AtomicInteger(0) - def callCount: Int = calls.get() + private val callsCount = new AtomicInteger(0) + private val recordedCalls: AtomicReference[List[CliCall]] = + AtomicReference(Nil) + def callCount: Int = callsCount.get() + def calls: List[CliCall] = recordedCalls.get().reverse def run( args: Seq[String], stdin: String, env: Map[String, String], cwd: os.Path ): CliResult = - val _ = calls.incrementAndGet() + val _ = callsCount.incrementAndGet() + val _ = + recordedCalls.updateAndGet(cs => + CliCall(args.toList, stdin, env, cwd) :: cs + ) responses(math.min(next.getAndIncrement(), responses.size - 1)) def spawnPiped( args: Seq[String], @@ -76,10 +94,19 @@ class OsGitHubToolTest extends munit.FunSuite: val (_, gh) = stubGh(CliResult(0, "no url here", "")) val _ = intercept[OrcaFlowException](gh.createPr("t", "b")) - test("createPr returns Left(PrAlreadyExists) when gh reports a duplicate"): - val (_, gh) = stubGh( - CliResult(1, "", "a pull request for branch 'feat' already exists") + test( + "createPr returns Left(PrAlreadyExists) when gh reports a duplicate and no open PR is found" + ): + // gh pr create reports duplicate; git rev-parse gives branch name; gh pr list + // returns empty → fallback Left(PrAlreadyExists). + val cli = new SequencedCliRunner( + List( + CliResult(1, "", "a pull request for branch 'feat' already exists"), + CliResult(0, "feat\n", ""), // git rev-parse + CliResult(0, "[]", "") // gh pr list — no open PR + ) ) + val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) assert(gh.createPr("t", "b").left.exists(_.isInstanceOf[PrAlreadyExists])) test( @@ -314,3 +341,141 @@ class OsGitHubToolTest extends munit.FunSuite: .left .exists(_.isInstanceOf[BuildTimedOut]) ) + + // ── createPr idempotency ────────────────────────────────────────────────── + + test( + "createPr returns Right(existing PR) and emits 'Reusing existing PR' when gh reports 'already exists'" + ): + // Call 1: gh pr create exits 1 with "already exists" + // Call 2: git rev-parse to get the current branch name + // Call 3: gh pr list returns JSON with the existing PR + val prListJson = + """[{"number":42,"url":"https://github.com/acme/widgets/pull/42"}]""" + val listener = new CapturingListener + val cli = new SequencedCliRunner( + List( + CliResult(1, "", "a pull request for branch 'feat' already exists"), + CliResult(0, "feat\n", ""), // git rev-parse + CliResult(0, prListJson, "") // gh pr list + ) + ) + val gh = new OsGitHubTool( + cli, + events = listener, + readRetry = Schedule.immediate + ) + val pr = gh.createPr("feat: hi", "hello").orThrow + assertEquals(pr, samplePr) + // Prove the idempotency path went through git rev-parse and gh pr list + val callArgs = cli.calls.map(_.args) + assert( + callArgs.exists( + _.containsSlice(Seq("git", "rev-parse", "--abbrev-ref", "HEAD")) + ), + s"expected git rev-parse in calls but got: $callArgs" + ) + assert( + callArgs.exists(a => + a.containsSlice( + Seq("gh", "pr", "list", "--head", "feat", "--state", "open") + ) + ), + s"expected gh pr list --head feat --state open in calls but got: $callArgs" + ) + assert( + listener.events.exists: + case OrcaEvent.Step(msg) => msg.contains("Reusing existing PR") + case _ => false + ) + + // ── upsertComment ──────────────────────────────────────────────────────── + + test( + "upsertComment on PrHandle creates a new comment (with marker) when no comment matches" + ): + val commentsJson = + """[{"id":1,"body":"unrelated","user":{"login":"alice"}}]""" + val cli = new SequencedCliRunner( + List( + CliResult(0, commentsJson, ""), // fetchIdentifiedComments + CliResult(0, "", "") // writeComment create + ) + ) + val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + gh.upsertComment(samplePr, "<!-- orca:abc:reject -->", "Rejected.") + // The create call must embed the marker in the body + val createCall = cli.calls.last + assert(createCall.args.containsSlice(Seq("gh", "pr", "comment"))) + val markedBody = "Rejected.\n\n<!-- orca:abc:reject -->" + assert(createCall.args.contains(markedBody)) + + test( + "upsertComment on PrHandle updates existing comment via PATCH when marker found" + ): + val commentsJson = + """[{"id":99,"body":"Old text\n\n<!-- orca:abc:reject -->","user":{"login":"alice"}}]""" + val cli = new SequencedCliRunner( + List( + CliResult(0, commentsJson, ""), // fetchIdentifiedComments + CliResult(0, "", "") // PATCH update + ) + ) + val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + gh.upsertComment(samplePr, "<!-- orca:abc:reject -->", "New text.") + // Exactly fetch + PATCH = 2 calls; no comment-create was made + assertEquals(cli.callCount, 2, "expected exactly fetch + PATCH, not more") + assert( + cli.calls.forall(!_.args.containsSlice(Seq("gh", "pr", "comment"))), + "no comment-create call must be made on the PATCH path" + ) + val patchCall = cli.calls.last + assert(patchCall.args.containsSlice(Seq("gh", "api", "-X", "PATCH"))) + // The path must include the comment id 99 + assert(patchCall.args.exists(_.contains("/comments/99"))) + // The body must include both the new text and the marker + val markedBody = "body=New text.\n\n<!-- orca:abc:reject -->" + assert(patchCall.args.contains(markedBody)) + + test( + "upsertComment on IssueHandle creates a new comment (with marker) when no comment matches" + ): + val commentsJson = + """[{"id":1,"body":"unrelated","user":{"login":"alice"}}]""" + val cli = new SequencedCliRunner( + List( + CliResult(0, commentsJson, ""), // fetchIdentifiedComments + CliResult(0, "", "") // writeComment create + ) + ) + val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val issue = IssueHandle("acme", "widgets", 7) + gh.upsertComment(issue, "<!-- orca:abc:triage -->", "Not a bug.") + val createCall = cli.calls.last + assert(createCall.args.containsSlice(Seq("gh", "issue", "comment"))) + val markedBody = "Not a bug.\n\n<!-- orca:abc:triage -->" + assert(createCall.args.contains(markedBody)) + + test( + "upsertComment on IssueHandle updates existing comment via PATCH when marker found" + ): + val commentsJson = + """[{"id":77,"body":"Old.\n\n<!-- orca:abc:triage -->","user":{"login":"alice"}}]""" + val cli = new SequencedCliRunner( + List( + CliResult(0, commentsJson, ""), // fetchIdentifiedComments + CliResult(0, "", "") // PATCH update + ) + ) + val gh = new OsGitHubTool(cli, readRetry = Schedule.immediate) + val issue = IssueHandle("acme", "widgets", 7) + gh.upsertComment(issue, "<!-- orca:abc:triage -->", "Updated.") + // Exactly fetch + PATCH = 2 calls; no comment-create was made + assertEquals(cli.callCount, 2, "expected exactly fetch + PATCH, not more") + assert( + cli.calls.forall(!_.args.containsSlice(Seq("gh", "issue", "comment"))), + "no comment-create call must be made on the PATCH path" + ) + val patchCall = cli.calls.last + assert(patchCall.args.containsSlice(Seq("gh", "api", "-X", "PATCH"))) + assert(patchCall.args.exists(_.contains("/comments/77"))) diff --git a/tools/src/test/scala/orca/tools/OsGitToolTest.scala b/tools/src/test/scala/orca/tools/OsGitToolTest.scala index 3f66fa4d..8696f2ce 100644 --- a/tools/src/test/scala/orca/tools/OsGitToolTest.scala +++ b/tools/src/test/scala/orca/tools/OsGitToolTest.scala @@ -1,29 +1,27 @@ package orca.tools +import orca.InStage import orca.events.{OrcaEvent, OrcaListener} +import orca.testkit.GitRepo import ox.either.orThrow import java.util.concurrent.atomic.AtomicReference class OsGitToolTest extends munit.FunSuite: + // Tests exercise gated git mutators directly; mint the in-stage token once for + // the whole suite (package `orca.tools` can reach `InStage.unsafe`). + private given InStage = InStage.unsafe + private def withRepo(body: (OsGitTool, os.Path) => Unit): Unit = - val dir = os.temp.dir() - val _ = os.proc("git", "init", "-b", "main").call(cwd = dir) - val _ = - os.proc("git", "config", "user.email", "test@example.com").call(cwd = dir) - val _ = os.proc("git", "config", "user.name", "Test").call(cwd = dir) + val dir = GitRepo.empty() body(new OsGitTool(dir), dir) /** Variant that captures the events the tool emits. */ private def withRepoCapturingEvents( body: (OsGitTool, os.Path, AtomicReference[List[OrcaEvent]]) => Unit ): Unit = - val dir = os.temp.dir() - val _ = os.proc("git", "init", "-b", "main").call(cwd = dir) - val _ = - os.proc("git", "config", "user.email", "test@example.com").call(cwd = dir) - val _ = os.proc("git", "config", "user.name", "Test").call(cwd = dir) + val dir = GitRepo.empty() val seen = new AtomicReference[List[OrcaEvent]](Nil) val listener: OrcaListener = (e: OrcaEvent) => val _ = seen.updateAndGet(e :: _) @@ -95,6 +93,32 @@ class OsGitToolTest extends munit.FunSuite: // No remote-tracking refs at all → none of the fallbacks resolve. val _ = intercept[orca.OrcaFlowException](git.defaultBase()) + test("defaultBranch reads the remote HEAD's short name"): + withRepo: (git, dir) => + os.write(dir / "file.txt", "x") + git.commit("seed").orThrow + // Point origin/HEAD at a non-main/master branch to prove it isn't + // hard-coded: create `trunk`, set origin's symbolic ref to it. + val _ = os + .proc("git", "update-ref", "refs/remotes/origin/trunk", "HEAD") + .call(cwd = dir) + val _ = os + .proc( + "git", + "symbolic-ref", + "refs/remotes/origin/HEAD", + "refs/remotes/origin/trunk" + ) + .call(cwd = dir) + assertEquals(git.defaultBranch(), Some("trunk")) + + test("defaultBranch returns None when origin/HEAD is unset"): + withRepo: (git, dir) => + os.write(dir / "file.txt", "x") + git.commit("seed").orThrow + // No remote / no origin/HEAD → best-effort None. + assertEquals(git.defaultBranch(), None) + test("diffVsBase returns the cumulative branch diff vs base"): withRepo: (git, dir) => // base branch with one commit @@ -356,6 +380,60 @@ class OsGitToolTest extends munit.FunSuite: assert(msg.contains("(clean)"), msg) assert(msg.contains("(no issues reported)"), msg) + test("deleteBranch removes an existing branch"): + withRepo: (git, dir) => + os.write(dir / "seed.txt", "x") + git.commit("seed").orThrow + git.createBranch("to-delete").orThrow + git.checkout("main").orThrow + git.deleteBranch("to-delete") + // The branch should no longer be listed. + val result = + os.proc("git", "branch", "--list", "to-delete").call(cwd = dir) + assertEquals(result.out.text().trim, "") + + test("deleteBranch is a no-op for a non-existent branch"): + withRepo: (git, dir) => + os.write(dir / "seed.txt", "x") + git.commit("seed").orThrow + // Must not throw — best-effort. + git.deleteBranch("ghost-branch") + + test("deleteBranch does not delete the current branch"): + withRepo: (git, dir) => + os.write(dir / "seed.txt", "x") + git.commit("seed").orThrow + // Attempt to delete the currently checked-out branch: must silently skip. + git.deleteBranch("main") + assertEquals(git.currentBranch(), "main") + + test("diffBranchExcludingOrca is empty when only .orca/ differs"): + withRepo: (git, dir) => + os.write(dir / "seed.txt", "seed") + git.commit("seed").orThrow + val startBranch = git.currentBranch() + git.createBranch("feature/orca-only").orThrow + os.makeDir(dir / ".orca") + os.write(dir / ".orca" / "progress-abc.json", "{}") + git.commit("orca: progress log").orThrow + val diff = git.diffBranchExcludingOrca(startBranch, "feature/orca-only") + assert(diff.isBlank, s"expected empty diff, got: $diff") + + test("diffBranchExcludingOrca is non-empty when code changes exist"): + withRepo: (git, dir) => + os.write(dir / "seed.txt", "seed") + git.commit("seed").orThrow + val startBranch = git.currentBranch() + git.createBranch("feature/has-code").orThrow + os.write(dir / "feature.txt", "new feature") + git.commit("add feature").orThrow + val diff = git.diffBranchExcludingOrca(startBranch, "feature/has-code") + assert(!diff.isBlank, "expected non-empty diff for code changes") + assert( + diff.contains("feature.txt"), + "diff should mention the changed file" + ) + test("commit on a corrupted repo throws with status + fsck diagnostics"): // Integration check that the formatter is actually wired into the commit // path: corrupt the index so `git add -A` fails, then confirm the thrown diff --git a/tools/src/test/scala/orcacaps/InStageNegativeTest.scala b/tools/src/test/scala/orcacaps/InStageNegativeTest.scala new file mode 100644 index 00000000..4276e3a0 --- /dev/null +++ b/tools/src/test/scala/orcacaps/InStageNegativeTest.scala @@ -0,0 +1,48 @@ +package orcacaps + +/** Negative compile test: verifies that InStage.unsafe is not accessible from + * outside the orca package (ADR 0018 §2.2). This file is intentionally in + * package orcacaps — not orca — so that private[orca] members are hidden. + */ +class InStageNegativeTest extends munit.FunSuite: + + test("InStage.unsafe is not accessible outside the orca package"): + val errors = compileErrors("orca.InStage.unsafe") + assert( + errors.nonEmpty, + "expected a compile error when accessing InStage.unsafe outside orca" + ) + assert( + errors.contains("access") || errors.contains("private") || errors + .contains("cannot be accessed"), + s"expected error to mention an access/visibility restriction, got: $errors" + ) + + test("a gated git mutation does NOT compile without an InStage in scope"): + // The real B2 enforcement: with a `GitTool` in scope but NO `InStage`, + // `git.commit(...)` must fail to compile, pointing at the missing capability. + // Proves the gate works end-to-end — mutation is impossible outside a stage. + val errors = compileErrors( + """ + val git: orca.tools.GitTool = ??? + git.commit("x") + """ + ) + assert( + errors.nonEmpty, + "expected a compile error for git.commit without an InStage" + ) + // `InStage`'s `@implicitNotFound` makes the error user-facing — it tells the + // author to move the call into a `stage(...)` rather than naming the internal + // `InStage` type. Pin that message (and that the cryptic default is gone). + assert( + errors.contains("inside a `stage(...)` body") && + errors.contains("side-effecting"), + s"expected the friendly stage-required message, got: $errors" + ) + assert( + !errors.contains("No given instance of type orca.InStage"), + s"the cryptic default message should be replaced by @implicitNotFound, got: $errors" + ) + +end InStageNegativeTest