feat: WebUI redesign + i18n, cloud agent deployment, and backend features#52
Open
wuchulonly wants to merge 385 commits into
Open
feat: WebUI redesign + i18n, cloud agent deployment, and backend features#52wuchulonly wants to merge 385 commits into
wuchulonly wants to merge 385 commits into
Conversation
Port key improvements from feature/comprehensive-improvements-v3: - swarm: broadcast message adds "type" field, fixes aide-ui unknown display - agent: DefaultMaxRetries 2→9, reduces task interruption from transient API errors - ioa: resolvingClient transparently resolves space names to IDs with cache; --space/--space_name aliases for --space_id; unknown names list available spaces - passive: dynamic Usage shows configured sources; shodan-idb validates IP/CIDR input with actionable error; sourceList sorted for stable output - resources: FullFingers merge preserves fingerprinthub template fingerprints; NewEngineWithFingers replaces NewEngine for correct initialization - command: auto-inject --no-color for scan in registry (agent context); remove --no-color from scan/katana Usage to reduce LLM cognitive load - docs: SKILL.md clarifies tmux pane as sole scan output channel Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nd CLI - Remove boundClient/toolAdapter/SDK NewTools indirection — three commands (ioa_space, ioa_send, ioa_read) now call client.Space/Send/Read directly - Organize each command with subcommands: - ioa_space: join, list, nodes, topics - ioa_send: (broadcast), to --node, reply --to - ioa_read: (default), all, thread --id, new --after - Auto-join configured space at startup via InitIOA + SetDefaultSpace - LLM never needs to know space_id — binding handled internally - 24 unit tests + 1 DeepSeek LLM integration test (TestLLMIOAToolUsage) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace 5 overlapping event mechanisms (Config.Emit, EventsWriter, combineEventHandlers, recorder, aiTraceCollector) with a single generic EventBus[T] that both agent and scan pipeline use. - Add pkg/eventbus with generic Bus[T] (Subscribe/Emit) - Agent: Config.Bus replaces Config.Emit, loop calls bus.Emit() directly - Pipeline: Config.Bus replaces Config.Observe, collector is a subscriber - Derive() propagates Bus so sub-agent events flow to parent - Delete: recorder.go, ai_trace.go, combineEventHandlers, EventHandler type - Delete: orphaned AITurn/AIRequest/AIToolCall/AIToolResult output types - scan -f uses scanJSONLWriter subscribing to pipeline bus - --events-file uses eventsFileSubscriber subscribing to agent bus Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove --space_id from all examples (auto-bound after join) - Update ioa_space/ioa_send/ioa_read to show subcommand syntax - Update task dispatch example to use `ioa_send to --node` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CheckpointTool now supports pluggable sinks via OnCheckpoint(). When IOA
is configured, a sink is wired that sends checkpoint results as structured
messages (using IOAContent/IOAMeta) to the current space — other agents
can read findings without polling scan output.
Wiring: InitIOA → ioa.CheckpointSink → App.CheckpointSink →
scan.WithCheckpointSink → CheckpointTool.OnCheckpoint
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Delete --events-file CLI flag; scan -f now writes both scan and agent data to the same JSONL file via scanJSONLWriter subscribing to both pipeline bus and parent agent bus - Add SessionID to Event and EventJSON so parallel agents can be distinguished; Derive() generates a new SessionID per child agent - Add emitter type (bus + sessionID) to stamp SessionID on every emit without burdening Config with runtime behavior - Merge normalizeConfig + prepareConfig into Config.init() — single entry point for all config initialization - Export agent event serialization (SerializableEvent, EventJSON) to pkg/agent/event_json.go so scan package can serialize agent events - Delete dead telemetry.Recorder (zero external references) - Fold AISkill/AIResponse into TypeToolCall via provider.ToolCall format - Harness uses AISCAN_EVENTS_FILE env var instead of deleted CLI flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
External consumers no longer need to import pkg/agent/provider directly. The agent package now serves as the single public API surface via type aliases in export.go. The command package switches to ioa.ToolDefinition to avoid a circular dependency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the entire AI skill runner mechanism (aiskill.go, agent.go), the finding interface and its 6 wrapper types, and the CheckpointSink callback injection. Replace with a single output.Loot struct that uniformly represents all scan outputs (fingerprint, weakpass, vuln). - Delete AISkill/AISkillConfig/CheckpointSink/finding abstractions - Introduce output.Loot as the unified scan output type - Add ioa_send checkpoint subcommand (replaces CheckpointSink) - Simplify collector from 5 result slices to 1 loots slice - Reduce JSONL record types: TypeFinding → TypeLoot Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix checkpoint subcommand unreachable due to content nil check order - Fix TypeService incorrectly parsed as Loot in format_markdown - Add ioa_send checkpoint tests (schema, missing args, no space) - Add Loot JSONL round-trip and JSON schema tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove export.go — all type aliases, constant re-exports, and function re-exports now live in types.go alongside agent-specific types. All internal agent files (loop.go, retry.go, subagent.go, event_json.go, agent.go, loop_tool.go) and their tests no longer import provider directly — only types.go retains the provider import. This makes provider a pure internal implementation detail. Also adds ErrCallTimeout/ErrStreamStalled re-exports and skips python3-dependent tests when python3 is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oading - Replace string-builder prompt construction with text/template - Add agent.SystemPromptFn for dynamic per-turn prompt generation - Add LoadedSkills to PromptConfig: -s scan/verify embeds skill body directly into system prompt (no LLM read tool call needed) - Support sub-file skill paths (scan/verify → skills/scan/verify.md) - Delete pkg/prompt/ over-engineered Compiler package Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rride Replace custom line-based frontmatter parser with gopkg.in/yaml.v3. Add multi-level skill loading: embedded < .aiscan/skills/ < .agent/skills/ < CLI -s path. External same-name skills override embedded ones. Store methods (ReadBody, FormatInvocation, ReadVirtual) are now source-aware for local skills. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix two legacy cycles that forced unbounded global seenEvents/seenRuns maps: - Remove redundant redirect webTarget emit from deriveWebProbeResult (spray engine follows redirects internally) - Remove redundant webTarget emit from deriveWeakpassResult (URL already discovered by gogo) Replace event-loop pipeline with subscription-based DAG routing: - Each capability declares explicit Route subscriptions (From + Accept) - Pipeline builds route index for O(matching routes) dispatch - DAG validated at construction via DFS cycle detection - Per-route dedup maps replace global seenEvents/seenRuns - All dedup state freed in cleanup() after Run() - Remove crawlSources hack, acceptsNonCrawlTarget, RegisterCrawlSource Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove local swarm and checkpoint implementations, replaced by ioa/protocols/swarm and ioa/protocols/checkpoint packages. - Delete pkg/swarm/ entirely (node.go, message.go, cron.go, tests) - Remove execCheckpoint from pkg/tools/ioa/commands.go - Use protocols.SendHandler for dynamic protocol dispatch - Import _ "ioa/protocols/checkpoint" in register.go to activate - Adapt RegisterNode calls to 3-arg signature (name, description, meta) - go.mod: add replace directive for local ioa development Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…andalone build Merge browser/recon build tags into a single "full" tag. Release now produces only two binaries: aiscan (base) and aiscan-full (includes passive, katana, playwright, sqlite). The ioa standalone binary is removed — use `aiscan ioa`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…proxy/passive Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace all --llm-* prefixes with current --provider/--base-url/--api-key/--model - Add agent built-in tools table (search tavily/fetch/cyberhub, tmux, proxy) - Expand skills table from 6 to 12 (add search, report, ioa, playwright, passive, katana) - Add AI capabilities section (--sniper/--deep/--ai) in scan docs - Add -F/--view scan record replay documentation - Update scenario table with new AI modes - Add aiscan-full version description in quickstart - Fix quickstart AI examples to use current syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update IOA to v0.0.0-20260608071724 (content_type, declarative schema) - Add swarm protocol blank import for CLI registration - Pass content_type from send args to SendMessage envelope Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…proxy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ayer) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- peerToInboxMessage: stop discarding Refs, handle fork notifications (ContentType == "ioa/fork") as OriginSystem messages - buildNodeMeta: replace buildSkillRefs — construct Node.Meta with kind/hostname/skills directly, swarm protocol no longer owns this - renderContent: include message_id in XML envelope for LLM context - NodeConfig: use Meta instead of Skills/SkillRefs, set ForkDepth=1 - go.mod: local replace for IOA during development Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Import ioa/protocols for core types, ioa/api for API DTOs - ToolDefinition defined once in provider/types.go, command/ uses type alias - Remove ioa root dependency — ioa root package is now empty - peerToInboxMessage preserves Refs, handles fork notifications - buildNodeMeta replaces buildSkillRefs — skills in Node.Meta - ServerOptions.Logger removed — IOA uses standard log Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…m worker - Unify store: merge store_sqlite.go + store_memory.go into single openStore function. Remove build tags. Use server.NewSQLiteStore for file-based, server.NewMemoryStore for in-memory. - Internalize swarm worker: move swarm.Node runtime from IoA's deleted protocols/swarm/node.go into pkg/ioaswarm/. Update runner.go import. - Fix test imports: replace deleted root ioa package and api package references with protocols.X types throughout test files. - Fix RegisterNode call signature: add missing description parameter. - Fix TestSendCheckpoint: assert ContentType field instead of content["type"] to match checkpoint protocol's actual behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nal events Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…c assets Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
wuchulonly
force-pushed
the
feat/webui-redesign-i18n-deploy
branch
from
f821dcf to
8b92e72
Compare
…rr, errcheck, staticcheck) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ata race runConnectionOnce signalled its ping/ctx goroutines via close(done) but never waited for them, so they outlived the function and raced on the keepalive globals. Add a WaitGroup and wait after close(done). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The WebUI redesign changed the DOM, so the rod selectors block forever under plain 'go test ./...'. Skip unless AISCAN_E2E=1 (matches the dedicated e2e job). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
installLocalPortPreset moved to the package-level PortPreset var; the test still checked the old global, tripping SA1019 and failing the install assertion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The loopback SSH banner read timed out at 8s under 'go test -race' load (passes in ~0.1s locally). Widen the deadlines so the interception path has headroom under the race detector. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The graded-output rewrite loaded the default "keys" category alongside the -e expression rule; sharing a scan group let the builtin rules shadow the expression's extracts, so 'proton -e' silently matched nothing. When only expressions are given (no explicit -t/--id), load just the expression rule. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
proton v0.3.3's false-positive filter suppresses the low-confidence private-key extract during a mixed aggregate scan; assert the rule via --id instead (a realistic key can't be committed due to push protection). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
wuchulonly
force-pushed
the
feat/webui-redesign-i18n-deploy
branch
from
6384422 to
a01936e
Compare
Under the race detector the loopback mitm interception starves for tens of seconds (>30s vs ~0.1s normally), timing out the banner read. Skip just this test under -race via a build-tagged raceEnabled flag; it still runs in a normal 'go test'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
WebUI redesign + i18n, cloud agent deployment, and a set of backend features, split into granular per-feature commits (frontend kept as a single commit). Based on the latest
master.~20 commits, 384 files, +22.9k / −3.5k.
Backend (one feature per commit)
golang.org/x/crypto,tjfoc/gmsmand transitive deps.aiscan.yaml,.gitignore.Frontend (single commit)
WebUI redesign, i18n (zh/en), cloud deploy panel, asset pool UI, plus rebuilt static assets (code-split bundles + subsetted fonts).
Verification
At the branch tip:
go build ./...✓go build -tags full ./...✓ (embedsweb/static)go test -tags full ./pkg/web/...(E2E skipped) ✓Notes
aiscan.yamlships with emptyapi_key/fofa_keyplaceholders; test fixtures use non-credential-looking AccessKey IDs.cyber-uisubmodule has local source edits (button/card/theme-toggle/chat components +ScopeGlyph,timeline-extensions) that are not part of this PR — they belong in a separatecyber-uirepo commit + pointer bump. The rebuiltweb/staticassets in this PR already reflect them.