feat(llm): configurable analyzer fan-out concurrency via SKILLSPECTOR_MAX_LLM_CONCURRENCY (part of #303)#305
Open
Mark2Mac wants to merge 1 commit into
Open
Conversation
`arun_batches` bursts up to a hardcoded `max_concurrency=10`. On a rate-limited provider (free tiers with a low RPM) that burst guarantees 429s, and 429'd batches are dropped from the result — silently losing analyzer coverage, including the security-critical `semantic_security_discovery` batch, while the report still renders a full risk_assessment. Add `SKILLSPECTOR_MAX_LLM_CONCURRENCY` so users can serialize the fan-out (set it to 1) and stay under the provider's rate limit. The default stays 10; an explicit `max_concurrency=` argument still wins. Invalid values fall back to the default, values < 1 clamp to 1. Part of NVIDIA#303 (the retry/backoff and partial-pass surfacing are separate). Adds unit tests for the env resolution. Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
arun_batchesfans out up to a hardcodedmax_concurrency=10. On a rate-limited provider (free tiers with a low RPM) that burst guarantees 429s, and — per the method's own contract — 429'd batches are logged and omitted from the result. The scan silently loses analyzer coverage, including the security-criticalsemantic_security_discoverybatch, while still emitting a fullrisk_assessment. A genuinely malicious skill can then read as clean.This adds an env knob so users on such tiers can serialize the fan-out and stay under the limit:
max_concurrency=argument still wins.< 1clamp to 1.Test
TestResolveMaxConcurrency: unset→default, valid→value, blank→default, invalid→default,0→1. Fulltest_llm_analyzer_base.pysuite green (116 passed); ruff clean.Scope
This is the first of the three asks in #303 (configurable concurrency). The retry/backoff on 429 and surfacing a partial LLM pass in the verdict are larger changes left for separate PRs / discussion.
Refs #303