Skip to content

feat: chunk batch reflection scoring#1957

Open
chiefmojo wants to merge 1 commit into
MemTensor:dev-20260604-v2.0.19from
chiefmojo:feat/chunk-batch-reflection
Open

feat: chunk batch reflection scoring#1957
chiefmojo wants to merge 1 commit into
MemTensor:dev-20260604-v2.0.19from
chiefmojo:feat/chunk-batch-reflection

Conversation

@chiefmojo

Copy link
Copy Markdown

Problem

Reflection scoring processes all pending traces in a single batch. On stores with large episode backlogs (100+ traces) this produces a single massive LLM call that hits token limits, times out, or produces degraded output quality.

Changes

Adds a reflectionBatchSize config option (default: 20). Reflection scoring splits pending traces into chunks of reflectionBatchSize and processes each chunk sequentially, reusing the same LLM client across chunks.

  • Zero/negative values fall back to unbounded (existing behavior)
  • Each chunk produces its own scored output; results are merged before persisting
  • No change to the scoring algorithm or output format

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant