Skip to content

Fix LangChain JS token total metrics#2181

Draft
Stephen Belanger (Qard) wants to merge 3 commits into
mainfrom
langchain-js-token-total-metrics
Draft

Fix LangChain JS token total metrics#2181
Stephen Belanger (Qard) wants to merge 3 commits into
mainfrom
langchain-js-token-total-metrics

Conversation

@Qard

Copy link
Copy Markdown
Collaborator

Summary

  • Emit canonical metrics.tokens for LangChain JS LLM spans.
  • Prefer explicit LangChain totals when present.
  • Synthesize tokens from prompt and completion counts when LangChain omits an explicit total.
  • Preserve existing cache token mappings.

Root Cause

LangChain JS callback spans were already logging prompt and completion token metrics, but the callback handler only emitted the legacy total_tokens field for totals. Braintrust cost/token insights expect canonical metrics.tokens, so spans with complete prompt/completion data could still look like they were missing total token usage.

Validation

  • mise exec -- pnpm --dir js exec vitest run src/wrappers/langchain/callback-handler.test.ts
  • mise exec -- pnpm --dir js exec vitest run src/instrumentation/plugins/langchain-plugin.test.ts
  • mise exec -- pnpm exec prettier --check js/src/wrappers/langchain/callback-handler.ts js/src/wrappers/langchain/callback-handler.test.ts .changeset/langchain-js-token-total-metrics.md
  • git diff --check

@Qard Stephen Belanger (Qard) force-pushed the langchain-js-token-total-metrics branch from 0f590ca to 98e3bed Compare July 1, 2026 17:53
Comment on lines +1266 to 1267
"tokens": 97,
"total_tokens": 97

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the actual diff between tokens and total_tokens?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants