fix: reattach video tool results for openai providers#1019
Conversation
🦋 Changeset detectedLatest commit: d204867 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d2048672bf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| function toolResultImageParts(message: Message): OpenAIContentPart[] { | ||
| const images: OpenAIContentPart[] = []; | ||
| function isReattachableToolMediaPart(part: ContentPart): boolean { | ||
| return part.type === 'image_url' || part.type === 'video_url'; |
There was a problem hiding this comment.
When a custom/MCP tool returns a video_url while the provider is the default OpenAI Chat Completions backend (or another compatible endpoint that only accepts text/image/audio/file user parts), this now queues that video as a follow-up user content part and the next request is rejected for unsupported video_url. Previously those tool videos stayed as an inline omission note, so OpenAI users could continue the turn; this should only reattach video for models/endpoints known to accept video_url.
Useful? React with 👍 / 👎.
1 participant
Related Issue
No related issue. This fixes a provider conversion bug where video media returned by a tool is omitted before the next OpenAI-compatible chat request.
Problem
OpenAI-compatible chat providers can accept
video_urlcontent parts, and regular user messages already serialize them. However, when a tool returns video media, the tool message is converted to text and only images are reattached in a follow-up user message. Videos are replaced with an omission note, so video-capable OpenAI-compatible providers never receive the media.What changed
video_urltool result media alongsideimage_urlmedia in the follow-up user message.Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.Verification
pnpm exec vitest run packages/kosong/test/openai-legacy.test.tspnpm --filter @moonshot-ai/kosong run typecheck