Skip to content

fix(inbound): download and pass all mediaPaths for multi-image rich text messages#545

Merged
soimy merged 4 commits into
mainfrom
fix/issue-542-multi-image-mediapaths
May 1, 2026
Merged

fix(inbound): download and pass all mediaPaths for multi-image rich text messages#545
soimy merged 4 commits into
mainfrom
fix/issue-542-multi-image-mediapaths

Conversation

@soimy

@soimy soimy commented May 1, 2026

Copy link
Copy Markdown
Owner

背景

Issue #542: 用户发送包含多张图片的富文本消息时,bot 只收到第一张图片。

解析层 message-utils.ts 已正确将多图的 downloadCode 提取到 content.mediaPaths 数组,类型定义 src/types.ts 也支持 mediaPaths?: string[]mediaTypes?: string[]。但入站处理层 inbound-handler.ts 只使用了 content.mediaPath(单数),完全忽略了数组字段。

目标

  • 下载富文本消息中的所有图片(而非仅第一张)
  • 将完整的多媒体路径数组传递给 finalizeInboundContext
  • 保持向后的单图兼容性

实现

src/inbound-handler.ts 三处语义变更(其余 diff 为 oxfmt 格式整理):

  1. 新增变量: mediaPaths: string[]mediaTypes: string[] 用于收集所有下载结果
  2. 遍历下载: 优先使用 content.mediaPaths(richText 多图),fallback 到 content.mediaPath(单图/文件等),逐个 downloadMedia
  3. 传递数组: 向 finalizeInboundContext 新增 MediaPaths / MediaUrls / MediaTypes 字段,同时保留 MediaPath / MediaUrl 单数字段向后兼容

实现 TODO

  • 媒体下载部分改为遍历 content.mediaPaths
  • finalizeInboundContext 传入 MediaPaths / MediaUrls / MediaTypes
  • 保持 MediaPath 单数字段不变(向后兼容)
  • 预下载媒体路径(sub-agent 场景)同步填充数组
  • 单图/文件/音频消息行为不变

验证 TODO

  • pnpm run type-check 通过
  • pnpm run lint 0 errors(88 pre-existing warnings)
  • pnpm test 1069/1069 通过(包括已有的 richText 自身多图时保留首图并暴露 mediaPaths 单元测试)
  • pnpm test:coverage 通过
  • 真机验证:钉钉富文本多图消息 → bot 收到所有图片

AI-assisted PR,作者已理解并确认所有变更和验证结果

…ext messages

When a richText message contains multiple images, the parser correctly
extracts them into content.mediaPaths, but the inbound handler only
downloaded content.mediaPath (the first image) and only passed the
singular MediaPath/MediaUrl to finalizeInboundContext, discarding
the rest.

Now the handler iterates over content.mediaPaths (falling back to
content.mediaPath for single-media messages), downloads each one,
and passes the complete MediaPaths/MediaUrls/MediaTypes arrays to
the context alongside the legacy singular fields for backward compat.

Closes #542
@greptile-apps

greptile-apps Bot commented May 1, 2026

Copy link
Copy Markdown

Greptile Summary

此 PR 修复了钉钉富文本多图消息仅传递第一张图片的问题(Issue #542)。inbound-handler.ts 新增遍历下载逻辑、引用多图缓存恢复路径,并向 finalizeInboundContext 传递完整的 MediaPaths/MediaTypes 数组;agent-routing.ts 同步修复了 sub-agent 预下载场景;类型定义与存储层(message-context-store.ts)也相应扩展了 downloadCodes 字段,向后兼容性保持良好。

Confidence Score: 5/5

变更逻辑清晰,向后兼容性完善,测试覆盖充分,可安全合并。

仅有一处 P2 风格问题(downloadCodes 未 trim),不影响运行时正确性;核心多图下载与传递逻辑正确,sub-agent 与引用路径均已修复并有单元测试覆盖。

src/message-context-store.ts 的 normalizeMedia 函数(downloadCodes trim 问题)

Important Files Changed

Filename Overview
src/inbound-handler.ts 核心修改:新增 mediaPaths/mediaTypes 数组收集所有下载结果,遍历 content.mediaPaths 下载多图,新增引用多图恢复路径,向 finalizeInboundContext 传递完整数组字段;其余为 oxfmt 格式整理,逻辑正确。
src/message-context-store.ts 新增 downloadCodes 字段到 MessageRecord 和缓存/反序列化逻辑;filter 未 trim 各元素,与 downloadCode 存在轻微不一致。
src/targeting/agent-routing.ts 预下载逻辑重写,遍历 extractedContent.mediaPaths(fallback 到 mediaPath),正确构建 preDownloadedMedia 完整数组字段。
src/types.ts HandleDingTalkMessageParams.preDownloadedMedia 新增 mediaPaths/mediaTypes 可选字段,其余为 oxfmt 格式整理。
tests/unit/inbound-handler-quote-multi-image.test.ts 新增测试文件,覆盖多图缓存写入、单图兼容性、引用多图恢复三个场景,测试结构完整。

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[收到入站消息] --> B{preDownloadedMedia?}
    B -- 有(sub-agent) --> C[读取 preDownloadedMedia.mediaPaths]
    C --> D[填充 mediaPaths / mediaTypes]
    B -- 无 --> E{robotCode?}
    E -- 否 --> J
    E -- 是 --> F{content.mediaPaths?}
    F -- 有(多图) --> G[遍历下载每个 downloadCode]
    F -- 无 --> H{content.mediaPath?}
    H -- 有(单图) --> I[下载单图]
    H -- 无 --> J
    G --> K[push 到 mediaPaths / mediaTypes]
    I --> K
    K --> L[缓存 downloadCode + downloadCodes]
    J --> M{quotedRecord.downloadCodes > 1?}
    L --> M
    M -- 是(多图引用恢复) --> N[遍历 downloadCodes 下载并 push 到 mediaPaths]
    M -- 否 --> O[其他引用路径:quoted picture/file/doc]
    N --> P[finalizeInboundContext MediaPath / MediaPaths / MediaUrls / MediaTypes]
    O --> P
Loading
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
src/message-context-store.ts:167-170
**`downloadCodes` 各元素未 trim,与 `downloadCode` 不一致**

第 169 行的 filter 只过滤空字符串,但不裁剪各元素的首尾空白;而上方的 `downloadCode`(第 165 行)做了 `.trim()`。若某个 code 含首尾空白,会原样存入缓存并在后续恢复时作为请求参数传给 DingTalk API,可能导致下载失败。

```suggestion
      ? candidate.downloadCodes.filter((c: unknown) => typeof c === "string" && (c as string).trim()).map((c: unknown) => (c as string).trim())
```

Reviews (4): Last reviewed commit: "fix(inbound): persist multi-image downlo..." | Re-trigger Greptile

Comment thread src/inbound-handler.ts
Comment thread src/inbound-handler.ts
@soimy soimy linked an issue May 1, 2026 that may be closed by this pull request
soimy added 3 commits May 1, 2026 16:47
…paths

Address review feedback on #545:
- sub-agent pre-download now iterates over content.mediaPaths (plural)
  instead of only content.mediaPath, and passes the full arrays through
  preDownloadedMedia
- quoted-picture download result is also pushed to mediaPaths/mediaTypes
  so singular and plural fields stay consistent
- preDownloadedMedia branch always pushes to both arrays to keep them
  aligned, falling back to "file" when mediaType is unavailable

Refs #542
… paths

Extend the mediaPaths/mediaTypes array pushes to the remaining six
resolution points: DM doc card download, quoted file (steps 0/1/2),
and quoted doc card (steps 1/2). Now every path that sets mediaPath
also pushes to the plural arrays, keeping MediaPath and MediaPaths
semantically consistent regardless of how media was resolved.

Refs #542
…ore for quote recovery

Previously, only the first `downloadCode` was written to the message context store,
so quoting a multi-image richText message would only recover one image. The state
normalization pipeline (`normalizeMedia`) also silently stripped the `downloadCodes`
array during `normalizeState` → `normalizeMessageRecord` → `normalizeMedia`.

- Add `downloadCodes?: string[]` to `MessageRecord.media` and `BaseUpsertParams.media`
- Update `normalizeMedia` to extract and validate the `downloadCodes` array
- Update `mergeMedia` to merge `downloadCodes` from next or existing record
- Write all `content.mediaPaths` as `downloadCodes` during cache write when count > 1
- Add multi-image quote recovery block: iterate cached `downloadCodes` to recover all images
- Add regression test: `inbound-handler-quote-multi-image.test.ts` (3 tests)
@soimy soimy merged commit 6f0f73c into main May 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

单条消息多张图片只收到第一张,mediaPaths 未处理

1 participant