Repository navigation
feat(ai-components): chat options, suggestions and attachment preview fixes - #3931
Merged
Merged
Conversation
2 of 6 tasks
isekovanic
force-pushed
the
feat/ai-composer-extras
branch
from
November 5, 2026 16:57
eb380d4 to
ff67ab5
Compare
Contributor
SDK Size
|
isekovanic
added a commit
that referenced
this pull request
Nov 6, 2026
Ticket: https://linear.app/stream/issue/RN-485 Stacked on #3931. ## π― Goal Let an AI agent call tools that run on the person's phone, as StreamChatAI's `AIClientToolRunner` does. React has no runner, so this ports Swift's. ## π Implementation details **`AIClientToolRunner`** (`tool-calls/AIClientToolRunner.ts`) - Swift's types: `AIClientTool` (a definition, optional instructions, and `run`), `AIClientToolDefinition` (MCP's tool JSON: `name`, `description`, `inputSchema`, `annotations.readOnlyHint`), `AIClientToolResult` (`output`, `summary`, `failure`), with `completedToolResult` and `failedToolResult`. - `run(parts, send)` takes a reply's steps as they update. It runs a call when it is `awaiting_client` for this person and this install (`isAwaitingClient`), the runner has a tool of that name, and the call hasn't run here before. The result goes to the app's `send`. - A result that couldn't be sent is offered again on a later update without running the tool again, up to `maxAttempts` (3). A tool that throws is reported as failed. The first tool of a name wins. - `registrations` describes the tools to the agent, sorted by name, as Swift's do. **ExpoChatGPTSample** - One phone tool, `get_local_time` (the phone's clock and time zone), shown on the reply as "Checking your phone's time". - The tool is declared on the router session (calypso `Tools`, `executor: 'client'`), and the session's socket answers the router with what the runner reported. - The router addresses a call to an install only when the question names it, and only its `respond` endpoint takes `client_id`. So plain questions go through `respond` with this install's id, and the runner runs their calls from the reply. - `responses` (calypso's `responses.create`) is the only one that takes photos and a reasoning effort, and it takes no install. A question with either goes there, and the socket runs the tool itself, since no step will name this phone. ## π¨ UI Changes None in the SDK. The sample shows the phone tool's step, "Waiting for the device" and then done. ## π§ͺ Testing - Unit tests port StreamChatAI's runner tests: a call runs once and only for its person and install, an unsent result is sent again without running the tool again, giving up after the attempts, finished calls, the first tool of a name, and the registrations. Plus a call still running not starting again, and a throwing tool reported as failed. - On the iOS simulator and an Android emulator against a local Vision-Agents router, "What time is it on my phone?": - a plain question: the step waits for this install, the runner runs the tool once and the router gets the result (0.2 s on iOS); - with "Think harder": the step names no install and the socket answers it. - On the emulator the app once stopped getting chat updates mid-reply. The channel had the whole reply, and reopening the conversation showed it with the tool not run again. That's the sample's reconnect handling, already planned for the polish pass. - `yarn lint`, `yarn typecheck`, `yarn build` and the unit tests pass. Router follow-up: `POST /v1/agents/sessions/{id}/responses` should take `client_id` as `respond` does, and calypso's `responses.create` should send it. Then the sample can drop the `respond` path. ## βοΈ Checklist - [x] I have signed the [Stream CLA](https://docs.google.com/forms/d/e/1FAIpQLScFKsKkAJI7mhCr7K9rEIOpqIDThrWxuvxnwUq2XkHyG154vQ/viewform) (required) - [ ] PR targets the `develop` branch (stacked on #3931) - [ ] Documentation is updated - [ ] New code is tested in main example apps, including all possible scenarios - [ ] SampleApp iOS and Android - [x] Expo iOS and Android
4 of 6 tasks
isekovanic
added a commit
that referenced
this pull request
Nov 12, 2026
Ticket: https://linear.app/stream/issue/RN-485 ## π― Goal Bring the AI agent work we shipped on `v9-release` to V10: the same components, APIs and sample apps, adapted to stream-chat 10 and the V10 i18n. The long-press Copy/Edit menu on the person's own messages isn't here. It isn't on v9 either, and it comes later as a V10 feature. ## π Implementation details The branch is one commit per v9 PR, in merge order, then the V10-only changes, so it can be reviewed commit by commit. **Ported from v9** - #3920 parse agent reply steps from message attachments - #3921 AI translations through the chat SDK, sample on the agents router - #3924 agent reply steps: reasoning, tool calls, approvals - #3926 native `AIAgent` module for the install id and tool-call sign-in - #3928 message views, reply actions and icon overrides - #3929 `AIConversation` on LegendList - #3931 chat options, suggestions and attachment preview fixes - #3933 client tool runner - #3934 polish of agent conversations - #3936 generated UI with an A2UI renderer - #3944 follow-up polish of agent conversations - #3958 A2UI list-row secrets and malformed messages - #3957 conversation state with `AIConversationController` - #3960 `AIConversation` in its own entry point - #3963 the crew assistant demo app - #3974 alignment with the other SDKs - #3975 StrictMode-safe controller lifecycle and `custom.uiAction` docs - #3976 crew drawer layout - #3978 the samples on `@stream-io/stream-agents` 0.4.2, in place of calypso and their own chat layer Where `develop` had already changed the same lines (comment cleanups, the i18n move, a11y docs), `develop`'s version wins. **What's different on V10** - **Fields under `custom`.** stream-chat 10 keeps `mime_type`, `file_size`, `ai_generated`, `generating`, `ai_ui` and the step fields under `custom`. `parseMessageParts` and `fromStreamMessage` lift them with a small `withCustomFields` helper, as the React SDK does. - **Translation keys.** The 55 AI and A2UI strings use catalog keys with their English inline, named after React's (`aiComponents.replyActions.copy.accessibilityLabel`, `a2ui.modal.close.label`, β¦), and `keys.ts` is regenerated. `useAITranslation` returns core's typed `t`. - **Entry isolation.** The AI entry points may reach V10's translation context. `@legendapp/list` and `react-native-keyboard-controller` are AI-only packages. - **LegendList stays at 3.7.1**, the version v9 resolves. 3.7.2 keeps fewer rows mounted after a jump, so the reused-rows test now only compares rows that stay mounted. It passes on both versions. - **Sample data layer.** Both samples use stream-agents' `AgentChat`, whose stream-chat 10 reader reads `channel.messagePaginator`, activates the channel so the client reloads it after a reconnect, and loads older pages with `toTail()`. The samples read the reply fields from `custom`, and the crew sample uploads voice memos with `uploadChannelFile`. - **Sample dependencies.** Both samples depend on `@stream-io/state-store` and `use-sync-external-store`, and pin `@stream-io/state-store` to one copy in `tsconfig` paths. They stay on stream-chat `^10.0.1`: stream-agents asks for `^10.0.2`, but only when it builds its own client, and the samples hand it theirs. ## π¨ UI Changes None compared with v9. The screenshots are in the v9 PRs above. ## π§ͺ Testing `yarn lint`, `yarn typecheck`, `test:typecheck`, the unit tests and `yarn build` pass, and both samples typecheck. On an Android emulator and an iOS simulator: - **ChatGPT sample:** reasoning, the device time tool, the generated Lisbon plan, the drawer, reopening a conversation from history, and starters on a new one. - **Crew sample:** - all eight flows, including the approval cards, the checklist and form actions, and the agent-first ones; - the equipment deep link, photo troubleshooting and the team lead view; - voice memos, recorded on the simulator with `say` and answered; - the clock-out form on iOS. - **Reconnect:** the network was dropped mid-reply on both samples. The client recovered the active channel, and the full reply appeared. - **Older messages:** with the page size briefly set to 3, older pages loaded until there were none left. After the move to stream-agents, both samples were checked again on both platforms: replies, the device time tool, the agent-first Clock in flow, the drawer, reopening from history, handbook sources, and an approval card into A2UI. Reconnect, older pages and voice memos were tested on the samples' previous chat layer, not again since. Not tested here: voice mode (the simulator's speech recognition doesn't hear `say`), dictation, Cal.com sign-in, and charts or photos in the ChatGPT sample. ## βοΈ Checklist - [x] I have signed the [Stream CLA](https://docs.google.com/forms/d/e/1FAIpQLScFKsKkAJI7mhCr7K9rEIOpqIDThrWxuvxnwUq2XkHyG154vQ/viewform) (required) - [x] PR targets the `develop` branch - [x] Documentation is updated - [ ] New code is tested in main example apps, including all possible scenarios - [ ] SampleApp iOS and Android - [x] Expo iOS and Android
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket: https://linear.app/stream/issue/RN-485
Stacked on #3929.
π― Goal
The composer extras of StreamChatAI's
AIComposerViewand React'sAIMessageComposer(#3376): chat options such as "Think harder", the chip of the one picked, and starter suggestions for an empty conversation.π Implementation details
Chat options (
message-composer/ChatOptions.tsx)AIChatOptionis React's type:id,title, and optionaldescription,icon,shortTitleandcustomData.ComposerViewtakeschatOptionsand lists them in its sheet under Camera and Photos, as Swift's picker does. Picking one closes the sheet; picking the picked one again removes it.SelectedChatOptionChip, Swift's look), and is sent with each message aschatOptioninonSendMessage. Like Swift and Juno, it stays picked after sending, until the person removes it.statestore aschatOption, so an app that passes its own store can read it.useComposerChatOptionreads it inside the composer.ChatOptionsListandSelectedChatOptionChipare exported for custom composers.AISuggestions(suggestions/)suggestionsandonSelect. A row of chips scrolling sideways, two lines each at most 160 wide, as Swift'sSuggestionsView. Where it goes is up to the app.Attachment preview
Image, as React's preview does. The built-in pickers only pick photos, so this shows only with a custommediaPickerService.Also
chatOptionsandsuggestionstheme sections, with Swift's colours from the shared tokens. Sizes and spacing come from the tokens.arandhecould use a native speaker's look.ExpoChatGPTSample
reasoningEffort: 'high', as Juno does.π¨ UI Changes
The sheet with "Think harder" under Camera and Photos, the "Think" chip above the input, and the starters above the composer on a new conversation.
π§ͺ Testing
thinkHarder: truewith the chip andfalsewithout.yarn lint,yarn typecheck,yarn buildand the unit tests pass.βοΈ Checklist
developbranch (stacked on feat(ai-components): AIConversation on LegendListΒ #3929)