Fix char-boundary panic and non-prefix rewrite corruption in AI code streaming (APP-5288) - #14936
Draft
warp-agent-staging[bot] wants to merge 2 commits into
Draft
Fix char-boundary panic and non-prefix rewrite corruption in AI code streaming (APP-5288)#14936warp-agent-staging[bot] wants to merge 2 commits into
warp-agent-staging[bot] wants to merge 2 commits into
Conversation
AI fenced-code streaming (`AIBlock::handle_code_section_stream_update` in block.rs and its duplicate in block/cli.rs) stored the previously rendered code length as a raw `usize` and sliced the next full code string at that offset with no `is_char_boundary` check: `view.append_at_end(&code[embedded_view.length..], ctx)`. When the stored length lands inside a multi-byte UTF-8 codepoint (a non-prefix rewrite, or a fence edge), the slice panics with "byte index is not a char boundary". `RequestedCommand::apply_streamed_update`'s shrink path had the same bug class: an unguarded `self.command_text.truncate(command.len())`. Its grow path was already guarded (APP-1956); this fix mirrors that pattern for the shrink direction. Fix: - Factor the append/truncate/reset decision for block.rs and block/cli.rs into a shared `streamed_code_update` (pure, unit tested) plus `apply_streamed_code_update` helper. On grow, only slice when the offset is a valid char boundary; otherwise reset the editor to the full `code` string via `CodeEditorView::reset`. - Extract `apply_streamed_command_text` in requested_command.rs and guard both the grow and shrink branches, falling back to replacing `command_text` wholesale when the offset isn't a valid boundary. - `CodeEditorView::truncate` itself was already panic-safe (it resolves byte offsets against the buffer's rope via `ToBufferCharOffset`, which uses `str::get` rather than raw slicing), so no change was needed there; only the raw string slice/truncate call sites needed guards. Tests: added regression tests for both new pure helpers with multi-byte streamed code/commands, covering grow (append/reset) and shrink (truncate/reset) paths. Co-Authored-By: Warp Agent <agent@warp.dev>
…w-up) Address adversarial-review findings on the initial APP-5288 fix: 1. `streamed_code_update` only checked that the previous length was a UTF-8 char boundary before appending/truncating; it never verified the new value actually extends (or is extended by) the previous value. A boundary-aligned but non-prefix rewrite (e.g. "abc" -> "XYZq", or any same-length correction) would silently corrupt the buffer instead of resetting it. Now compares byte contents via `strip_prefix`/`starts_with` for all three cases (grow, shrink, equal-length), which is also inherently char-boundary-safe. `EmbeddedCodeEditorView` now stores the full previously-rendered text (`rendered_code: String`) instead of just its length, since a correctness check needs the content, not just the length. 2. `RequestedCommandView::apply_streamed_update` had the same flaw in its editor-sync half: it independently re-derived an update by comparing `command_text`'s new length against the *editor's own current length*, which can drift into the same "boundary lines up but content diverged" bug. It's now computed once via `streamed_code_update` and the exact same decision (`StreamedCodeUpdate`) is applied to both `command_text` and the editor, via the new `apply_streamed_command_editor_update`, so they can no longer diverge. 3. Replaced the tests that only invoked the newly-introduced helpers with tests that exercise `apply_streamed_code_update` and `apply_streamed_command_editor_update` against a real `CodeEditorView` (the same production `CodeEditorView`/state transitions that `AIBlock`/`CLISubagentView`/`RequestedCommandView` use), asserting the buffer's actual rendered text for grow, shrink, equal-length rewrite, and reset-of-an-already-populated-editor. Constructing a full `AIBlock` or `RequestedCommandView` in a unit test isn't practical here -- their dependencies are private fields of `TerminalView`, which isn't visible outside `crate::terminal::view`'s module tree without new pub(crate) accessors -- so this is the closest real, non-reimplemented state transition reachable from these modules' own test suites. 4. Visual proof: not captured. Computer use is unavailable in this environment, and reproducing this specific defect (a boundary-aligned, non-prefix streamed rewrite) through the live GUI would require either driving the real app with computer use, or standing up a `crates/integration` real-display GPU test with a mocked non-prefix code stream -- a disproportionate new-test-infrastructure investment for a fix whose only visible effect (in the rare case it's hit at all) is that the code block now shows the correct final text instead of corrupted text or a crash. Flagging this explicitly per review request rather than skipping it silently. Co-Authored-By: Warp Agent <agent@warp.dev>
Contributor
Author
There was a problem hiding this comment.
Overview
This fixes the byte index is not a char boundary panic in streamed AI code by replacing raw length/offset arithmetic with a content-prefix comparison that resets the buffer on any non-prefix rewrite, and applies the same decision to the requested-command text and its editor. The change looks correct; two gaps are left for your judgment rather than blocked on.
Concerns
- No visual proof of the new fallback. The reset path re-renders the buffer wholesale during streaming, so it can plausibly affect scroll position or selection in a way unit tests cannot show, and no recording or screenshot is attached. Capturing it needs either computer use (unavailable to the agent that wrote this) or a new
crates/integrationreal-display test driving a non-prefix code stream — your call on whether that investment is warranted for a fallback whose visible effect is "correct text instead of corrupted text or a crash". - Tests cover the shared decision function and the real
CodeEditorView, but not the three production entry points (AIBlock::handle_code_section_stream_update, itsCLISubagentViewtwin,RequestedCommandView::apply_streamed_update). Constructing those views in a unit test needs privateTerminalViewfields (ai_action_model,cli_subagent_controller) that would require newpub(crate)accessors, which was judged out of scope for a crash fix. Accept the current level or ask for those accessors.
Verdict
Checks: build pass, tests pass, CI green, visual proof missing
Found: 0 critical, 0 important, 2 questions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes a client panic reported via a crash-report bot post and Slack thread:
panic: byte index is not a char boundary while streaming AI code(APP-5288).Root cause: AI fenced-code streaming (
AIBlock::handle_code_section_stream_updateinblock.rs, duplicated inblock/cli.rs) stored the previously-rendered code length as a rawusize(EmbeddedCodeEditorView::length) and sliced the next full code string at that offset with no boundary check:When the stored length lands inside a multi-byte UTF-8 codepoint — e.g. a non-prefix rewrite from the server, or a code-fence edge — this slice panics with exactly the reported message.
A secondary instance of the same bug class:
RequestedCommand::apply_streamed_update's shrink path did an unguardedself.command_text.truncate(command.len()). Its grow path was already guarded (APP-1956) — mirrored that pattern for the shrink direction too.Revision (this push): an adversarial review of the first pass found that a boundary-only check (checking only that the split offset was a valid UTF-8 char boundary) doesn't verify the new value actually extends the old one. A boundary-aligned but non-prefix rewrite — e.g. rendered
"abc"followed by"XYZq"(offset 3 is a valid boundary in both) — would append"q"and silently corrupt the buffer to"abcq"instead of resetting to"XYZq". Same-length rewrites were also missed entirely (anOrdering::Equalshort-circuited to a no-op). Fixed by comparing byte contents (strip_prefix/starts_with), not just byte lengths/boundaries, for grow, shrink, and equal-length cases — this is inherently char-boundary-safe too, sincestrprefix/equality comparisons never panic regardless of alignment.Changes
app/src/ai/blocklist/block.rs:streamed_code_updatenow takes the previous value (not just its length) and returnsAppend/Truncate/Reset/NoOpbased on content, not length.EmbeddedCodeEditorViewnow storesrendered_code: Stringinstead oflength: usize.apply_streamed_code_updateapplies the decision to aCodeEditorView, resetting viaCodeEditorView::reseton any non-prefix-stable rewrite.app/src/ai/blocklist/block/cli.rs: reuses the same shared helper (no duplicated logic).app/src/ai/blocklist/inline_action/requested_command.rs:apply_streamed_updatenow computes theStreamedCodeUpdatedecision once (via the sharedstreamed_code_update, madepub(super)) and applies it identically to bothcommand_textand the editor via the newapply_streamed_command_editor_update, instead of the editor half independently re-deriving its own update from a length-only comparison against its current content (the same bug class, just for the editor).ByteOffset/CharOffsetnewtypes.Linked Issue
Linear: APP-5288
Testing
app/src/ai/blocklist/block_tests.rs:streamed_code_updatecases: append/grow, multi-byte append, boundary-aligned non-prefix grow reset (the"abc"→"XYZq"case), non-prefix grow with multi-byte content, valid shrink, non-prefix shrink reset, equal-length rewrite reset, no-op.CodeEditorViewcases (constructed via the same singleton-mock pattern ascode::editor::view::view_tests) exercising the actualapply_streamed_code_updatefunction used byAIBlock/CLISubagentView: append-then-reset-on-non-prefix-rewrite of an already-populated editor, truncate-on-valid-shrink, reset-on-non-prefix-shrink. These assert the buffer's real rendered text, not just the decision enum.app/src/ai/blocklist/inline_action/requested_command_tests.rs: real-CodeEditorViewcases exercisingapply_streamed_command_editor_update(the exact functionRequestedCommandView::apply_streamed_updateuses to sync its editor): append, reset-on-non-prefix-rewrite of an existing editor, truncate-on-valid-shrink, no-op.AIBlock/CLISubagentView/RequestedCommandViewthemselves: constructing these directly in a unit test would require read access to several privateTerminalViewfields (e.g.ai_action_model,cli_subagent_controller) that aren't visible outsidecrate::terminal::view's own module tree without adding newpub(crate)test accessors to that large, sensitive file. I judged that out of scope for this fix and instead tested the exact shared functions these types call, against a realCodeEditorView— the closest practical "real state transition," and enough to prove the buffer itself ends up correct."byte index is not a char boundary"panic reproduces with the vulnerable slicing pattern../script/format— clean.cargo clippy -p warp --lib --tests -- -D warnings— clean (fixed onemanual_stripclippy finding from the content-comparison rewrite).--all-featureshits pre-existing, unrelated clippy failures inwarp_completer, confirmed present onmasterwithout this change.cargo test -p warp --libforai::blocklist::block(115 tests) andai::blocklist::inline_action::requested_command(16 tests) — all pass. Fullai::blocklist::suite passes (753 tests); onesecret_redactiontest flakes intermittently under parallel execution and is reproducible as pre-existing flakiness onmasteralone, unrelated to this change.crates/integrationreal-display GPU test that mocks a non-prefix code stream through the actual AI conversation pipeline — a disproportionate new-test-infrastructure investment for a fix whose only visible effect, in the rare case it's hit, is that the block now shows the correct final text instead of corrupted text or a crash. Flagging this explicitly rather than skipping it silently.Agent Mode
Conversation: https://staging.warp.dev/conversation/5c0e523f-9044-4c99-9d26-b18751eb982f
Run: https://oz.staging.warp.dev/runs/019ff0fd-9630-7070-adcf-9744f5f41308
This PR was generated with Oz.