Skip to content

Fix char-boundary panic and non-prefix rewrite corruption in AI code streaming (APP-5288) - #14936

Draft
warp-agent-staging[bot] wants to merge 2 commits into
masterfrom
factory/app-5288-char-boundary-panic
Draft

Fix char-boundary panic and non-prefix rewrite corruption in AI code streaming (APP-5288)#14936
warp-agent-staging[bot] wants to merge 2 commits into
masterfrom
factory/app-5288-char-boundary-panic

Conversation

@warp-agent-staging

@warp-agent-staging warp-agent-staging Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes a client panic reported via a crash-report bot post and Slack thread: panic: byte index is not a char boundary while streaming AI code (APP-5288).

Root cause: AI fenced-code streaming (AIBlock::handle_code_section_stream_update in block.rs, duplicated in block/cli.rs) stored the previously-rendered code length as a raw usize (EmbeddedCodeEditorView::length) and sliced the next full code string at that offset with no boundary check:

view.append_at_end(&code[embedded_view.length..], ctx);

When the stored length lands inside a multi-byte UTF-8 codepoint — e.g. a non-prefix rewrite from the server, or a code-fence edge — this slice panics with exactly the reported message.

A secondary instance of the same bug class: RequestedCommand::apply_streamed_update's shrink path did an unguarded self.command_text.truncate(command.len()). Its grow path was already guarded (APP-1956) — mirrored that pattern for the shrink direction too.

Revision (this push): an adversarial review of the first pass found that a boundary-only check (checking only that the split offset was a valid UTF-8 char boundary) doesn't verify the new value actually extends the old one. A boundary-aligned but non-prefix rewrite — e.g. rendered "abc" followed by "XYZq" (offset 3 is a valid boundary in both) — would append "q" and silently corrupt the buffer to "abcq" instead of resetting to "XYZq". Same-length rewrites were also missed entirely (an Ordering::Equal short-circuited to a no-op). Fixed by comparing byte contents (strip_prefix/starts_with), not just byte lengths/boundaries, for grow, shrink, and equal-length cases — this is inherently char-boundary-safe too, since str prefix/equality comparisons never panic regardless of alignment.

Changes

  • app/src/ai/blocklist/block.rs: streamed_code_update now takes the previous value (not just its length) and returns Append/Truncate/Reset/NoOp based on content, not length. EmbeddedCodeEditorView now stores rendered_code: String instead of length: usize. apply_streamed_code_update applies the decision to a CodeEditorView, resetting via CodeEditorView::reset on any non-prefix-stable rewrite.
  • app/src/ai/blocklist/block/cli.rs: reuses the same shared helper (no duplicated logic).
  • app/src/ai/blocklist/inline_action/requested_command.rs: apply_streamed_update now computes the StreamedCodeUpdate decision once (via the shared streamed_code_update, made pub(super)) and applies it identically to both command_text and the editor via the new apply_streamed_command_editor_update, instead of the editor half independently re-deriving its own update from a length-only comparison against its current content (the same bug class, just for the editor).
  • Kept the change scoped to the boundary/content guard itself; did not migrate the surrounding code to the repo's ByteOffset/CharOffset newtypes.

Linked Issue

Linear: APP-5288

Testing

  • app/src/ai/blocklist/block_tests.rs:
    • Pure streamed_code_update cases: append/grow, multi-byte append, boundary-aligned non-prefix grow reset (the "abc""XYZq" case), non-prefix grow with multi-byte content, valid shrink, non-prefix shrink reset, equal-length rewrite reset, no-op.
    • Real-CodeEditorView cases (constructed via the same singleton-mock pattern as code::editor::view::view_tests) exercising the actual apply_streamed_code_update function used by AIBlock/CLISubagentView: append-then-reset-on-non-prefix-rewrite of an already-populated editor, truncate-on-valid-shrink, reset-on-non-prefix-shrink. These assert the buffer's real rendered text, not just the decision enum.
  • app/src/ai/blocklist/inline_action/requested_command_tests.rs: real-CodeEditorView cases exercising apply_streamed_command_editor_update (the exact function RequestedCommandView::apply_streamed_update uses to sync its editor): append, reset-on-non-prefix-rewrite of an existing editor, truncate-on-valid-shrink, no-op.
    • Note on AIBlock/CLISubagentView/RequestedCommandView themselves: constructing these directly in a unit test would require read access to several private TerminalView fields (e.g. ai_action_model, cli_subagent_controller) that aren't visible outside crate::terminal::view's own module tree without adding new pub(crate) test accessors to that large, sensitive file. I judged that out of scope for this fix and instead tested the exact shared functions these types call, against a real CodeEditorView — the closest practical "real state transition," and enough to prove the buffer itself ends up correct.
  • Verified regression-test validity: reverted to the pre-fix logic locally and confirmed the exact "byte index is not a char boundary" panic reproduces with the vulnerable slicing pattern.
  • ./script/format — clean.
  • cargo clippy -p warp --lib --tests -- -D warnings — clean (fixed one manual_strip clippy finding from the content-comparison rewrite). --all-features hits pre-existing, unrelated clippy failures in warp_completer, confirmed present on master without this change.
  • cargo test -p warp --lib for ai::blocklist::block (115 tests) and ai::blocklist::inline_action::requested_command (16 tests) — all pass. Full ai::blocklist:: suite passes (753 tests); one secret_redaction test flakes intermittently under parallel execution and is reproducible as pre-existing flakiness on master alone, unrelated to this change.
  • Visual proof: not captured. Computer use is unavailable in this environment. Reproducing this specific defect (a boundary-aligned, non-prefix streamed code/command rewrite) end-to-end in the live GUI would require either computer use, or standing up a new crates/integration real-display GPU test that mocks a non-prefix code stream through the actual AI conversation pipeline — a disproportionate new-test-infrastructure investment for a fix whose only visible effect, in the rare case it's hit, is that the block now shows the correct final text instead of corrupted text or a crash. Flagging this explicitly rather than skipping it silently.

Agent Mode

  • Warp Agent Mode - This PR was created via Warp's AI Agent Mode

Conversation: https://staging.warp.dev/conversation/5c0e523f-9044-4c99-9d26-b18751eb982f
Run: https://oz.staging.warp.dev/runs/019ff0fd-9630-7070-adcf-9744f5f41308

This PR was generated with Oz.

AI fenced-code streaming (`AIBlock::handle_code_section_stream_update`
in block.rs and its duplicate in block/cli.rs) stored the previously
rendered code length as a raw `usize` and sliced the next full code
string at that offset with no `is_char_boundary` check:
`view.append_at_end(&code[embedded_view.length..], ctx)`. When the
stored length lands inside a multi-byte UTF-8 codepoint (a non-prefix
rewrite, or a fence edge), the slice panics with "byte index is not a
char boundary".

`RequestedCommand::apply_streamed_update`'s shrink path had the same
bug class: an unguarded `self.command_text.truncate(command.len())`.
Its grow path was already guarded (APP-1956); this fix mirrors that
pattern for the shrink direction.

Fix:
- Factor the append/truncate/reset decision for block.rs and
  block/cli.rs into a shared `streamed_code_update` (pure, unit
  tested) plus `apply_streamed_code_update` helper. On grow, only
  slice when the offset is a valid char boundary; otherwise reset the
  editor to the full `code` string via `CodeEditorView::reset`.
- Extract `apply_streamed_command_text` in requested_command.rs and
  guard both the grow and shrink branches, falling back to replacing
  `command_text` wholesale when the offset isn't a valid boundary.
- `CodeEditorView::truncate` itself was already panic-safe (it
  resolves byte offsets against the buffer's rope via
  `ToBufferCharOffset`, which uses `str::get` rather than raw
  slicing), so no change was needed there; only the raw string
  slice/truncate call sites needed guards.

Tests: added regression tests for both new pure helpers with
multi-byte streamed code/commands, covering grow (append/reset) and
shrink (truncate/reset) paths.

Co-Authored-By: Warp Agent <agent@warp.dev>
@cla-bot cla-bot Bot added the cla-signed label Aug 11, 2026
…w-up)

Address adversarial-review findings on the initial APP-5288 fix:

1. `streamed_code_update` only checked that the previous length was a UTF-8
   char boundary before appending/truncating; it never verified the new
   value actually extends (or is extended by) the previous value. A
   boundary-aligned but non-prefix rewrite (e.g. "abc" -> "XYZq", or any
   same-length correction) would silently corrupt the buffer instead of
   resetting it. Now compares byte contents via `strip_prefix`/`starts_with`
   for all three cases (grow, shrink, equal-length), which is also
   inherently char-boundary-safe. `EmbeddedCodeEditorView` now stores the
   full previously-rendered text (`rendered_code: String`) instead of just
   its length, since a correctness check needs the content, not just the
   length.

2. `RequestedCommandView::apply_streamed_update` had the same flaw in its
   editor-sync half: it independently re-derived an update by comparing
   `command_text`'s new length against the *editor's own current length*,
   which can drift into the same "boundary lines up but content diverged"
   bug. It's now computed once via `streamed_code_update` and the exact same
   decision (`StreamedCodeUpdate`) is applied to both `command_text` and the
   editor, via the new `apply_streamed_command_editor_update`, so they can
   no longer diverge.

3. Replaced the tests that only invoked the newly-introduced helpers with
   tests that exercise `apply_streamed_code_update` and
   `apply_streamed_command_editor_update` against a real `CodeEditorView`
   (the same production `CodeEditorView`/state transitions that
   `AIBlock`/`CLISubagentView`/`RequestedCommandView` use), asserting the
   buffer's actual rendered text for grow, shrink, equal-length rewrite, and
   reset-of-an-already-populated-editor. Constructing a full `AIBlock` or
   `RequestedCommandView` in a unit test isn't practical here -- their
   dependencies are private fields of `TerminalView`, which isn't visible
   outside `crate::terminal::view`'s module tree without new pub(crate)
   accessors -- so this is the closest real, non-reimplemented state
   transition reachable from these modules' own test suites.

4. Visual proof: not captured. Computer use is unavailable in this
   environment, and reproducing this specific defect (a boundary-aligned,
   non-prefix streamed rewrite) through the live GUI would require either
   driving the real app with computer use, or standing up a
   `crates/integration` real-display GPU test with a mocked non-prefix code
   stream -- a disproportionate new-test-infrastructure investment for a
   fix whose only visible effect (in the rare case it's hit at all) is that
   the code block now shows the correct final text instead of corrupted
   text or a crash. Flagging this explicitly per review request rather than
   skipping it silently.

Co-Authored-By: Warp Agent <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot changed the title Fix char-boundary panic in AI code streaming (APP-5288) Fix char-boundary panic and non-prefix rewrite corruption in AI code streaming (APP-5288) Aug 11, 2026

@warp-agent-staging warp-agent-staging Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This fixes the byte index is not a char boundary panic in streamed AI code by replacing raw length/offset arithmetic with a content-prefix comparison that resets the buffer on any non-prefix rewrite, and applies the same decision to the requested-command text and its editor. The change looks correct; two gaps are left for your judgment rather than blocked on.

Concerns

  • No visual proof of the new fallback. The reset path re-renders the buffer wholesale during streaming, so it can plausibly affect scroll position or selection in a way unit tests cannot show, and no recording or screenshot is attached. Capturing it needs either computer use (unavailable to the agent that wrote this) or a new crates/integration real-display test driving a non-prefix code stream — your call on whether that investment is warranted for a fallback whose visible effect is "correct text instead of corrupted text or a crash".
  • Tests cover the shared decision function and the real CodeEditorView, but not the three production entry points (AIBlock::handle_code_section_stream_update, its CLISubagentView twin, RequestedCommandView::apply_streamed_update). Constructing those views in a unit test needs private TerminalView fields (ai_action_model, cli_subagent_controller) that would require new pub(crate) accessors, which was judged out of scope for a crash fix. Accept the current level or ask for those accessors.

Verdict

Checks: build pass, tests pass, CI green, visual proof missing

Found: 0 critical, 0 important, 2 questions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant