Wan-Animate-2 (authored by @kelseyee) - #14413

Model (`transformer_wan_animate_2.py`):
- Replace the `forward(*args, method=...)` dispatch and the split
  `forward_ref`/`forward_gen` with a single documented
  `forward(..., kv_cache_mode="extract"|"cached")` returning
  `Transformer2DModelOutput`, following the Flux2 KV-cache precedent. The
  `SelfAttention`/`CrossAttention` pre/post split becomes a regular
  `WanAnimate2Attention` (`AttentionModuleMixin`) with processors that run
  through `dispatch_attention_fn` - native SDPA by default, any backend via
  `set_attention_backend`; only the in-context generation path is pinned to
  `flex`, since its attention pattern is expressed as a `BlockMask`. The hard
  `flash_attn` requirement is gone.
- `IncontextAttentionBlock` was a pure pass-through around `AttentionBlock`;
  merged into one `WanAnimate2TransformerBlock` (checkpoint keys lose the
  `.block.` segment, handled in the single-file mapping).
- KV cache is a `WanAnimate2KVCache` object instead of bare dicts passed
  through `forward`. Accelerate hooks copy dict arguments, so the dict version
  breaks under `enable_model_cpu_offload` (the reference pass fills a copy and
  the generation pass KeyErrors); the object passes through by reference, and
  `_skip_keys = ["kv_cache"]` covers group offloading.
- Remove all autocast in favour of the `transformer_wan.py` dtype discipline
  (fp32 modulation with `.type_as` casts at block boundaries), so the model
  runs natively in bf16 and is no longer CUDA-only in principle. Replace the
  local float64 `sinusoidal_embedding_1d` with the existing `Timesteps` class.
- Remove dead code: the unreachable padding mask in the reference path (the
  pipeline always fills `seq_len` exactly), `init_weights`,
  `load_from_official_state_dict`, and the unused `window_size`/`qk_norm`/
  `sparse_type`/`log_scale` config flags (`log_scale` ships as 0.0, making the
  flex `score_mod` a no-op).

Pipeline: call sites updated to the merged forward, autocast wrappers replaced
with explicit casts at the call boundary.

Also:
- Fill in `convert_wan_animate_2_transformer_to_diffusers` with the actual
  key mapping (block unwrap + attention renames); it was a prefix-strip no-op.
- Revert the `pipeline_utils.py` try/except import shims - the stub exception
  classes silently break real `except OfflineModeIsEnabled` handling; upgrade
  `huggingface_hub` to the `setup.py` pin instead.
- Drop the modular pipeline for now; it needs its own pass and is not part of
  the initial release surface.

Numerics: the refactored model matches the reference implementation at
2.47e-05 max relative difference in fp32 over all 40 layers on real weights,
and end-to-end outputs match the reference pipeline to a max pixel difference
of 7e-5 (PSNR 119 dB) when kernels and environment are held fixed. The
checkpoint key rename is a pure rename - all 1303 tensors bitwise identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A list of frames does not carry the rate it was sampled at, so a pipeline that
has to resample its input to the frame rate the model works at cannot get that
number from `load_video` — even though imageio hands it to us and we throw it
away. Add an opt-in `return_fps`; GIFs get it from the frame duration.

Opt-in, so every existing caller keeps returning a plain list of images.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The pipeline reached for decord and cv2 to do what the processors already do.
decord was an undeclared dependency imported inside `__call__`, and cv2 is not
a diffusers requirement — it is in the deps table but not in `install_requires`,
and only consisid and `export_utils` touch it, both behind local imports.

`WanAnimateImageProcessor` already letterboxes for `WanAnimatePipeline`, which
is what `padding_resize` and `resize_by_area` were hand-rolling: keep the aspect
ratio, fill the remainder with black. Add the video counterpart and use both, so
the driving video arrives as frames from `load_video` like every other video
pipeline takes it, with `driving_video_fps` carrying the one thing a frame list
cannot — decord used to read the source rate itself, and the 30 -> 24 resample
is load-bearing.

Also drop the `seed` argument in favour of the `generator` we already accept,
and sample with `self.scheduler` instead of building a scheduler per call from
`flow_solver` and `sample_shift`. The registered scheduler was dead weight
before; the checkpoints now carry the right one, and swapping it is documented.
`_encode_vae` was defined but unused while its body was inlined three times.

Preprocessing is equivalent, not identical: output dimensions and resampled
frame indices match exactly, and content-aligned the difference is 0.2% of full
range — PIL lanczos against cv2 INTER_AREA. PIL also centres the paste one row
lower than cv2 did; each path's crop follows its own paste. Seeded end to end on
the distilled model that lands at 23.7 dB, the same band the attention refactor
already sits in, with an identical contact sheet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Seed-matched against the approved commit, everything the pipeline hands the
transformer is bit-identical -- noise, timesteps, text embeddings, geometry --
except the resized pixels, and the entire pixel residual traced to three
resize facts, each verified in isolation (decode and normalization are proven
byte-identical):

- kernel choice: bilinear matches the driving frames' `INTER_LINEAR` upscale
  (same filter; only exact-half ties round differently, at most one 8-bit
  level per pixel), bicubic measures closest to `INTER_AREA` for the
  reference image's downscale of the PIL kernels diffusers exposes
- paste placement: cv2 letterboxes at `(height - src_h) // 2`, PIL at
  `height // 2 - src_h // 2` -- one row apart when the frame is even and the
  content odd, previously the largest input difference
- the interim `WanAnimateVideoProcessor` inherited `WanAnimateImageProcessor`'s
  `__init__`, whose bare `super().__init__()` re-registers every shared config
  field with the parent's defaults -- `resample` was silently lanczos

`WanAnimate2VideoProcessor` replaces it: one class, its own
`register_to_config` init that does not chain into the decorated parents, the
reference paste convention, and per-instance kernels (bicubic for the
reference image, bilinear for the driving video). `WanAnimateImageProcessor`
and the merged Wan-Animate pipeline are untouched.

First-segment output agreement with the approved commit at matched seed is
27.5 dB (base) / 28.9 dB (distilled), above the 25.1 dB the approved commit
scores against itself when only the attention backend changes. Divergence in
later segments is the chained conditioning amplifying any perturbation,
numerical noise included, and is documented where the processors are built.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…sets

- tests/modular_pipelines/wan_animate_2/: ModularPipelineTesterMixin +
  ModularGuiderTesterMixin against YiYiXu/tiny-wan-animate-2-modular and
  -distilled-modular (25 passed / 15 skipped); batch tests skipped (the
  pipeline is unbatched), guider test threshold lowered with rationale
- Use randn_tensor for segment noise in both pipelines so CPU generators
  work; CUDA-generator path unchanged (parity re-verified bit-identical)
- Describe every InputParam (templates where available); no auto-docstring
  TODOs remain
- Narrow the core denoise steps' outputs to segment_frames — the only
  product the decode step consumes
- Make the distilled blockset file self-contained: it assembles its own
  image/video encoder groups from the leaf blocks instead of importing
  the base blockset's classes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Modular-only pipeline page (ModularPipeline.from_pretrained example with
offloading + compile, both presets, area-based height/width semantics) and
the transformer model page; toctree entries sorted. References the official
Wan-AI hub ids, which will need the converted weights and
modular_model_index.json before the examples run as written.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o pr-14413

Under eager initialization (DIFFUSERS_SLOW_IMPORT, used by the doc build)
`pipelines` -> `pipeline_wan_animate_2` -> `modular_pipelines.wan_animate_2`
-> `modular_pipeline` -> `pipelines` is a cycle; importing the processor
inside `__init__` breaks it. Goes away entirely with the standard
pipeline's pre-merge removal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>