Skip to content

Fix SEG _gaussian_blur_2d: finite blur_sigma is ignored (inverted infinite-blur branch)#14187

Open
XIONGPEILIN wants to merge 2 commits into
huggingface:mainfrom
XIONGPEILIN:fix-seg-finite-sigma-inverted-branch
Open

Fix SEG _gaussian_blur_2d: finite blur_sigma is ignored (inverted infinite-blur branch)#14187
XIONGPEILIN wants to merge 2 commits into
huggingface:mainfrom
XIONGPEILIN:fix-seg-finite-sigma-inverted-branch

Conversation

@XIONGPEILIN

@XIONGPEILIN XIONGPEILIN commented Jul 14, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes a branch inversion in SmoothedEnergyGuidanceHook / _gaussian_blur_2d
(src/diffusers/hooks/smoothed_energy_guidance_utils.py) where the finite-sigma and
infinite-blur code paths are swapped. As a result any finite blur_sigma is silently
discarded and replaced by a uniform (spatial-mean) query, so sweeping finite sigma has no effect.

The bug

is_inf = sigma > sigma_threshold_inf
...
if is_inf:                       # sigma "infinite"
    ... 2D Gaussian conv ...     # <- should be the uniform/mean case
else:                            # finite sigma
    query_slice[:] = query_slice.mean(dim=(-2, -1), keepdim=True)   # <- discards sigma

This is inverted relative to:

  1. This code's own documented behavior. The guider/pipeline doc for seg_blur_sigma states:
    "Setting this value greater than 9999.0 results in infinite blur, which means uniform queries."
    i.e. sigma > threshold should give the spatial mean, and a finite sigma should be an actual
    Gaussian blur.
  2. The original SEG-SDXL implementation this function is "Copied/Modified from"
    (pipeline_seg.py#L171-L175):
    if not self.inf_blur:   # finite sigma
        query_ptb = gaussian_blur_2d(query_ptb, kernel_size, self.blur_sigma)
    else:                   # infinite
        query_ptb[:] = query_ptb.mean(dim=(-2, -1), keepdim=True)

Because the default seg_blur_sigma=9999999.0 always takes the (huge-sigma ≈ uniform) conv branch,
the bug is masked for default usage and only surfaces when a finite sigma is requested.

Reproduction

import math, torch
from diffusers.hooks.smoothed_energy_guidance_utils import _gaussian_blur_2d

q = torch.randn(2, 1024, 8)  # 1024 = 32x32 image-token grid
def run(sig):
    ks = math.ceil(6 * sig) + 1 - math.ceil(6 * sig) % 2
    return _gaussian_blur_2d(q.clone(), ks, sig, 9999.9)

print(torch.equal(run(4.0), run(16.0)))   # main: True  -> finite sigma has no effect

On main, different finite sigmas are byte-identical (max|sig4 - sig32| = 0.0). With this PR they
differ (larger sigma smooths more), while sigma > threshold returns the exact spatial mean.

Fix

Swap the two branches: is_inf → spatial mean (exact uniform queries); finite sigma → the real 2D
Gaussian blur of standard deviation sigma.

Note on behavior change

For the default (sigma > threshold) the output changes from a huge-kernel Gaussian conv (an
approximate uniform) to the exact spatial mean — matching the documented "uniform queries".
Finite blur_sigma values now actually control the blur amount. SEG is marked experimental in the
code, so this may affect users who relied on the (buggy) finite-sigma-as-uniform behavior.

Before submitting

  • Read the contributor guideline.
  • ruff check / ruff format pass on the changed file.
  • Happy to add a small regression test (finite sigmas differ, inf == spatial mean) if preferred.

…d infinite-blur branch)

In `_gaussian_blur_2d`, the `is_inf` (sigma > threshold) and finite-sigma branches
are swapped relative to both this function's documented behavior and the original
SEG-SDXL implementation it is "Copied/Modified from":

- The guider/hook docs state "setting `blur_sigma` greater than 9999.0 results in
  infinite blur, which means uniform queries", i.e. sigma>threshold should yield the
  spatial mean, and a finite sigma should be an actual Gaussian blur.
- Original SEG-SDXL (pipeline_seg.py L171-175): `if not inf_blur: gaussian_blur_2d(...);
  else: query.mean(...)`.

The current code does the opposite: `if is_inf:` runs the Gaussian conv and `else:`
(finite sigma) replaces the query with its spatial mean. As a result any finite
`blur_sigma` is discarded and collapses to a uniform query, so sweeping finite sigma
has no effect (e.g. sigma=4/16/32 produce byte-identical outputs). The bug is masked
by the default `seg_blur_sigma=9999999.0`, which always took the conv branch (and with
such a large sigma the kernel is ~uniform anyway).

Fix: swap the branches so infinite blur -> spatial mean (exact uniform), finite sigma
-> real 2D Gaussian blur of std `sigma`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added size/S PR with diff < 50 LOC hooks and removed size/S PR with diff < 50 LOC labels Jul 14, 2026
Covers the fixed behavior: different finite `blur_sigma` values produce different
outputs (and are not the uniform spatial mean), and `sigma` above the threshold
equals the exact spatial mean. These tests fail on the previous (inverted-branch)
implementation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added size/M PR with diff < 200 LOC tests and removed size/M PR with diff < 200 LOC labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant