Open-World Scene Deocclusion

K.M.S.J. Kumarasinghe1 Neil A. Dodgson1 Fang-Lue Zhang2

1Victoria University of Wellington 2University of New South Wales

WACV 2027

Deocclusion across three scene types: an object hidden behind another, interlocked objects, and an object cut by the image frame
Three kinds of occlusion: (a) an object hidden behind another, (b) interlocked objects that occlude each other, and (c) an object cut by the image frame. From the original scene (I), the method recovers the background and every object as a complete layer (II), which can then be rearranged into a new scene (III).

Abstract

Recovering the complete appearance of occluded objects from a single image is a fundamental problem for scene understanding and image editing. Real-world scenes are challenging due to clutter, complex spatial occlusions, and diverse object categories. Existing methods often struggle when occlusion involves background surfaces or objects that are not explicitly detected, and fail to resolve the cyclic occlusion dependencies that frequently occur in natural scenes.

We introduce a training-free, open-world scene deocclusion framework that recovers the full amodal appearance of detected or user-selected objects from a single RGB image, decomposing the scene into depth-ordered RGBA layers that are geometrically plausible, semantically faithful, and visually consistent with the visible scene. Extensive experiments show that our approach improves scene-level deocclusion, producing coherent and realistic reconstructions in real-world settings.

Method

The pipeline works in two stages. Scene-level reasoning runs once per image and decides what hides what. Instance-level completion then runs once per object and paints in what was hidden.

Overview of the layered amodal scene deocclusion framework

Our contribution Off-the-shelf model

  1. Instance segmentation

    YOLO boxes prompt SAM2 for precise masks. Users can also point at any object the detector misses.

  2. Geometric scene reasoning

    Pairwise occlusion and dense depth are combined. Depth-validated detection finds background occluders that segmentation misses, and objects cut by the frame are flagged for outpainting.

  3. Cycle-safe layer ordering

    Mutual-occlusion cycles are collapsed into strongly connected components, objects are grouped by their longest occlusion chain, and depth orders objects within each group.

  4. Inpainting mask generation

    Each object's mask covers every object that hides it, directly or indirectly, kept within depth-consistent reach of the occluded region.

  5. Prompt refinement

    Open-vocabulary tags are ranked against the object with CLIP to choose a precise text prompt.

  6. Amodal completion

    Diffusion in- and outpainting completes each object, repeating until boundary, colour, texture and plausibility scores pass. Each object is saved as an RGBA layer.

Interactive demo

Choose a scene and press Deocclude. Then rearrange the result: drag objects, change which one is in front, resize, flip or hide them. Completed regions stay consistent wherever an object goes.

Citation

The BibTeX entry will be added once the paper is published.