git404hub

what is occlusionformer fr?

fudancvl/occlusionformer — explained in plain English

Analysis updated 2026-05-18

16PythonAudience · researcherComplexity · 4/5Setup · hard

tl;dr

Inference code and demo for the ICML 2026 OcclusionFormer paper, which composes overlapping objects with explicit Z-order on top of FLUX.

vibe map

mindmap
  root((OcclusionFormer))
    Inputs
      Layout JSON
      FLUX base model
      OcclusionFormer checkpoint
    Outputs
      Layered image
      Correct Z order
    Use Cases
      Layout to image research
      Occlusion ablations
      Demo via Streamlit
    Tech Stack
      Python
      PyTorch
      FLUX
      Streamlit
      Hugging Face

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Reproduce ICML 2026 results on layout-to-image with overlapping boxes

VIBE 2

Compose a custom scene from a layout JSON with correct front-to-back order

VIBE 3

Benchmark against the SA-Z dataset with amodal annotations

VIBE 4

Try the Streamlit demo to explore Z-order conditioned generation

what's the stack?

PythonPyTorchFLUXStreamlit

how it stacks up fr

fudancvl/occlusionformer920linjerry-stack/capital-studioadya84/ha-world-cup-2026
Stars161616
LanguagePythonPythonPython
Setup difficultyhardeasyeasy
Complexity4/53/52/5
Audienceresearcherresearchergeneral

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1day+

Needs Python 3.11 conda env, the FLUX base model weights, and the OcclusionFormer checkpoint from Hugging Face.

in plain english

OcclusionFormer is a research project from Fudan University that accompanies a paper accepted at the ICML 2026 machine learning conference. It tackles a specific problem in image generation: when you tell an AI model to draw a scene by giving it bounding boxes for each object, and those boxes overlap, current methods often blend the textures together or get the order wrong, so an object that should be behind ends up looking like it is in front. The authors propose handling the front to back order, which they call Z-order, as an explicit step in the model. The approach has three pieces according to the README: each object instance is generated separately, then composed using a method borrowed from volume rendering that decides how much each layer shows through, and finally a queried alignment step keeps each object in its correct spatial position. Alongside the model the team is releasing a dataset called SA-Z, which adds occlusion order and amodal annotations (information about parts of objects hidden behind other objects) to layout data. This repository is the inference and demo package. It contains the model code, a Streamlit web demo, a command line inference script, example layout JSON files, and a requirements file. The model weights and the SA-Z dataset are hosted on Hugging Face, and the paper itself is on arXiv. To run it, the README walks through creating a Python 3.11 conda environment, installing the requirements, downloading the checkpoint, and either starting the Streamlit demo or calling the CLI script with paths to a base FLUX model, the OcclusionFormer checkpoint, and a layout JSON. One open task remains: organizing the amodal annotations on Hugging Face.

prompts (copy fr)

prompt 1
Walk me through setting up a Python 3.11 conda env for OcclusionFormer and downloading the FLUX base plus checkpoint
prompt 2
Show me how to write a layout JSON with overlapping bounding boxes and run the CLI inference script
prompt 3
Help me run the Streamlit demo and understand the volume rendering composition step
prompt 4
Explain how SA-Z amodal annotations are used during training versus inference in OcclusionFormer

Frequently asked questions

what is occlusionformer fr?

Inference code and demo for the ICML 2026 OcclusionFormer paper, which composes overlapping objects with explicit Z-order on top of FLUX.

What language is occlusionformer written in?

Mainly Python. The stack also includes Python, PyTorch, FLUX.

How hard is occlusionformer to set up?

Setup difficulty is rated hard, with roughly 1day+ to a first successful run.

Who is occlusionformer for?

Mainly researcher.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.