One state, multiple modalities
RGB, depth, skeleton, DensePose, segmentation, and optical flow rendered from the same generated state.
Beneath the Darkness
Well-aligned auxiliary modalities are rendered directly from shared generated states of extremely dark scenes, without relying on visible details in the rendered RGB.
Rendering encoded states
We render states encoded from real videos and compare WorldFlow’s outputs with test-set annotations.
Rendering generated states
We render states generated from text prompts and compare WorldFlow’s outputs with pretrained estimators’ predictions from the generated RGB.
What does Stage 2 condition on?
Removing state tokens disrupts scene structure; removing or editing the text prompt mainly affects appearance. Changing the Stage 1 seed provides a reference for structural variation.
“No state” sets state features to zero. “New state seed” changes only the Stage 1 seed; the Stage 2 seed is held fixed.