git404hub

what is omnidirector fr?

lisj575/omnidirector — explained in plain English

Analysis updated 2026-05-18

19Audience · researcherComplexity · 4/5Setup · hard

tl;dr

Research code for AI video generation that applies a reference video's camera movements to animate a single still image.

vibe map

mindmap
  root((repo))
    What it does
      Transfers camera motion to a still image
      Camera grid representation
      Hierarchical prompt expansion
    Tech stack
      Research codebase
      AI video generation
    Use cases
      Reproduce camera movement in new footage
      Study the paper's method
    Audience
      Researchers
      Video production teams
    Notes
      Sparse README
      Paper and arXiv link only

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Reproduce a reference video's camera pans and zooms on a different source image.

VIBE 2

Study the camera grid representation as a way to encode multi-shot camera motion.

VIBE 3

Read the paper and citation to understand the hierarchical prompt expansion agent approach.

what's the stack?

Python

how it stacks up fr

lisj575/omnidirector16nic/comfyui-agnes-ai521xueweihan/hgdoll
Stars191919
LanguagePythonKotlin
Setup difficultyhardmoderatehard
Complexity4/52/54/5
Audienceresearchervibe coderdeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1day+

The README has no installation instructions, usage examples, or model weights, only the paper and links.

in plain english

OmniDirector is the code release for a research paper by teams at Kuaishou Technology, Tsinghua University, and Peking University. The system addresses a specific problem in AI video generation: taking camera movements from a multi-shot video and applying those same movements to animate a still image, without needing a training dataset that pairs the source and target footage together. The core idea is a "camera grid representation," which is the paper's proposed way of encoding camera motion from a sequence of different shots into a format that a model can learn from and apply. Alongside this, the system uses what the authors call a hierarchical prompt expansion agent, which combines multiple types of input signals, including text descriptions and visual cues, to guide the animation process. The practical use case is in video production contexts where you have a reference video whose camera work you want to reproduce: pans, zooms, tracking shots, and so on. OmniDirector is designed to apply those movements to a different source image, effectively letting users direct the camera of a new video by example rather than by manual keyframing. The README for this repository is sparse and contains mainly the paper abstract, author list, citation block, and a link to the project page and arXiv preprint. It does not include installation instructions, code usage examples, or model weights in the README itself. Readers looking for implementation details would need to consult the project page or the arXiv paper directly.

prompts (copy fr)

prompt 1
Summarize the OmniDirector paper abstract in plain English.
prompt 2
Explain what a camera grid representation is and how it encodes camera motion.
prompt 3
Find the arXiv preprint and project page links for OmniDirector.
prompt 4
Help me understand how hierarchical prompt expansion combines text and visual cues here.

Frequently asked questions

what is omnidirector fr?

Research code for AI video generation that applies a reference video's camera movements to animate a single still image.

How hard is omnidirector to set up?

Setup difficulty is rated hard, with roughly 1day+ to a first successful run.

Who is omnidirector for?

Mainly researcher.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.