Encode a feature film into a compact text description of its scenes.
Regenerate a reconstructed version of a film from those text descriptions using AI video models.
Compare original and AI-reconstructed footage side by side in a browser tool.
Experiment with different AI video and audio backends to trade off cost, speed, and quality.
| willhs/lossy | 1038lab/agnes-ai | 3eyedtiger/video2vrcemote | |
|---|---|---|---|
| Stars | 4 | 4 | 4 |
| Language | Python | Python | Python |
| Setup difficulty | hard | easy | easy |
| Complexity | 4/5 | 2/5 | 1/5 |
| Audience | researcher | vibe coder | vibe coder |
Figures from each repo's GitHub metadata at analysis time.
Needs a rented GPU (RunPod) or paid API keys for video, audio, and vision models before the pipeline produces output.
This project takes a whole movie and compresses it into a small text file, then uses AI video and audio generation to rebuild a new version of the film from that text. It calls itself a lossy codec, meaning the rebuilt film is not an exact copy, it is a reconstruction based on written descriptions of what happens in each shot. The process works in stages. First the film is split into roughly two thousand shots using scene detection tools, and keyframes are pulled from each one. Next, an AI vision model looks at sampled frames and writes a detailed description of each shot, including things like camera movement and sound. Those descriptions, plus some audio classification, get bundled into a single JSON file that ends up under one megabyte once compressed. That tiny file is effectively the whole film boiled down to text. To turn the text back into video, the project sends each shot's description to an AI video generation model, which can be run on a rented cloud GPU or through a paid API service, and several different backend options are supported with different costs and speed tradeoffs. Sound effects and music are generated separately by other AI audio models, and any spoken dialogue is recreated using text to speech based on the film's original subtitles. Finally all the generated clips and audio are stitched back together into one output video, and a browser based tool lets you compare the original and reconstructed versions side by side. According to the author's write-up, reconstructing a full feature film this way costs around thirty dollars in rented GPU time plus about a dollar for the description step. The project is written in Python, needs FFmpeg and the uv tool to install its dependencies, and requires API keys for the AI services it depends on. It is released under the MIT license.
A tool that compresses a movie into under 1MB of text shot descriptions, then rebuilds a new version of the film using AI video, audio, and speech generation.
Mainly Python. The stack also includes Python, FFmpeg, Gemini.
MIT license: use, copy, modify, and distribute freely, including for commercial purposes, as long as the copyright notice is kept.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
double-check against the repo, no cap.