anliyuan/feathertalk — explained in plain English
Analysis updated 2026-05-18
Train a personalized talking head model from a short video of a face.
Use the lightweight FeatherHuBERT audio encoder instead of the heavier original HuBERT model.
Export a trained model to ONNX for faster mobile or edge deployment.
Experiment with the MobileOne style UNet for lower compute inference.
| anliyuan/feathertalk | 0whitedev/detranspiler | 2951461586/mulerun-pool | |
|---|---|---|---|
| Stars | 21 | 21 | 21 |
| Language | Python | Python | Python |
| Setup difficulty | hard | hard | moderate |
| Complexity | 5/5 | 4/5 | 3/5 |
| Audience | researcher | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires preparing training video and audio data plus a GPU for training the digital human model.
FeatherTalk is a research project for building lightweight talking head videos, where a still face is animated to speak in sync with an audio clip. It is a cleaned up and extended version of an earlier project called Ultralight Digital Human, and it focuses on training a personalized model from a short video, using smaller audio processing so it can eventually run on mobile devices. The main new piece is a component called FeatherHuBERT. Digital human systems like this normally use a large speech recognition model called HuBERT to turn audio into a feature representation that drives mouth movement, but that model is heavy and more powerful than the task actually needs, since no text recognition is required, only a stable signal of what the mouth should be doing. FeatherHuBERT is a much smaller version trained to copy HuBERT's output while using roughly forty times less compute, and it keeps the same input and output shape so the rest of the training and inference pipeline does not need to change. Other additions include support for a second, mobile-friendly network design called MobileOne alongside the original one, extra training losses that specifically target the mouth region to reduce jitter around the lips, removal of a component called SyncNet from the default training path, and support for resuming training from a checkpoint and exporting models to the ONNX format for faster deployment. To try it, the project provides a downloadable demo package containing a short training video and a pretrained FeatherHuBERT checkpoint, which can be preprocessed with a provided Python script to quickly test the training flow before using your own video. This project is aimed at researchers and developers working on talking head or digital human generation who want a smaller, more deployment-friendly alternative to the heavier original pipeline.
A lightweight framework for training a personalized talking head video model from a short video and driving it with audio.
Mainly Python. The stack also includes Python, PyTorch, HuBERT.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
double-check against the repo, no cap.