Train a simulated robot hand to manipulate tools like scissors or a screwdriver.
Reproduce results from the REGRIND dexterous manipulation research paper.
Retarget human hand demonstration data onto a robot hand's joint structure.
Study reinforcement learning approaches for fine robotic manipulation tasks.
| yunhaif/regrind | 0xtotem/peek-dspy | ant-research/memdreamer | |
|---|---|---|---|
| Stars | 42 | 42 | 42 |
| Language | Python | Python | Python |
| Setup difficulty | hard | moderate | hard |
| Complexity | 5/5 | 3/5 | 5/5 |
| Audience | researcher | developer | researcher |
Figures from each repo's GitHub metadata at analysis time.
Requires installing IsaacSim 5.1.0, IsaacLab 2.3.0, and optionally a Mosek license for the retargeting solver.
REGRIND is a research project for teaching robot hands to manipulate objects using reinforcement learning, a training method where an AI system improves by trial and error rather than being explicitly programmed. It is the code that goes along with a paper called A Minimalist Retargeting Guided Reinforcement Learning Recipe for Dexterous Manipulation, and it focuses specifically on dexterous manipulation, meaning fine, hand-like movements such as picking up and using tools like scissors or a screwdriver. The project works in two main stages. First, human hand motions are retargeted, meaning translated, into movements a specific robot hand can perform, using recorded human demonstrations of using an object. Retargeted example movements are already included in the repository, so someone can skip straight to training if they want. Second, those retargeted movements guide a reinforcement learning process that trains a simulated robot hand, either a LeapHand or a WujiHand model, to perform the task on its own in a physics simulator. Everything runs inside NVIDIA's IsaacSim and IsaacLab robotics simulation software, which needs to be installed separately before this project will work. Training happens with thousands of simulated environments running in parallel to speed up learning, and results can optionally be tracked using a tool called wandb. The retargeting step also depends on an optimization library called Drake, which by default uses a solver called Mosek that requires a free academic or trial license, though a license-free alternative solver is also supported.
A research codebase that trains simulated robot hands to perform dexterous tasks like using scissors, guided by retargeted human hand demonstrations.
Mainly Python. The stack also includes Python, IsaacSim, IsaacLab.
The README does not state a license.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
double-check against the repo, no cap.