zju-real/perceive-to-reason — explained in plain English
Analysis updated 2026-05-18
Reproduce the paper's training pipeline for a two-stage perception and reasoning vision model.
Evaluate a vision language model on fine-grained visual reasoning benchmarks like V-Star and HR-Bench.
Study the PRA-GRPO reinforcement learning method for alternating multi-role model training.
Try the pretrained P2R model through the hosted demo or download it from Hugging Face.
| zju-real/perceive-to-reason | chandar-lab/semantic-wm | djlougen/hive | |
|---|---|---|---|
| Stars | 30 | 30 | 30 |
| Language | Python | Python | Python |
| Setup difficulty | hard | hard | easy |
| Complexity | 5/5 | 5/5 | 3/5 |
| Audience | researcher | researcher | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires GPUs, running multiple coordinated training services, and downloading large datasets and model checkpoints.
Perceive-to-Reason, also called P2R, is a research project from Zhejiang University that tackles how AI vision models answer detailed questions about images. Many current vision language models handle both looking at an image and reasoning about it in a single tangled step, which makes them worse at tasks that need close visual attention, like spotting a tiny detail in a high resolution photo. This project splits that process into two separate roles: one model that focuses purely on carefully perceiving the image, and a second model that focuses on reasoning through the answer using what the first model reported. The two models are trained together using a method the authors call PRA-GRPO, which is a reinforcement learning strategy that alternates training between the two roles. While one model is being trained, the other is frozen and used only as a fixed reference service, and a third verifier service checks whether the final answers are correct so that reward signals can be sent back to whichever model is currently training. According to the README, this approach performs better than standard vision language model baselines on both benchmarks built around high resolution fine grained detail and more general multimodal reasoning tests. The repository includes the training code, along with scripts to install the environment through conda, download training and evaluation datasets from Hugging Face, and start the reasoner, perceiver, and verifier as separate networked services before launching either training stage. There is also a batch evaluation script for running the trained model against several benchmark datasets. The authors have also released a trained model, a training dataset, a paper, and an interactive demo, all linked from the README. The project builds on an existing training framework called veRL and draws its training data from a few other open research projects. The README does not state a license for this code.
A research project that splits AI image understanding into a separate perceiving model and reasoning model, trained together to improve detailed visual question answering.
Mainly Python. The stack also includes Python, PyTorch, veRL.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
double-check against the repo, no cap.