what is unibench fr?

facebookresearch/unibench — explained in plain English

Analysis updated 2026-07-17 · repo last pushed 2026-06-18

★ 228Jupyter NotebookAudience · researcherComplexity · 3/5ActiveSetup · moderate

vibe map

mindmap
  root((repo))
    What it does
      Benchmarks vision language models
      Compares 60 plus models
      Runs 40 plus tasks
    Tech stack
      Jupyter Notebook
      Python
      Embeddings
    Use cases
      Compare model performance
      Test custom models
      Add new benchmarks
      View precomputed results
    Audience
      AI researchers
      Product teams
    Setup
      Install via pip
      CLI or Python scripts

mindmap root((repo)) What it does Benchmarks vision language models Compares 60 plus models Runs 40 plus tasks Tech stack Jupyter Notebook Python Embeddings Use cases Compare model performance Test custom models Add new benchmarks View precomputed results Audience AI researchers Product teams Setup Install via pip CLI or Python scripts

what do people make with this?

VIBE 1

Compare how 60+ existing vision-language models perform across 40+ established benchmarks without writing evaluation code.

VIBE 2

Test a custom vision-language model against UniBench's standard benchmark suite to see how it stacks up.

VIBE 3

Add a new custom dataset as a benchmark and evaluate all existing models against it automatically.

VIBE 4

Download and analyze precomputed results from researchers to compare models without running new tests.

what's the stack?

Jupyter NotebookPython

how it stacks up fr

	facebookresearch/unibench	facebookresearch/sparsh	krishnaik06/text-summarization-nlp-project
Stars	228	228	198
Language	Jupyter Notebook	Jupyter Notebook	Jupyter Notebook
Last pushed	2026-06-18	2025-02-27	2024-08-17
Maintenance	Active	Stale	Stale
Setup difficulty	moderate	hard	hard
Complexity	3/5	4/5	4/5
Audience	researcher	researcher	developer

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · moderate time til it works · 1h+

Full evaluation suite requires downloading multiple pretrained models and benchmark datasets.

Not specified in the explanation.

prompts (copy fr)

prompt 1

How do I install UniBench and run its full benchmark suite against a pretrained CLIP-style model?

prompt 2

Show me how to add my own image-text dataset as a new benchmark in UniBench and evaluate existing models on it.

prompt 3

How do I download UniBench's precomputed results to compare two vision-language models without running new evaluations?

prompt 4

Explain how UniBench compares image and text embeddings to score a model's classification accuracy.

Frequently asked questions