lum1104/eibench — explained in plain English
Analysis updated 2026-07-30 · repo last pushed 2025-09-30
Benchmark your vision-language model's ability to recognize emotions in images
Compare your AI model against established baselines like GPT-4, Claude-3, and LLaVA
Evaluate whether a chatbot or assistant can pick up on emotional cues from images
Test if your model can reason about the social context behind someone's emotions
| lum1104/eibench | 410979729/scope-recall | abdullahselek/reducepy | |
|---|---|---|---|
| Stars | 33 | 33 | 33 |
| Language | Python | Python | Python |
| Last pushed | 2025-09-30 | — | 2022-04-03 |
| Maintenance | Quiet | — | Dormant |
| Setup difficulty | moderate | moderate | moderate |
| Complexity | 3/5 | 3/5 | 3/5 |
| Audience | researcher | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires downloading two external image datasets (EmoSet-118K and CAER-S) and access to an LLM API (LLaMA-3 or ChatGPT-3.5) for automatic scoring.
EIBench is a benchmark for testing how well AI models can understand and reason about human emotions from images. It accompanies a 2025 CVPR paper titled "Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models." The project gives researchers a standardized way to measure whether an AI can look at a picture and correctly infer what someone in that picture might be feeling and why. The benchmark comes in two tiers: Basic and Complex. The Basic level tests straightforward emotion recognition, identifying visible emotions in an image. The Complex level goes further, asking the model to reason about emotional nuances and context. To use the benchmark, you download two image datasets (EmoSet-118K and CAER-S), run your AI model against the test questions, and then use another AI model, either LLaMA-3 or ChatGPT-3.5, to score how well your model performed. The project also includes a "long-term coherence" evaluation that checks whether a model's emotional reasoning stays consistent over a conversation. This project is aimed at AI researchers and engineers building vision-language models, systems that combine image understanding with text generation. If you're developing an AI assistant, chatbot, or tool that needs to pick up on emotional cues from images, EIBench lets you compare your model against established baselines like GPT-4, Claude-3, LLaVA, MiniGPT4, and Otter. It answers the question: "Can my model read a room, or is it missing emotional context that a human would catch?" The project is notable for its two-tier structure, which separates surface-level emotion detection from deeper emotional reasoning. Rather than just asking "what emotion is this," the Complex tier probes whether the model can explain the social or contextual factors behind that emotion, a harder problem that today's models still struggle with.
EIBench is a benchmark that tests how well AI models understand and reason about human emotions from images, with two tiers: basic emotion recognition and complex emotional reasoning. It accompanies a 2025 CVPR paper.
Mainly Python. The stack also includes Python, LLaMA-3, ChatGPT-3.5.
Quiet — no commits in 6-12 months (last push 2025-09-30).
No license information is provided in this repository, so usage terms are unclear.
Setup difficulty is rated moderate, with roughly 1h+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
double-check against the repo, no cap.