git404hub

what is pareto fr?

humancompatibleai/pareto — explained in plain English

Analysis updated 2026-05-18

3Jupyter NotebookAudience · researcherComplexity · 2/5Setup · easy

tl;dr

A research dataset from a large-scale human study measuring whether AI responses to political questions receive balanced approval across political viewpoints, with Jupyter notebooks to reproduce the paper's figures.

vibe map

mindmap
  root((repo))
    What it is
      Research dataset
      AI neutrality study
      Human evaluation
    Data Included
      Model responses CSV
      Survey ratings
      Participant demographics
    Analysis Code
      Jupyter notebooks
      PCA and correlations
      Paper figures
    Research Context
      Multiple AI models
      Prolific participants
      Political questions

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Reproduce the figures and statistical analysis from the PARETO paper on AI political neutrality

VIBE 2

Study how different demographic groups rated AI responses to political questions using the survey and demographic CSV data

VIBE 3

Use the PARETO dataset as a benchmark or comparison baseline in new research on AI political bias or neutrality

what's the stack?

PythonJupyter Notebookpandas

how it stacks up fr

humancompatibleai/paretoabdurrafey237/rag-chatbotzafar-lab/spddb
Stars333
LanguageJupyter NotebookJupyter NotebookJupyter Notebook
Setup difficultyeasymoderatemoderate
Complexity2/53/54/5
Audienceresearchergeneralresearcher

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · easy time til it works · 30min

Run the Jupyter notebooks to reproduce figures, standard Python data science packages needed such as pandas, matplotlib, and scikit-learn.

in plain english

PARETO is a research dataset released alongside an academic paper studying whether AI systems can respond to politically sensitive questions in a way that feels fair to people across the political spectrum. The central question the paper explores is what political neutrality actually means for an AI: not simply avoiding strong opinions, but producing answers that people on different sides of political issues find roughly equally acceptable. The dataset contains responses from multiple AI models to a set of politically framed questions. Each response was shown to a large group of human survey participants (recruited through the Prolific research platform) who rated how much they approved of what the AI said. The survey also collected demographic information and written qualitative feedback. Prolific IDs were hashed to a consistent anonymized identifier to protect participant privacy while keeping responses linkable across the data files. The repository is organized into four main areas: raw AI model responses stored as CSV files, the same response pairings formatted as PNG images for display in the survey interface, the survey results (including numeric ratings and free-text comments from participants), and Jupyter notebooks that reproduce the charts and figures used in the published paper. The analysis code uses principal component analysis and correlation statistics to examine patterns in how different demographic groups rated AI responses. This is not a tool to run or install. It is a data archive for researchers who want to study AI political neutrality, reproduce the paper's findings, or use the survey methodology as a starting point for related work.

prompts (copy fr)

prompt 1
What is the PARETO dataset measuring, and how does it define political neutrality as balanced approval across political viewpoints?
prompt 2
How do I run the final_analyses.ipynb notebook to reproduce the PARETO paper's figures? What Python packages does it require?
prompt 3
What data files are in the survey_data directory of the PARETO dataset, and how are participant responses linked to model responses across files?
prompt 4
How were AI model responses presented to survey participants in the PARETO study? What format are the stimuli PNG files and how are they organized?

Frequently asked questions

what is pareto fr?

A research dataset from a large-scale human study measuring whether AI responses to political questions receive balanced approval across political viewpoints, with Jupyter notebooks to reproduce the paper's figures.

What language is pareto written in?

Mainly Jupyter Notebook. The stack also includes Python, Jupyter Notebook, pandas.

How hard is pareto to set up?

Setup difficulty is rated easy, with roughly 30min to a first successful run.

Who is pareto for?

Mainly researcher.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.