git404hub

what is pgbouncer-multicore-limit fr?

karan-jadhav/pgbouncer-multicore-limit — explained in plain English

Analysis updated 2026-05-18

3PythonAudience · ops devopsComplexity · 4/5Setup · hard

tl;dr

A benchmark project that measures how running multiple PgBouncer processes in parallel scales PostgreSQL throughput past PgBouncer's single-core limit.

vibe map

mindmap
  root((pgbouncer-multicore-limit))
    What it does
      Benchmarks PgBouncer scaling
      Local Docker Compose test
      AWS measured experiment
    Tech stack
      Python
      Rust
      Terraform
      Docker
    Use cases
      Test PgBouncer process scaling
      Reproduce benchmark locally
      Run measured AWS experiment
    Audience
      Backend engineers
      Database performance researchers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Reproduce a local functional test of running multiple PgBouncer processes against one PostgreSQL database.

VIBE 2

Run the full measured benchmark on AWS to see real throughput and latency numbers at different process counts.

VIBE 3

Use the included Rust load generator to simulate API, export, and connection-cancellation traffic patterns.

what's the stack?

PythonRustTerraformDockerPostgreSQL

how it stacks up fr

karan-jadhav/pgbouncer-multicore-limit0marildo/imago100/geotwitter
Stars333
LanguagePythonPythonPython
Last pushed2015-09-10
MaintenanceDormant
Setup difficultyhardeasymoderate
Complexity4/52/53/5
Audienceops devopsgeneralgeneral

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1h+

Local smoke test needs Docker, Rust, and uv, the full measured benchmark requires an AWS account and Terraform plus Ansible.

No license information is stated in the README.

in plain english

pgbouncer-multicore-limit is a research project that reproduces and measures a specific performance limit in PgBouncer, the popular PostgreSQL connection pooler. PgBouncer normally runs as a single process, which caps it at using one CPU core no matter how powerful the server is. This repo tests what happens when you run several PgBouncer processes side by side, sharing the same pool of database connections, to see how much extra throughput that unlocks. You can run a smaller functional version of the experiment on your own laptop using Docker Compose, which needs Docker, Rust, the uv Python tool, and Terraform installed. A single command builds PostgreSQL, builds PgBouncer from source with checksum verification, builds a custom Rust load testing tool, and then runs a full local test sequence: it loads a fixed dataset, sends traffic through one PgBouncer process, then through four processes sharing a port, checks an exported data stream, and finally validates and files each run as accepted or rejected. The local run uses a small dataset so it is meant for checking the process works, not for drawing performance conclusions. The custom Rust load generator can simulate several kinds of traffic, including steady API requests, data exports, new connections, cancellations, and mixed workloads, and it records detailed timing statistics for each run, down to full latency histograms. The real measured results come from a matching setup on AWS, built and torn down with Terraform and Ansible, and none of that cloud infrastructure gets created until you explicitly run a specific setup command and approve the plan. In the completed AWS test, using 256 clients and a fixed total connection budget, going from one PgBouncer process to eight roughly quadrupled throughput, from about 45,000 to about 210,000 queries per second, while the response time got faster too. At that point PostgreSQL itself became the bottleneck rather than PgBouncer. A blog post linked from the README explains the full methodology and its limitations, and every run raw data, logs, and validation status are saved for inspection, whether the run passed or was rejected. The project includes commands to generate charts and summaries from all accepted results.

prompts (copy fr)

prompt 1
Help me run the local Docker Compose smoke test for pgbouncer-multicore-limit using make local-smoke.
prompt 2
Explain why PgBouncer's single-process design limits it to one CPU core and how this project works around it.
prompt 3
Walk me through setting up the Terraform AWS experiment for this project without accidentally creating cloud resources.
prompt 4
Summarize the queries-per-second results this project found when scaling from 1 to 8 PgBouncer processes.

Frequently asked questions

what is pgbouncer-multicore-limit fr?

A benchmark project that measures how running multiple PgBouncer processes in parallel scales PostgreSQL throughput past PgBouncer's single-core limit.

What language is pgbouncer-multicore-limit written in?

Mainly Python. The stack also includes Python, Rust, Terraform.

What license does pgbouncer-multicore-limit use?

No license information is stated in the README.

How hard is pgbouncer-multicore-limit to set up?

Setup difficulty is rated hard, with roughly 1h+ to a first successful run.

Who is pgbouncer-multicore-limit for?

Mainly ops devops.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.