git404hub

what is fastconstmap fr?

lemire/fastconstmap — explained in plain English

Analysis updated 2026-05-18

28CAudience · developerComplexity · 3/5Setup · easy

tl;dr

A fast, memory-efficient read-only lookup table for mapping strings to numbers, built once and never changed after that.

vibe map

mindmap
  root((fastconstmap))
    What it does
      Read only string to number lookup
      Built once then frozen
      Faster and smaller than dict
    Tech stack
      C
      Python
      pip install
    Variants
      ConstMap fast no checks
      VerifiedConstMap safe checks
    Use cases
      Tokenization
      Category to ID mapping
      Shared memory across workers
    Audience
      Developers
      High throughput systems

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Speed up tokenization by mapping words to integer IDs at high throughput

VIBE 2

Map category or label names to numeric IDs in a classification pipeline

VIBE 3

Share a large lookup table across multiple worker processes using shared memory

what's the stack?

CPython

how it stacks up fr

lemire/fastconstmapcocomelonc/tabbyqhencoder/sysrestoredriver
Stars282929
LanguageCCC
Setup difficultyeasymoderatehard
Complexity3/55/55/5
Audiencedeveloperresearcherops devops

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · easy time til it works · 5min

Installs via pip, requires building or using a prebuilt C extension.

in plain english

Fastconstmap is a Python library that gives you a very fast, read-only lookup table for mapping text strings to numbers. Think of it like a Python dictionary, except it is built once and then frozen, you cannot add or change entries after creation, but in exchange it is roughly twice as fast to look up values and uses about thirteen times less memory than a regular Python dict. The library comes in two flavors. The simpler one, called ConstMap, uses around nine bytes of memory per entry and answers lookups with a single hash calculation plus three array reads. The trade-off is that if you look up a key that was never in the map, you get back garbage data with no warning. The second flavor, VerifiedConstMap, doubles the memory usage to about eighteen bytes per key, but returns None or raises a KeyError when you ask for a missing key, behaving more like a familiar Python dictionary. Both versions can be saved to a file or a block of raw bytes, and loaded back later without rebuilding. For programs that run multiple worker processes at once, the library supports placing the map in shared memory, a chunk of RAM shared across processes, so that millions of lookups can happen simultaneously without each process needing its own copy of the data. You would reach for this library when you have a fixed set of string-to-number mappings that you build once at startup and look up millions of times, such as tokenizing text, mapping category names to integer IDs, or any high-throughput classification task. It is written in C and installed via pip. The full README is longer than what was provided.

prompts (copy fr)

prompt 1
Show me how to install fastconstmap with pip and build a ConstMap from a list of strings
prompt 2
Explain the difference between ConstMap and VerifiedConstMap and when to use each
prompt 3
Help me save a fastconstmap lookup table to a file and load it back later
prompt 4
How would I use fastconstmap's shared memory support across multiple worker processes

Frequently asked questions

what is fastconstmap fr?

A fast, memory-efficient read-only lookup table for mapping strings to numbers, built once and never changed after that.

What language is fastconstmap written in?

Mainly C. The stack also includes C, Python.

How hard is fastconstmap to set up?

Setup difficulty is rated easy, with roughly 5min to a first successful run.

Who is fastconstmap for?

Mainly developer.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.