git404hub

what is inversebloomfilter fr?

tylertreat/inversebloomfilter — explained in plain English

Analysis updated 2026-07-18 · repo last pushed 2015-02-03

15GoAudience · developerComplexity · 2/5DormantSetup · easy

tl;dr

A small Go tool that helps programs skip duplicate data in fast-moving event streams. It never wrongly flags new data as a duplicate but may occasionally miss a repeat that appears much later.

vibe map

mindmap
  root((repo))
    What it does
      Skips duplicate events
      Never false positives
      May miss late duplicates
    Tech stack
      Go
      FNV-1a hashing
    Use cases
      Dedupe event streams
      Skip duplicate notifications
      Filter pipeline data
    Audience
      Developers
      Data engineers
    Design
      Small focused tool
      Based on Jeff Hodges concept

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Deduplicate events in a fast-moving data pipeline where duplicates appear close together in time.

VIBE 2

Skip duplicate notifications in a messaging system that occasionally sends the same message twice within seconds.

VIBE 3

Filter out repeated items in an event stream without storing every item you have ever seen.

VIBE 4

Build a lightweight duplicate detector for time-localized repeats in streaming data.

what's the stack?

Go

how it stacks up fr

tylertreat/inversebloomfilterbunnymq/bunnymqd4l3k/wikigopher
Stars151515
LanguageGoGoGo
Last pushed2015-02-032019-06-25
MaintenanceDormantDormant
Setup difficultyeasyhardmoderate
Complexity2/55/53/5
Audiencedeveloperops devopsdeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · easy time til it works · 5min

Small Go library with no external dependencies, just import the package and start filtering duplicates.

The repository does not include explicit license information, so usage rights are unclear.

in plain english

Inverse Bloom filter is a small tool written in Go that helps programs quickly figure out whether they've already seen a particular piece of data. It's designed for situations where you're processing a fast-moving stream of events and want to skip duplicates without keeping a massive list of everything that's come before. Traditional Bloom filters can tell you "maybe I've seen this" but sometimes guess wrong in the other direction, occasionally saying something is new when it's actually a duplicate. This project flips that trade-off: it will never tell you an item is a duplicate if it truly hasn't been seen before, but it might occasionally miss a duplicate and let it through. Think of it like a security guard who never wrongly accuses an innocent person of being a repeat visitor, but every so often fails to recognize someone they've actually met before. This makes it useful for deduplicating events in a data pipeline, especially when duplicates tend to appear close together in time rather than scattered far apart. For example, if a messaging system accidentally sends the same notification twice within a few seconds, this filter would likely catch the second one and skip it. But if the same event reappeared hours later, the filter might have already forgotten about it and process it again. The implementation is based on an original concept by Jeff Hodges, with one notable change: it uses a faster, simpler hashing method called FNV-1a instead of the heavier MD5 algorithm. Since this tool isn't doing security-related work, the lighter hash makes sense for performance. The project is quite small and focused, doing one specific thing rather than trying to be a general-purpose deduplication library.

prompts (copy fr)

prompt 1
Write a Go service that processes events from a Kafka topic and uses an Inverse Bloom filter to skip duplicate messages that arrive within seconds of each other.
prompt 2
Create an event-stream deduplication pipeline in Go using the inversebloomfilter package, and explain when it might miss a duplicate that appears hours later.
prompt 3
Implement a notification sender in Go that uses an Inverse Bloom filter to avoid sending the same notification twice when retries happen close together.
prompt 4
Compare a traditional Bloom filter and an Inverse Bloom filter for a streaming deduplication use case, and write Go code showing how each handles false positives and false negatives.

Frequently asked questions

what is inversebloomfilter fr?

A small Go tool that helps programs skip duplicate data in fast-moving event streams. It never wrongly flags new data as a duplicate but may occasionally miss a repeat that appears much later.

What language is inversebloomfilter written in?

Mainly Go. The stack also includes Go.

Is inversebloomfilter actively maintained?

Dormant — no commits in 2+ years (last push 2015-02-03).

What license does inversebloomfilter use?

The repository does not include explicit license information, so usage rights are unclear.

How hard is inversebloomfilter to set up?

Setup difficulty is rated easy, with roughly 5min to a first successful run.

Who is inversebloomfilter for?

Mainly developer.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.