git404hub

what is furl fr?

mhdashikofficial/furl — explained in plain English

Analysis updated 2026-05-18

25PythonAudience · developerComplexity · 2/5LicenseSetup · easy

tl;dr

furl is a Python command line tool that gathers every historically known URL for a domain from the Wayback Machine, AlienVault OTX, Common Crawl, and URLScan.io.

vibe map

mindmap
  root((furl))
    What it does
      Fetches historical URLs
      Four data sources
      Async fetching
      Deduplicates results
    Tech stack
      Python
      aiohttp
      asyncio
    Use cases
      Bug bounty recon
      Filter by status code
      Export JSON
      Proxy through Tor
    Audience
      Security researchers
      Penetration testers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Find every historical URL ever associated with a domain for a bug bounty or security assessment.

VIBE 2

Filter discovered URLs by HTTP status code, file type, or date range to focus on relevant pages.

VIBE 3

Export discovered URLs as JSON Lines for feeding into other security tooling.

VIBE 4

Route lookups through an HTTP or SOCKS5 proxy such as Tor while gathering URLs.

what's the stack?

Pythonaiohttpasyncio

how it stacks up fr

mhdashikofficial/furlalexrosbach/replibookarlandaren/proagents
Stars252525
LanguagePythonPythonPython
Setup difficultyeasyeasyeasy
Complexity2/52/51/5
Audiencedeveloperops devopsvibe coder

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · easy time til it works · 5min

Just needs Python 3.8+, git, and a pip install, no external accounts or API keys required.

in plain english

furl is a command line tool written in Python that finds every publicly known web address, or URL, that has ever been associated with a given domain name. Instead of crawling a website itself, it asks four outside services that already keep historical records: the Wayback Machine, AlienVault OTX, Common Crawl, and URLScan.io. It pulls the results from all four at once, using async networking so it can wait on several requests in parallel, then removes duplicate URLs before showing them to you. The tool is built for people who do security work, such as bug bounty hunters and penetration testers, who often need a full picture of every page and endpoint a target domain has ever exposed, including old or forgotten ones that still show up in historical archives. It is a Python rewrite of an existing tool called gau, aiming to match its command options closely while adding installation through pip. Once installed, you run it by typing the tool's name followed by a domain, and it prints matching URLs to your screen or to a file. You can narrow the results by choosing which of the four sources to use, filtering by HTTP status code, filtering by file type such as images or fonts you want to skip, or limiting results to a date range. Output can be plain text or structured JSON, and the tool can also route its requests through an HTTP or SOCKS5 proxy. Settings you use often can be stored in a small configuration file so you do not have to type them every time. Setup requires Python 3.8 or newer and git, and the README gives separate install steps for Ubuntu, Fedora, Arch Linux, Windows, and macOS, generally involving creating a virtual environment and installing the package with pip.

prompts (copy fr)

prompt 1
Show me how to install furl on Ubuntu and run it against a single domain.
prompt 2
Explain how to filter furl's output to only include URLs that returned a 200 status code.
prompt 3
Walk me through configuring furl to skip image and font files while scanning a domain.
prompt 4
How do I set up a .gau.toml config file so furl always uses my preferred providers and settings?

Frequently asked questions

what is furl fr?

furl is a Python command line tool that gathers every historically known URL for a domain from the Wayback Machine, AlienVault OTX, Common Crawl, and URLScan.io.

What language is furl written in?

Mainly Python. The stack also includes Python, aiohttp, asyncio.

How hard is furl to set up?

Setup difficulty is rated easy, with roughly 5min to a first successful run.

Who is furl for?

Mainly developer.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.