git404hub

what is web-scraping fr?

hadley/web-scraping — explained in plain English

Analysis updated 2026-07-30 · repo last pushed 2024-07-08

75RAudience · dataComplexity · 2/5DormantSetup · easy

tl;dr

A hands-on R tutorial that teaches you how to automatically collect data from websites using the rvest package, turning web pages into clean tables without manual copy-pasting.

vibe map

mindmap
  root((repo))
    What it does
      Scrapes web data with R
      Turns pages into tables
      Automates data collection
    Topics covered
      Web page structure
      Browser dev tools
      Scraping multiple pages
      Dynamic JavaScript sites
    Tools used
      rvest package
      Chrome browser
      Large language models
    Use cases
      Gather product prices
      Compile contact lists
      Collect news headlines
    Audience
      Data analysts
      Researchers
      R users

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Gather product prices from an online store automatically instead of copying them by hand.

VIBE 2

Compile a directory of contacts into a clean spreadsheet by scraping the listing pages.

VIBE 3

Collect headlines from a news site across multiple pages in one automated pass.

VIBE 4

Scrape data from a JavaScript-heavy website that loads content dynamically.

what's the stack?

RrvestChrome

how it stacks up fr

hadley/web-scrapingbrendangregg/perfmodelsjosephmisiti/ml_for_hackers
Stars757278
LanguageRRR
Last pushed2024-07-082014-12-052014-12-30
MaintenanceDormantDormantDormant
Setup difficultyeasymoderatemoderate
Complexity2/53/52/5
Audiencedataops devopsgeneral

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · easy time til it works · 5min

Requires installing the rvest R package and having a standard Chrome browser for the JavaScript-heavy site sections.

No license is specified in this repository, so default copyright terms apply and reuse may be restricted.

in plain english

This repository is a tutorial for learning how to scrape data from websites using R. It accompanies a workshop on the rvest package and teaches you how to automatically collect information from web pages and turn it into clean, organized tables. Instead of manually copying and pasting data from a site, you learn how to write code that does it for you. The lessons start with the basics of how a web page is structured and how to identify the specific pieces of a page you want to grab. It walks through using browser tools to pinpoint the data you need, then shows how to use rvest to extract that information and format it into a tidy data frame. Beyond the basics, it also covers how to scrape multiple pages at once, handle websites that load their content dynamically with JavaScript, and pull data from hidden or unofficial APIs that websites use behind the scenes. It even includes tips on using large language models to help with the process. This is ideal for data analysts, researchers, or anyone who uses R and needs to collect data that only exists on a website. For example, if you want to gather product prices from an online store, compile a list of contacts from a directory, or collect headlines from a news site, this tutorial shows you how to automate that work. It is designed for a hands-on audience, the materials include a slide deck and code you can run yourself. To follow along, you just need to install a couple of R packages and have a standard Chrome browser on your computer. The Chrome requirement specifically supports the sections on scraping dynamic, JavaScript-heavy sites.

prompts (copy fr)

prompt 1
Help me install rvest in R and write a simple script to scrape all the headings from a webpage using read_html and html_elements.
prompt 2
I have a website with multiple pages of product listings. Write R code using rvest to loop through each page and collect product names and prices into a data frame.
prompt 3
A website loads its data with JavaScript so rvest returns empty results. Walk me through using a browser tool or hidden API to get that data into R.
prompt 4
I want to scrape a contacts directory. Help me identify the right CSS selectors using my browser's dev tools and then extract names, emails, and phone numbers with rvest.

Frequently asked questions

what is web-scraping fr?

A hands-on R tutorial that teaches you how to automatically collect data from websites using the rvest package, turning web pages into clean tables without manual copy-pasting.

What language is web-scraping written in?

Mainly R. The stack also includes R, rvest, Chrome.

Is web-scraping actively maintained?

Dormant — no commits in 2+ years (last push 2024-07-08).

What license does web-scraping use?

No license is specified in this repository, so default copyright terms apply and reuse may be restricted.

How hard is web-scraping to set up?

Setup difficulty is rated easy, with roughly 5min to a first successful run.

Who is web-scraping for?

Mainly data.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.