git404hub

what is hadoop-on-gce fr?

theofpa/hadoop-on-gce — explained in plain English

Analysis updated 2026-07-18 · repo last pushed 2014-11-01

Audience · ops devopsComplexity · 4/5DormantSetup · hard

tl;dr

Scripts that automatically spin up a ready-to-use Hadoop big data cluster on Google Cloud with one command, then pause or tear it down when done.

vibe map

mindmap
  root((repo))
    What it does
      Creates Hadoop cluster
      Configures Cloudera CDH
      Pause and cleanup
    Tech stack
      Google Cloud
      Hadoop
      Cloudera CDH
    Use cases
      Temporary big data cluster
      Batch data processing
      Learning Hadoop ops
    Audience
      Data engineers
      Researchers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Spin up a 10-machine Hadoop cluster on Google Cloud in minutes for a data processing job.

VIBE 2

Pause a cluster to save cost while keeping disks intact, then restart it quickly later.

VIBE 3

Customize which services like Hive, HBase, or Spark are installed via the config file.

VIBE 4

Tear down all cloud resources with a single cleanup command once the job is finished.

what's the stack?

Google CloudHadoopCloudera CDHHiveHBaseSpark

how it stacks up fr

theofpa/hadoop-on-gce0verflowme/alarm-clock0verflowme/seclists
LanguageCSS
Last pushed2014-11-012022-10-032020-05-03
MaintenanceDormantDormantDormant
Setup difficultyhardeasyeasy
Complexity4/52/51/5
Audienceops devopsvibe coderops devops

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1h+

Requires a Google Cloud account, project is a work in progress and currently supports only identical machine types.

in plain english

This repository lets you quickly set up a big data cluster on Google Cloud in a hands-off way. Instead of manually creating virtual machines and installing software, you run a single command and the scripts handle everything, spinning up 10 machines, installing Hadoop and related tools, and configuring them to work together. The project wraps up Cloudera's Hadoop distribution (called CDH) so you can get a production-ready data processing cluster running on Google's cloud infrastructure. When you first run the build command, it creates virtual machines on Google Cloud, installs the necessary software on each one, and sets them up to communicate with each other. You can customize what services you want (like Hive, HBase, or Spark) during the setup process, and basic settings like machine size are stored in a configuration file you can edit beforehand. Once your cluster is live, you can pause it without losing your work or spending money on idle machines, the disks are saved so you can restart everything in seconds later. When you're truly done, a cleanup command removes all the machines and associated resources. The README notes this is still a work in progress, designed right now for clusters where all machines are identical, though the author plans to support mixed machine types in the future. This is most useful for data engineers or researchers who need a temporary cluster for processing large datasets or running Hadoop jobs without managing infrastructure manually. Instead of spending hours configuring servers, you get a functioning cluster in minutes. It's less hands-on than learning Hadoop from scratch, but still requires you to know broadly what you're doing with distributed data processing.

prompts (copy fr)

prompt 1
Walk me through setting up a Hadoop cluster on Google Cloud using this repo's build command.
prompt 2
Explain how to customize the config file to choose which Hadoop services get installed.
prompt 3
Show me how to pause and later restart this Hadoop cluster without losing data.
prompt 4
Help me clean up all the Google Cloud resources this project created when I'm done.

Frequently asked questions

what is hadoop-on-gce fr?

Scripts that automatically spin up a ready-to-use Hadoop big data cluster on Google Cloud with one command, then pause or tear it down when done.

Is hadoop-on-gce actively maintained?

Dormant — no commits in 2+ years (last push 2014-11-01).

How hard is hadoop-on-gce to set up?

Setup difficulty is rated hard, with roughly 1h+ to a first successful run.

Who is hadoop-on-gce for?

Mainly ops devops.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.