git404hub

what is gpt-neox fr?

blinkdl/gpt-neox — explained in plain English

Analysis updated 2026-07-18 · repo last pushed 2022-02-25

PythonAudience · researcherComplexity · 5/5DormantSetup · hard

tl;dr

A research toolkit for training billion-parameter GPT-style language models across many GPUs, built on NVIDIA Megatron and Microsoft DeepSpeed.

vibe map

mindmap
  root((gpt-neox))
    What it does
      Trains large LLMs
      Splits work across GPUs
      Configured via YAML
    Tech stack
      Python
      Megatron
      DeepSpeed
    Use cases
      Train custom LLM
      Fine-tune models
      Evaluate benchmarks
    Audience
      Researchers
      GPU cluster owners

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Train a large language model from scratch across multiple GPUs using YAML configs.

VIBE 2

Fine-tune an existing GPT-NeoX model on custom data.

VIBE 3

Evaluate a trained model's performance on standard benchmarks.

VIBE 4

Download and run inference on the pretrained GPT-NeoX-20B weights.

what's the stack?

PythonMegatronDeepSpeedPyTorch

how it stacks up fr

blinkdl/gpt-neox0xallam/my-recipe0xhassaan/nn-from-scratch
Stars——0
LanguagePythonPythonPython
Last pushed2022-02-252022-11-22—
MaintenanceDormantDormant—
Setup difficultyhardmoderatemoderate
Complexity5/52/54/5
Audienceresearchergeneraldeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1day+

Requires multiple high-end GPUs with substantial VRAM and deep familiarity with distributed training concepts.

prompts (copy fr)

prompt 1
Walk me through configuring a YAML file in gpt-neox to train a smaller model on my own GPU cluster.
prompt 2
Explain how gpt-neox splits both the model and data across multiple GPUs during training.
prompt 3
Show me how to download and run inference with the pretrained GPT-NeoX-20B weights using deepy.py.
prompt 4
Help me convert and tokenize my own text dataset into the format gpt-neox expects for training.

Frequently asked questions

what is gpt-neox fr?

A research toolkit for training billion-parameter GPT-style language models across many GPUs, built on NVIDIA Megatron and Microsoft DeepSpeed.

What language is gpt-neox written in?

Mainly Python. The stack also includes Python, Megatron, DeepSpeed.

Is gpt-neox actively maintained?

Dormant — no commits in 2+ years (last push 2022-02-25).

How hard is gpt-neox to set up?

Setup difficulty is rated hard, with roughly 1day+ to a first successful run.

Who is gpt-neox for?

Mainly researcher.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.

similar repos, if this ain't it