sabrydawood/shahd-llm-stack — explained in plain English
Analysis updated 2026-05-18
Study how a transformer, tokenizer, and training loop work by reading an implementation built from scratch
Train a tiny chat model on your own CPU using the built in data pipeline and training scripts
Experiment with agent tool calling and visible reasoning traces on a small local model
| sabrydawood/shahd-llm-stack | atomicstrata/atomicmemory | azerdsq131/mcpm | |
|---|---|---|---|
| Stars | 6 | 6 | 6 |
| Language | TypeScript | TypeScript | TypeScript |
| Setup difficulty | moderate | moderate | easy |
| Complexity | 5/5 | 4/5 | 2/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires the Bun runtime plus Go and C toolchains for native compute kernels.
Shahd is a from scratch language model stack built in TypeScript, using the Bun runtime along with native Go and C compute kernels. Rather than starting from someone else's model weights or fine tuning an existing model, the project builds every layer itself: a data pipeline, a tokenizer, a training loop, supervised fine tuning, and agent serving with a visible reasoning trace that shows how the model thinks before it answers. The codebase is organized into layers under a Brain folder, covering configuration, tensors and automatic differentiation, core math operations, the transformer itself, an optimizer, a tokenizer that mixes character and byte level encoding, a data collection pipeline with license checks and deduplication, a training loop, sampling with a key value cache, checkpointing, a safety filter, supervised fine tuning support, evaluation tools, a serving layer with tool calling and an agent loop, and reasoning techniques such as speculative decoding and self consistency. As of the current release, the project has trained four small checkpoints entirely on CPU, ranging from about 1 million to about 6.5 million parameters. The flagship model, called MicroChat, can greet a user, call a built in calculator tool with correct arguments, answer basic identity questions, hold short multi turn conversations, and display its reasoning steps live in a dashboard. The author is upfront that these are tiny, CPU trained models that perform well only within what they were trained on, not general purpose chat models, and that real capability would require much more scale. To try it, you install dependencies with Bun and can run commands to train a small model, sample from a saved checkpoint, or launch a local dashboard that walks through collecting data, training, and chatting with the result. The project enforces its own coding conventions through continuous integration, including naming rules and a file length limit. The license is proprietary: all rights are reserved by the author, so this is not an open source project you can freely reuse.
A from scratch large language model stack built in TypeScript, covering data collection, tokenization, training, and agent serving with visible reasoning.
Mainly TypeScript. The stack also includes TypeScript, Bun, Go.
Proprietary: all rights reserved by the author, not open source.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
double-check against the repo, no cap.