gregordinary/rocket-userspace — explained in plain English
Analysis updated 2026-05-18
Speed up machine learning model inference on an RK3588 single-board computer using its NPU.
Run parts of transformer or Whisper-style models, like attention and normalization, directly on-chip.
Build a custom AI backend, such as a ggml or TFLite delegate, on top of this driver library.
Benchmark NPU matrix multiplication against CPU performance on Rockchip hardware.
| gregordinary/rocket-userspace | grieferpig/esp32-s31-linux | kritagya123611/ascent | |
|---|---|---|---|
| Stars | 8 | 8 | 8 |
| Language | C | C | C |
| Setup difficulty | hard | hard | hard |
| Complexity | 5/5 | 5/5 | 5/5 |
| Audience | developer | developer | ops devops |
Figures from each repo's GitHub metadata at analysis time.
Requires physical RK3588 hardware running a Linux kernel with the rocket driver, plus elevated permissions to access the NPU device file.
rocket-userspace is a low-level software library, called librocketnpu, that lets programs use the dedicated AI processing chip built into the Rockchip RK3588, a system-on-chip found in many single-board computers. That AI chip is called an NPU, short for neural processing unit, and it is designed to run the math behind machine learning models much faster and more efficiently than the main CPU can. This project builds on top of an existing official Linux driver called rocket, and adds the actual math operations, most importantly matrix multiplication, that let real machine learning workloads run on the NPU. The README states plainly that the project was built primarily by an AI coding assistant, Claude Code, with a human mostly setting goals and providing hardware to test on, and that it is a side project offered with no guarantees about quality or how often it will be updated. The library provides matrix multiplication in several numeric precisions, general convolution operations used heavily in neural networks, pooling and reduction operations, a wide range of activation functions like sigmoid and GELU, and higher level building blocks used in transformer style AI models, such as attention and normalization layers. Together these are enough to run pieces of real language models and speech models directly on the NPU. As one concrete example, the README describes running the initial processing phase of a language model called Gemma-4-12B on the NPU through this library, while the slower step-by-step text generation phase still runs on the CPU because it does not suit the NPU's strengths. Using it requires an RK3588 board running a Linux kernel that includes the rocket driver, along with some supporting system libraries, and the README notes you generally need elevated permissions to access the AI chip's device file. It is currently only validated on the RK3588 chip specifically, though the project is designed so support for two related Rockchip chips could be added later once their internal hardware details are confirmed through testing. The README includes detailed build instructions using CMake, a short C code example for running a single matrix multiplication, and extensive performance measurements comparing NPU speed against the CPU. No license file is mentioned in the README.
A userspace C library, mostly AI-written, that lets programs run neural network math like matrix multiplication on the AI chip built into Rockchip RK3588 boards.
Mainly C. The stack also includes C, CMake, NEON.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly developer.
This repo across BitVibe Labs
double-check against the repo, no cap.