victorsteinbock/llama-cag-n8n-reworked — explained in plain English
Analysis updated 2026-05-18
Build a support chatbot that answers strictly from a real product manual using a local model.
Let a coding agent like Claude Code query a large spec document without spending its context window.
Set up a private team reference desk for contracts or compliance documents that can't leave your network.
Ask repeated questions about a dense document without reprocessing it every time.
| victorsteinbock/llama-cag-n8n-reworked | 0xallam/my-recipe | 0xhassaan/nn-from-scratch | |
|---|---|---|---|
| Stars | 0 | — | 0 |
| Language | Python | Python | Python |
| Last pushed | — | 2022-11-22 | — |
| Maintenance | — | Dormant | — |
| Setup difficulty | hard | moderate | moderate |
| Complexity | 4/5 | 2/5 | 4/5 |
| Audience | developer | general | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires Docker plus a one-time warm-up read of each document (minutes on CPU), scanned PDFs need an extra text-extraction pass.
llama-cag-n8n-reworked lets you ask questions about a document using a fully local AI model, without that model having to reread the whole document every time you ask something new. The idea it's built around is called Cache-Augmented Generation, or CAG. Normally, an AI model either has to process your entire document again for every question, which is slow, or it only sees small retrieved snippets from the document through a technique called RAG, which can miss connections between different parts of the text. This project takes a different path. You hand it a document once, and the model reads the whole thing a single time on your own computer. As it reads, it saves its internal understanding of the text, called the KV cache, to disk. From then on, every question you ask is answered by restoring that saved understanding rather than reading the document again, so answers come back in seconds instead of minutes, at no ongoing cost, and without any of the document leaving your machine. Everything runs inside Docker containers, so there's nothing to compile by hand on your own computer. Under the hood, llama.cpp's llama-server handles running the AI model and keeping the cache, a small FastAPI service coordinates the pieces, n8n handles automation and workflow logic, and PostgreSQL stores metadata. It's designed to work on Windows, macOS, or Linux. The README describes several situations where this approach works especially well: a support chatbot that needs to answer strictly from a real product manual instead of guessing, a way to let AI coding assistants like Claude Code ask questions about a large specification document without that document eating up their limited context window every session, and a private reference desk for a team's contracts, runbooks, or compliance documents that can't leave the building. It's also honest about where it doesn't fit. It's not meant for searching across thousands of documents at once, for documents too large to fit in the model's memory, for multi-user login systems, or for documents you'll only ever ask about once. The document needs to be readable as text, so scanned or image-based PDFs need an extra preparation step first. The local server also runs unauthenticated, relying on being reachable only from your own machine rather than the open internet. The project is released under the MIT license. The full README is longer than what was shown.
A self-hosted Docker stack that lets a local AI model read a document once and answer unlimited later questions from a saved cache, without rereading it.
Mainly Python. The stack also includes Python, llama.cpp, Docker.
Use freely for any purpose, including commercial use, as long as you keep the copyright notice.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly developer.
This repo across BitVibe Labs
double-check against the repo, no cap.