irenee28/akm-clr-governance-layer — explained in plain English
Analysis updated 2026-05-18
Add tenant and task authorization in front of a shared vLLM or OpenAI-compatible LLM backend.
Block cross-tenant or unauthorized requests before they reach the model, with full audit trace logs.
Pilot a governance layer for a multi-tenant RAG or fine-tuned adapter platform before production hardening.
| irenee28/akm-clr-governance-layer | 0xustaz/streamgate | a-bissell/unleash-lite | |
|---|---|---|---|
| Stars | 1 | 1 | 1 |
| Language | Python | Python | Python |
| Setup difficulty | hard | hard | hard |
| Complexity | 4/5 | 4/5 | 4/5 |
| Audience | ops devops | developer | researcher |
Figures from each repo's GitHub metadata at analysis time.
Described as a research prototype needing a GPU-backed vLLM setup for real integration testing, not a drop-in production tool.
AKM-CLR is a governance and control layer that sits in front of shared LLM serving systems such as vLLM, deciding whether a request from a particular tenant or task should be allowed through, sent to a fallback, or blocked before it ever reaches the model. It is aimed at platforms that serve many different customers, tasks, retrieval augmented generation sources, or fine-tuned adapters from the same shared infrastructure, where a mistake could let one tenant see data or responses meant for another. The project provides tenant and task authorization, routing rules for domains and adapters, blocking of unsafe or cross-tenant requests before inference happens, safe fallback behavior, checks on model output, audit trace logs, and usage metrics, and it integrates with vLLM or any OpenAI-compatible backend. The author ran a series of controlled experiments to test the idea. These included a large simulated test with forty tenants, eighty adapters, and twenty domains, and a run of one thousand simulated attacks, both reporting zero measured leakage between tenants or adapters. A separate test placed the governance layer above a real vLLM server running the Qwen2.5 model, where authorized requests passed through while unsafe or unauthorized ones, including tenant spoofing and cross-tenant probing, were blocked before they reached the model. Later experiments tested packaging the project with Docker, Kubernetes manifests, a Helm chart skeleton, Prometheus metrics, and OpenTelemetry tracing, and a further test checked that a tenant is authorized to use a specific workspace, session, cache entry, or retrieved context object before a request goes to inference. The author describes this clearly as a validated prototype, not a production ready product, and is looking for a small number of design partners running multi-tenant LLM platforms to pilot it with. The README does not state a license.
A prototype governance layer that sits in front of shared LLM servers like vLLM, blocking unsafe or cross-tenant requests before they reach the model.
Mainly Python. The stack also includes Python, vLLM, Docker.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly ops devops.
This repo across BitVibe Labs
double-check against the repo, no cap.