Run AI agent missions that automatically produce traces and evidence of what happened
Score an agent's output against held out data before deciding whether to promote it
Redact secrets from agent run artifacts before they are stored or shared
Compare multiple AI models or approaches using the router lab and navigator tools
| playbasis/agent-os | adun-denton/chinvat | agbara286/saint-cms | |
|---|---|---|---|
| Stars | 2 | 2 | 2 |
| Language | TypeScript | TypeScript | TypeScript |
| Setup difficulty | moderate | moderate | easy |
| Complexity | 4/5 | 4/5 | 2/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires Node 20+ and pnpm 10+, fixture mode needs no network, database, or LLM access.
Playbasis Agent OS is a framework for running AI agents in a way that always produces proof of what they actually did. Every mission the agent runs generates traces, artifacts, evaluation scores, a decision about whether the result should be promoted, and a report showing that any secrets were removed. The idea is that when an agent claims to have improved something, that claim has to be backed by inspectable evidence, and if it cannot be backed, that gets exposed instead of hidden. This project was built as a proof of value lane for a company called Playbasis, an API first engagement platform, but the core pieces, including how evidence and traces are formatted, are described as usable outside of that platform too. The authors are careful to state clearly what is proven and what is not. They say the full loop of running missions, iteratively improving results, scoring against held out data, deciding whether to promote a result, redacting secrets, storing run history, and generating dashboards all works and has been tested against deterministic and local integration setups. They are explicit that independently verified benchmark wins, measured business value, human reviewed output quality, and fully unsupervised production use are not yet proven and remain on the roadmap. The codebase is organized into packages: a kernel that runs missions and produces traces, an evals package that scores runs and makes promotion decisions, a hill climber for iterative improvement, a mission optimizer for generating and selecting candidates, a navigator for planning and voting between different approaches, a router lab for testing which AI model to route requests to, and a run warehouse that indexes all the runs and evidence produced. There are also folders for example missions, goals, schemas, and committed proof reports. To try it, you need Node version 20 or higher and pnpm version 10 or higher. Running install, typecheck, test, and a scan command completes a verification gate that does not require any network access, database, or calls to an AI model, since it uses a deterministic fixture mode. The project deliberately excludes private code, credentials, and the actual contents of its evaluation answer keys from the public repository, publishing only hashes and commitments instead. It is released under the Apache 2.0 license.
A framework for running AI agents that forces every claimed improvement to come with inspectable proof, such as traces, eval scores, and redaction reports.
Mainly TypeScript. The stack also includes TypeScript, Node.js, pnpm.
Use, modify, and distribute this code freely, including commercially, as long as you keep the license notice and state any changes you made.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
double-check against the repo, no cap.