git404hub

what is proveai-sdk fr?

prove-ai/proveai-sdk — explained in plain English

Analysis updated 2026-05-18

0PythonAudience · developerComplexity · 4/5Setup · moderate

tl;dr

A Python testing framework for multi-agent AI pipelines that pins a known good run as a snapshot, then flags when a later run drifts or regresses so you can catch it before merging.

vibe map

mindmap
  root((repo))
    What it does
      Snapshots agent traces
      Detects drift and regression
      Judges gray zone changes
    Tech stack
      Python
      httpx
      pydantic
      pytest plugin
    Use cases
      CI regression gate
      Prompt change testing
      Targeted re verify
    Audience
      Multi agent developers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Add a regression test suite for a multi-agent pipeline without maintaining a separate hand written eval set

VIBE 2

Catch a broken prompt change before it merges by wiring the CI verify step into GitHub Actions

VIBE 3

See exactly which agent in a chain broke and whether it was a direct edit or fallout from an earlier agent

VIBE 4

Run only the tests for agents whose code changed in a pull request instead of the entire pipeline

what's the stack?

Pythonhttpxpydanticpytest

how it stacks up fr

prove-ai/proveai-sdk0xallam/my-recipe0xhassaan/nn-from-scratch
Stars00
LanguagePythonPythonPython
Last pushed2022-11-22
MaintenanceDormant
Setup difficultymoderatemoderatemoderate
Complexity4/52/54/5
Audiencedevelopergeneraldeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · moderate time til it works · 30min

Needs API keys for whichever LLM provider your agents call, plus optional CI secret setup for automated checks.

The README does not state a license.

in plain english

ProveAI SDK is a Python testing framework for multi-agent AI pipelines, the kind of setup where several AI agents each do a small step, like classifying a support ticket, drafting a reply, then checking that reply for quality. The core idea is snapshot testing borrowed from web development: you run your pipeline once, let ProveAI record exactly what each agent received and produced, and pin that as a known good snapshot. From then on, running a verify command re-runs your pipeline and compares the fresh output against that snapshot, grading each agent as unchanged, drifted, or regressed, and failing the check if anything got worse. Getting started takes a few steps: install the package, add a monitor decorator to three to five of your agent functions, capture one real run wrapped in a Trace object, then run a command that pins the captured behavior as a snapshot. After that, whenever you edit a prompt or swap a model, running the verify command shows exactly what changed and whether it flowed downstream to other agents. The tool can also generate a ready made continuous integration workflow file, so this check runs automatically on every pull request, and in that automated setting it only re-checks the snapshots for agents whose underlying code the pull request actually touched, saving time on larger pipelines. When an agent's output changes in a way that is not simply pass or fail, ProveAI uses a separate AI model as a judge to decide whether the change is an equivalent rewording or an actual regression, and caches that judgment so repeated runs do not cost anything extra. There is also an older, still supported workflow for larger scale regression testing across many test cases at once, plus a pytest plugin for teams that prefer to run these checks inside their normal test suite. The project ships with built in support for OpenAI and Anthropic models, and needs extra configuration, such as an API key and endpoint URL, to work with any other provider. The full README is longer than what was shown.

prompts (copy fr)

prompt 1
Help me add the @monitor decorator to my three agent functions using proveai-sdk
prompt 2
Walk me through capturing my first trace and running proveai snapshot init
prompt 3
Set up proveai ci init for my repo using the Anthropic provider
prompt 4
Explain the difference between drifted and regressed in a proveai snapshot verify report

Frequently asked questions

what is proveai-sdk fr?

A Python testing framework for multi-agent AI pipelines that pins a known good run as a snapshot, then flags when a later run drifts or regresses so you can catch it before merging.

What language is proveai-sdk written in?

Mainly Python. The stack also includes Python, httpx, pydantic.

What license does proveai-sdk use?

The README does not state a license.

How hard is proveai-sdk to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is proveai-sdk for?

Mainly developer.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.