Monitor whether your production ML model's predictions are drifting as new real-world data comes in over time.
Evaluate whether your LLM chatbot responses are too short, too negative, or refusing requests they should answer.
Generate an HTML data quality report before deploying a new model version to catch problems early.
Set up a live monitoring dashboard to track AI system health metrics across real user traffic over time.
| evidentlyai/evidently | coells/100days | h2oai/h2o-3 | |
|---|---|---|---|
| Stars | 7,490 | 7,489 | 7,478 |
| Language | Jupyter Notebook | Jupyter Notebook | Jupyter Notebook |
| Setup difficulty | easy | easy | hard |
| Complexity | 3/5 | 1/5 | 4/5 |
| Audience | data | developer | data |
Figures from each repo's GitHub metadata at analysis time.
Evidently is an open-source Python library for evaluating and monitoring AI systems. If you have built a machine learning model or a chatbot powered by a large language model, Evidently gives you tools to check whether your system is behaving correctly, both during development and once it is running in production. The library works in two main ways. First, you can run Reports and Test Suites: you feed in your data and outputs, and Evidently calculates metrics like whether your data quality has changed, whether a model's predictions are drifting over time, or whether an AI assistant is giving responses that are too short, too negative, or likely denying requests it should answer. Over 100 built-in checks cover classification accuracy, recommendation diversity, text sentiment, and many more. You can view results interactively in a Jupyter notebook or export them as HTML or JSON for sharing. Second, Evidently includes a monitoring dashboard that tracks these metrics over time as your system handles real users. You can run the dashboard yourself on your own machine or sign up for a hosted version called Evidently Cloud, which adds alerting, team management, and other extras on top of a free tier. The library installs with a single pip command and is designed to be modular. You can start with a one-line preset that catches common problems in tabular data, then layer in custom metrics as you learn more about what your specific system needs to track. Evidently is developed by Evidently AI, the company behind it, and is open source under a public license. It is aimed at data scientists and ML engineers who want to catch problems in AI systems before or after they reach users, without having to build monitoring infrastructure from scratch.
Open-source Python library for evaluating and monitoring AI systems, check data drift, model accuracy, and LLM output quality with 100+ built-in checks, viewable in Jupyter or as exportable HTML reports.
Mainly Jupyter Notebook. The stack also includes Python, Jupyter Notebook, pip.
Open source, free to use and modify, including for commercial purposes.
Setup difficulty is rated easy, with roughly 30min to a first successful run.
Mainly data.
This repo across BitVibe Labs
double-check against the repo, no cap.