tonyd2wild/2wild-coding-agent-latency-monitor — explained in plain English
Analysis updated 2026-05-18
Stress test a local model server with many simultaneous coding agent requests.
Compare tokens-per-second and response speed across different models or hardware.
Monitor real GPU usage, memory, and temperature during a benchmark run.
Generate a shareable summary of benchmark results to post online.
| tonyd2wild/2wild-coding-agent-latency-monitor | abduznik/portfolio-dumper | adcbueno/flashcards | |
|---|---|---|---|
| Stars | 19 | 19 | 19 |
| Language | HTML | HTML | HTML |
| Setup difficulty | moderate | easy | easy |
| Complexity | 3/5 | 2/5 | 1/5 |
| Audience | developer | developer | general |
Figures from each repo's GitHub metadata at analysis time.
Needs an OpenAI-compatible model server already running, plus optional passwordless SSH access for real GPU metrics.
Coding Agent Latency Monitor is a small dashboard for testing how fast an AI model server responds when many coding agent requests hit it at the same time. It works with any endpoint that speaks the OpenAI-compatible API, such as vLLM, SGLang, llama.cpp server, or TGI, and it needs nothing beyond Python itself to run. The tool fires a chosen number of parallel streaming requests, up to 32 at once, and shows live numbers for each one: time to first token, tokens per second, total end to end time, and token count. Because the requests are launched from the server side and multiplexed back through one connection, the tool avoids the browser's usual limit on how many connections it can hold open at once. There is also a real stop button and a kill all button that actually cut every open connection to the model server, so nothing keeps running in the background after you stop a test. Alongside the per-run numbers, a hardware strip shows metrics pulled from vLLM, and can optionally show real GPU usage, memory, and temperature over SSH if you set up a small config file listing your machines. When a run finishes or is stopped, the dashboard calculates peak and average throughput, per-stream highs and lows, total tokens, and average timing, then offers a one-click copy button so you can share those results elsewhere. Every finished run is also saved automatically to a local file, and a Past Runs tab lets you search and filter through your history with a chart for each run. Two extra modes exist for fun: a Matrix-style visual theme, and an experimental Art Mode where multiple agents each draw one slice of the same picture, which doubles as a rough way to compare how well different sized models can follow shared instructions. Setup involves copying an example config file, editing it to point at your model endpoint, and running a single Python script that starts a local web server. The project only talks to the endpoint you configure and does not send data anywhere else. It is licensed under MIT, so it can be freely used, modified, and shared.</content>
A zero-dependency live dashboard that fires many parallel streaming requests at an AI model server and shows real-time speed and throughput metrics for each run.
Mainly HTML. The stack also includes Python, HTML, SSH.
Licensed under MIT, so anyone can use, modify, and redistribute it freely, including for commercial purposes, as long as the copyright notice is kept.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
double-check against the repo, no cap.