klabianco/oracle-duel — explained in plain English
Analysis updated 2026-05-18
Study how well Claude and GPT calibrate their confidence when forecasting real world prediction markets.
Run the pipeline in mock mode to test the forecasting and scoring logic without spending anything.
Review the pre registered go or no go criteria before deciding whether to allow live trading.
| klabianco/oracle-duel | 0xallam/my-recipe | 0xhassaan/nn-from-scratch | |
|---|---|---|---|
| Stars | 0 | — | 0 |
| Language | Python | Python | Python |
| Last pushed | — | 2022-11-22 | — |
| Maintenance | — | Dormant | — |
| Setup difficulty | moderate | moderate | moderate |
| Complexity | 4/5 | 2/5 | 4/5 |
| Audience | researcher | general | developer |
Figures from each repo's GitHub metadata at analysis time.
Needs API keys for Claude, GPT, and Kalshi, defaults to safe paper trading mode.
Oracle Duel is a research project that pits two AI language models, Claude and GPT, against each other as forecasters on real prediction markets run by Kalshi. Each model keeps its own five hundred word strategy prompt that it rewrites after every scored round, based on how accurate its past forecasts were using a standard scoring method for probability predictions. Most of what the models produce is just forecasts used purely to measure and improve their judgment, and only the two or three strongest predicted edges each round turn into small real money trades, and even those are sized by a separate, fixed risk control system that the forecasting agents cannot modify or influence. The project is explicit that it is a research experiment about how well language models can forecast and calibrate their confidence, not a betting product or financial advice, and it defaults to a paper trading mode with no real orders placed. Before it can be switched into live trading, a pre registered set of go or no go conditions has to be met, and those conditions were written down and locked before any results were seen, specifically so the project cannot quietly move the goalposts if the early results look good. The fixed risk engine limits position sizing using a conservative fraction of the total bankroll, caps the number of new positions and the total daily loss, and includes a drawdown circuit breaker that halts trading and requires a human to review the situation before it resumes. There is also a simple kill switch file that immediately blocks all new orders while leaving the forecasting itself running. The system can run in a fully mocked mode with no real spending for testing, a paper mode against real markets with no money at risk, and a live mode. It is written in Python, scheduled to run automatically on a daily cycle using macOS's launchd or Linux cron, and released under the MIT license.
A research project where Claude and GPT compete as automated forecasters on real Kalshi prediction markets, scored for accuracy with a locked risk control system.
Mainly Python. The stack also includes Python.
Use freely for any purpose, including commercial use, as long as you keep the copyright notice (MIT license).
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
double-check against the repo, no cap.