Serve the Qwen3.5-122B language model efficiently on a Mac Studio.
Keep long, multi-turn AI conversations fast by restoring saved state from disk instead of reprocessing.
Run large hybrid language models on Apple Silicon without needing cloud GPUs.
| marzukia/qmlx | ademola-lou/threejsgnm | alicankiraz1/codexqb | |
|---|---|---|---|
| Stars | 28 | 28 | 28 |
| Language | Python | Python | Python |
| Setup difficulty | hard | moderate | easy |
| Complexity | 5/5 | 3/5 | 3/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires an Apple Silicon Mac with 96GB or more of unified memory and a large model download.
qMLX is a specialized serving engine for running a very large AI language model, Qwen3.5 122B, on Apple Silicon Macs like the Mac Studio. It is a fork of an existing project called Rapid-MLX, rebuilt specifically around one model and one problem: keeping long conversations fast to respond to. This particular model is a hybrid, meaning most of its internal layers work in a way that cannot simply be paused and resumed from memory the way normal attention layers can. Because of that, the usual trick of keeping recent conversation state in memory does not work here at all, so the project relies entirely on saving that state to the computer's disk and reading it back for the next message in a conversation. The README says this is not a backup plan, it is the only mechanism that makes the cache work for this model. The payoff is significant: a repeated 32,000 word prompt that would normally take 88 seconds to reprocess can be restored in under a second instead. The project includes disk-based saving and restoring of conversation state, a way to decide which saved state to delete first when disk space runs low, honest performance measurements that do not exaggerate speed, and logging that helps pinpoint exactly where a cached conversation stopped matching. The project is explicitly built only for Apple Silicon Macs with a lot of unified memory, only for this one AI model for now, and only for a single user at a time, not many people using it simultaneously. The project is labeled alpha stage, meaning it is early and specialized rather than broadly tested. One known limitation is that if a long response gets interrupted partway through, that work is thrown away and has to start over from scratch next time. A fix for that is planned but not yet built. Installation is done through Python's package tools, either from a package registry or by cloning the project's source code directly. Once installed, it exposes an interface compatible with the same style of API used by OpenAI and Anthropic. The README does not mention a license.
A specialized engine that keeps long AI conversations fast on Apple Silicon Macs by saving conversation state to disk.
Mainly Python. The stack also includes Python, MLX, Apple Silicon.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly developer.
This repo across BitVibe Labs
double-check against the repo, no cap.