what is rabbit fr?

Rabbit is a tool, written in the Rust programming language, that lets a single ordinary computer run a huge AI language model that would normally need much more memory than most machines have. The model in question, called GLM-5.2, has 744 billion individual parameters, the internal values a language model uses to generate text, but Rabbit makes this possible on a machine with around 25 gigabytes of RAM by being clever about what it keeps in memory versus what it reads from disk as needed. The trick is that this kind of AI model, called a mixture of experts model, does not use all of its parameters for every single word it generates. Only a small portion actually gets used at any one time, and which portion changes based on what is being processed. Rabbit keeps the parts that are always needed sitting in RAM, shrunk down using a compression technique, while the tens of thousands of specialized sub sections of the model, called experts, stay stored on the hard drive and only get loaded the moment they are actually needed. A memory system remembers which experts get used often and keeps those ready faster over time. The project includes a command line tool for generating single responses to a prompt, a chat mode that keeps conversation history and can resume it later, and a web server mode that behaves like a common AI service so other applications can talk to it in a familiar way. The author has published detailed, honest performance numbers measured on real hardware, including how much time is spent reading from disk versus doing calculations, and how often the system's memory cache successfully avoids a slow disk read. This project is a from scratch reimplementation of a similar tool written in C called colibri, adapted into Rust with some additional performance improvements. It is explicitly described as a research project rather than a polished, ready to use product, and several features, including GPU support, are not built yet. The license for the code has not yet been decided.