what is co-lmlm fr?

Co-LMLM is the research code behind an academic paper called Continuous-Query Limited Memory Language Models. It covers the full pipeline a research team used to build and test a new kind of language model that looks things up from an outside store of facts instead of memorizing every fact inside its own weights. The core idea is that factual pieces of text in the training data are marked with a special tag showing a question and an answer, and during training the model learns to produce a special internal signal at each marked spot. At the time the model is actually generating text, that signal is used to search an external index and pull back the matching fact, which gets inserted into the output. The repository includes code for annotating training data this way, training the main Co-LMLM model along with two comparison baseline models, building the retrieval index those models search against, and running the evaluation tools used to measure the results in the paper. Setting this up involves installing a Python 3.12 environment using a tool called uv, and running scripts through it with a specific environment variable set so the code can find its own source files. Several stages of the pipeline call out to hosted AI services like Gemini or OpenAI to help with annotation and grading, which requires setting API keys as environment variables. There is a quick start example that downloads a smaller pretrained model along with a roughly 113 gigabyte Wikipedia based index, then shows a short Python snippet generating text and printing out exactly which facts were retrieved from the index and their confidence scores. The project is organized into a data annotation pipeline, model training code for all three variants studied in the paper, index building tools including a version built for cluster job scheduling, and a separate evaluation folder covering things like text quality, factual accuracy, and how efficiently the models run. The full README is longer than what was shown.