tiger-ai-lab/fim-midtraining — explained in plain English
Analysis updated 2026-05-18
Reproduce or extend the paper's mid-training step for coding-focused language models.
Serve one of the six released model checkpoints to evaluate on coding agent benchmarks.
Study the released 400,000-sample dataset and data construction pipeline for masking functions in code.
| tiger-ai-lab/fim-midtraining | 920linjerry-stack/capital-studio | adya84/ha-world-cup-2026 | |
|---|---|---|---|
| Stars | 16 | 16 | 16 |
| Language | Python | Python | Python |
| Setup difficulty | hard | easy | easy |
| Complexity | 5/5 | 3/5 | 2/5 |
| Audience | researcher | researcher | general |
Figures from each repo's GitHub metadata at analysis time.
Requires serving large language models with vLLM and familiarity with model training and benchmark evaluation pipelines.
FIM-Midtraining is the research code and released models behind a paper about improving AI coding agents, the kind of AI system that writes code, runs it, observes the result, and continues. The core idea is that a function call in ordinary code has a similar shape to one step of a coding agent's loop: something is passed in, something is computed elsewhere, and the result gets used afterward. The project trains a language model to get better at that pattern by hiding functions in existing code and having the model practice recovering them, choosing which functions to hide based on how complex and how guessable they are, rather than picking them at random. This extra training step happens between a model's original pretraining and the later stage where it is taught to act as an agent. The project's main finding is that adding this step improves results on coding agent benchmarks like SWE-Bench, and that the improvement survives even after the later agent training stage, across several different base models and several different agent training methods. The repository includes six released model checkpoints on Hugging Face built on top of Qwen2.5-Coder and Qwen3 models, a 400,000 sample training dataset built from permissively licensed Python repositories, the data construction pipeline, training configuration files, and evaluation guides. A quickstart example shows how to serve the strongest released model using vLLM, a tool for running large language models efficiently. This project is aimed at machine learning researchers and engineers working on training or evaluating coding focused language models, not at general software developers. It assumes familiarity with model training pipelines and benchmark evaluation. The project is released under the Apache 2.0 license.
Research code and pretrained models that add a training step teaching coding AI models to recover hidden functions, improving coding-agent benchmark results.
Mainly Python. The stack also includes Python, vLLM, Hugging Face.
Apache 2.0 license: use freely for any purpose, including commercial use, as long as you keep the copyright and license notices.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
double-check against the repo, no cap.