git404hub

what is caveman-steer fr?

rkique/caveman-steer — explained in plain English

Analysis updated 2026-05-18

2PythonAudience · researcherComplexity · 4/5Setup · hard

tl;dr

A research project testing whether nudging a coding model's internal activations can shrink its answers further than prompting alone, without hurting accuracy.

vibe map

mindmap
  root((caveman-steer))
    What it does
      Shortens AI code explanations
      Combines prompt and steering
      Measures token savings
    Method
      Diff-in-means steering vector
      Four test conditions
      Qwen2.5-Coder-7B-Instruct
    Findings
      Prompt plus steer shortest
      Steering alone weak
      Accuracy stays similar
    Tech stack
      Python
      AST parsing
      gpt-4o-mini scoring

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Study how activation steering combines with prompt instructions to shorten model output.

VIBE 2

Reproduce the four-condition benchmark comparing base, prompt, steer, and prompt plus steer.

VIBE 3

Learn a working example of building a diff-in-means steering vector.

VIBE 4

Reference the token savings data when deciding whether to try steering for concise AI agent output.

what's the stack?

PythonPyTorchAST

how it stacks up fr

rkique/caveman-steer0-bingwu-0/live-interpreter010zx00x1/faresnipe
Stars222
LanguagePythonPythonPython
Setup difficultyhardmoderateeasy
Complexity4/52/52/5
Audienceresearchergeneralgeneral

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1h+

Requires running a 7B parameter language model locally plus GPU access to compute and apply the steering vector.

Not stated in the README, no license information is given.

in plain english

This project is a research experiment about making AI coding assistants give shorter answers without losing accuracy. It builds on a separate popular tool called caveman, which tells an AI model, through its prompt instructions, to drop small words like the and a, and to answer in short fragments rather than full sentences, cutting the length of its answers by roughly two thirds while keeping the technical content correct. The author wanted to know whether a technique called activation steering could shrink answers even further. Activation steering works by nudging the internal numerical state of the model while it is generating text, rather than only changing the text of the instructions given to it. The steering direction used here was built by comparing the model's internal activity when it produces short, caveman style answers against its activity when producing normal, longer answers, and then using the difference between the two as a push in the desired direction. The experiments compare four setups: the model with no special instructions at all, the model given only the caveman style prompt, the model given only the steering nudge with no special prompt, and the model given both the prompt and the steering nudge together. The tests use a language model called Qwen2.5-Coder-7B-Instruct and ask it to explain what a piece of code does in plain language, using a standard benchmark dataset built for that exact task. The results show that using the caveman prompt by itself already shortens answers a great deal, and adding steering on top of the prompt shortens them further still, by about another 13 percent, while a separate check found that the combined approach used far fewer of the discouraged small words like the and a than the prompt alone. Steering by itself, without the prompt, made little difference. Correctness stayed roughly the same, around 90 to 95 percent, across all four setups, based on scoring done by a separate AI model. The repository is written in Python and includes the code used to prepare the data, run the experiments, and generate the charts and example outputs shown in the results.

prompts (copy fr)

prompt 1
Explain how this project builds its diff-in-means steering vector from concise versus verbose examples.
prompt 2
Walk me through how the caveman prompt and activation steering are combined at inference time.
prompt 3
Show me how the four conditions, base, prompt, steer, and prompt plus steer, are evaluated for correctness.
prompt 4
Help me understand why steering alone had little effect while combining it with the prompt worked better.

Frequently asked questions

what is caveman-steer fr?

A research project testing whether nudging a coding model's internal activations can shrink its answers further than prompting alone, without hurting accuracy.

What language is caveman-steer written in?

Mainly Python. The stack also includes Python, PyTorch, AST.

What license does caveman-steer use?

Not stated in the README, no license information is given.

How hard is caveman-steer to set up?

Setup difficulty is rated hard, with roughly 1h+ to a first successful run.

Who is caveman-steer for?

Mainly researcher.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.