git404hub

what is dream-cycle-lab fr?

wtznc/dream-cycle-lab — explained in plain English

Analysis updated 2026-05-18

0PythonAudience · researcherComplexity · 5/5Setup · hard

tl;dr

A research test bench for teaching language models new facts by updating their weights through LoRA adapters, while measuring how well they avoid forgetting old facts.

vibe map

mindmap
  root((Dream Cycle Lab))
    What it does
      Tests memory consolidation
      Trains LoRA adapters
      Measures forgetting
    Tech stack
      Python
      MLX
      LoRA
    Use cases
      Compare selection policies
      Test forgetting defenses
      Evaluate closed book recall
    Audience
      AI researchers
      ML engineers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Compare different strategies for selecting which facts a model should learn permanently.

VIBE 2

Measure how much a model forgets old facts after learning new ones.

VIBE 3

Test whether techniques like EWC or EMA adapters reduce catastrophic forgetting.

what's the stack?

PythonMLXLoRA

how it stacks up fr

wtznc/dream-cycle-lab0xhassaan/nn-from-scratch3ks/embedoc
Stars00
LanguagePythonPythonPython
Last pushed2023-06-08
MaintenanceDormant
Setup difficultyhardmoderatehard
Complexity5/54/51/5
Audienceresearcherdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1h+

Requires MLX and Apple hardware for in-process LoRA training and multi-cycle sweeps.

in plain english

Dream Cycle Lab is a research test bench for a specific problem in AI: how to make a language model actually learn new facts by updating its own internal weights, instead of relying on retrieval, meaning looking facts up at query time as most systems do today. The project calls this the write side problem, since the AI field has good solutions for reading from external memory but almost no tooling for writing new knowledge into model weights without breaking what the model already knows. The core metaphor is a dream cycle, based loosely on how biological memory gets consolidated during sleep. Experiences, which may be noisy or even contradictory facts, accumulate in a fast store called a replay buffer. Periodically, a sleep cycle evaluates those facts and selects a subset to train into a LoRA adapter, a small set of extra weights layered on top of the base model that updates only a tiny fraction of its parameters rather than retraining the whole thing. The lab tests five different policies for deciding which facts get selected for consolidation, ranging from random sampling to picking whichever facts currently surprise the model most, and it tests four defense strategies meant to prevent catastrophic forgetting, the tendency for new training to overwrite knowledge learned earlier. Those defenses include a penalty that protects weights considered important for past knowledge, keeping a running average of adapter weights across training cycles, and mixing old facts back into new training batches. Evaluation is closed book: at test time the model has no access to retrieval, so any correct answer has to come from what is actually stored in its weights. The README also documents a series of experiments and results, including a finding that an earlier evaluation method was overstating how well the model learned facts, because it was only checking whether the model repeated back the exact training sentence rather than genuinely answering a rephrased question. The project is written in Python and uses MLX for in-process training on Apple hardware. It is aimed at researchers interested in continual learning and memory consolidation for language models rather than at general users.

prompts (copy fr)

prompt 1
Explain how the dream cycle consolidation process works in this project.
prompt 2
Walk me through the difference between the five selection policies in this repo.
prompt 3
Help me understand how Elastic Weight Consolidation prevents catastrophic forgetting here.
prompt 4
Show me how to run a new sweep with a different LoRA rank in this codebase.

Frequently asked questions

what is dream-cycle-lab fr?

A research test bench for teaching language models new facts by updating their weights through LoRA adapters, while measuring how well they avoid forgetting old facts.

What language is dream-cycle-lab written in?

Mainly Python. The stack also includes Python, MLX, LoRA.

How hard is dream-cycle-lab to set up?

Setup difficulty is rated hard, with roughly 1h+ to a first successful run.

Who is dream-cycle-lab for?

Mainly researcher.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.