git404hub

what is memoryarena fr?

zexuehe/memoryarena — explained in plain English

Analysis updated 2026-05-18

15PythonAudience · researcherComplexity · 4/5Setup · hard

tl;dr

Academic benchmark that tests how well AI agents remember information across separate task sessions. Compares memory approaches like keyword search, embeddings, and graph retrieval across shopping, travel, search, and reasoning tasks.

vibe map

mindmap
  root((repo))
    Agents
      Task input
      Action output
      Multi-provider LLMs
    Memory Systems
      Long context window
      BM25 keyword search
      Text embeddings
      GraphRAG
      Third-party services
    Environments
      Web shopping
      Travel planning
      Web search
      Formal reasoning
    Benchmark Design
      Cross-session memory
      Interdependent tasks
      Comparative eval
    Setup
      API keys needed
      Per-env instructions
      Preview release

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Evaluate how different memory systems help AI agents carry knowledge from one session to the next

VIBE 2

Compare retrieval approaches like keyword search, embeddings, and graph-based memory side by side

VIBE 3

Test AI agent performance on realistic tasks like shopping and travel planning that require memory

VIBE 4

Use as a starting point for building or improving memory systems for your own AI agent

what's the stack?

PythonOpenAI APIAnthropic APIGoogle AI APIOpenRouterBM25GraphRAGMem0

how it stacks up fr

zexuehe/memoryarena13127905/deep-learning-based-air-gesture-text-recognition-6xvl/paralives-plugins-index
Stars151515
LanguagePythonPythonPython
Setup difficultyhardmoderateeasy
Complexity4/53/52/5
Audienceresearcherdevelopergeneral

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1day+

Requires API keys for OpenAI, Anthropic, Google, and OpenRouter, plus separate keys for any third-party memory services. Each benchmark environment has its own setup instructions in separate markdown files.

No license is stated. All rights are reserved by default, you can read the code but cannot legally reuse or modify it without permission from the authors.

in plain english

MemoryArena is the code release for an academic research paper that benchmarks how well AI agents remember information across multiple separate task sessions. The core research question is: if an AI agent completes a task in one session, and a later task depends on what it learned or did earlier, how reliably does the agent carry that memory forward? The paper introduces a suite of tasks designed so that sessions are interdependent, making memory a critical factor in performance. The codebase is a Python framework with three main parts: agents (which take in a task and produce actions), environments (which execute those actions and return observations), and memory systems (which store and retrieve information between steps or sessions). The flow for each task step is: the memory system wraps the incoming task prompt with relevant stored context, the agent generates an action, the environment executes it, and the result is stored back into memory for future steps. Several memory approaches are included so they can be compared against each other. These range from simply giving the agent a long context window, to retrieval systems based on keyword search (BM25) or text embeddings, to graph-based retrieval (GraphRAG), to third-party memory services (Letta, Mirix, Mem0). The benchmark environments cover web shopping, travel planning, web search, and formal reasoning tasks. Running the code requires API keys for multiple AI providers (OpenAI, Anthropic, Google, OpenRouter) as well as separate keys for any third-party memory services used. Setup instructions for each environment are in separate markdown files in the repository. The README describes this as a preview version that is still being actively maintained. No license is stated in the README.

prompts (copy fr)

prompt 1
I'm looking at the MemoryArena benchmark. Can you explain how the memory system wraps a task prompt with stored context before sending it to the agent?
prompt 2
In MemoryArena, what is the difference between the BM25, embedding-based, and GraphRAG memory retrieval approaches, and when would each perform best?
prompt 3
How do I add a new benchmark environment to MemoryArena? Walk me through the steps based on how existing environments are structured.
prompt 4
I want to run MemoryArena with Mem0 as the memory backend. What API keys do I need and how do I configure them?
prompt 5
Explain the evaluation methodology in MemoryArena, how does it measure whether an agent successfully used memory from a previous session?

Frequently asked questions

what is memoryarena fr?

Academic benchmark that tests how well AI agents remember information across separate task sessions. Compares memory approaches like keyword search, embeddings, and graph retrieval across shopping, travel, search, and reasoning tasks.

What language is memoryarena written in?

Mainly Python. The stack also includes Python, OpenAI API, Anthropic API.

What license does memoryarena use?

No license is stated. All rights are reserved by default, you can read the code but cannot legally reuse or modify it without permission from the authors.

How hard is memoryarena to set up?

Setup difficulty is rated hard, with roughly 1day+ to a first successful run.

Who is memoryarena for?

Mainly researcher.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.