What is LLM memory?
LLM memory is any way an application gives a language model information beyond the prompt in front of it, because the model itself keeps nothing between calls. It covers the context window, saved conversation history, stored memories, retrieval and fine-tuning. Each is filled from what someone said or wrote, so none of them changes when the user does something in your product. Memory of what they are doing right now is behavioral memory.
Does an LLM have memory?
Not between calls. A large language model is a fixed set of weights that turns a prompt into a response. When the response is finished, nothing about the exchange is written back into the model. Ask a follow-up and it knows the first question only because your application sent the whole conversation again.
Anthropic’s documentation on context windows spells this out: the input for each turn “contains all previous conversation history plus the current user message.” The same page calls the context window the model’s “working memory”, as distinct from the data it was trained on.
So when people say an LLM remembers, they mean that something around the model stored information and put it back in the prompt. That machinery is LLM memory. Every kind except fine-tuning ends the same way, as text placed in the context window before the model is called. Fine-tuning changes the weights.
What do people mean by LLM memory?
Usually one of five things, and they often get mixed up in the same conversation.
- The context window. Everything in the current request: the system prompt, the messages, tool results and documents. It is the only thing the model reads directly. It has a size limit, and Anthropic notes that recall degrades as it fills, which it calls context rot.
- Conversation history. The messages of one thread, saved by your application so the conversation can resume. LangGraph calls this short-term memory, scoped to a single thread and persisted with a checkpointer “so the thread can be resumed at any time.” That is wider than the working memory in short-term vs long-term agent memory, which ends with the task. LangGraph keeps the thread after the task is done.
- Saved memories. Facts kept across conversations and brought into new ones. ChatGPT’s memory keeps saved memories, details you ask it to remember or that it saves as useful, and can also reference past chats. You can turn memory off in settings, and a Temporary Chat neither uses nor creates memories. For developers, Anthropic’s memory tool lets Claude create, read and update files that persist between sessions, stored wherever your application puts them. LangGraph’s long-term memory is the same idea: a store scoped by namespace that any thread can read.
- Retrieval. Documents split into chunks, indexed, often in a vector store, and searched at question time so the closest matches go into the prompt. OpenAI’s guide to optimizing LLM accuracy calls this in-context memory, “stuffing relevant information into the context window”.
- Fine-tuning. Further training on examples, so the model carries a behavior or a body of knowledge in its weights. The same OpenAI guide calls this learned memory and points to it when a model must perform a task “consistently and reliably.”
The MemGPT paper, the research behind Letta, ties several of these together. It treats the context window like an operating system’s main memory and moves information between it and external storage as the conversation needs it.
How do the kinds of LLM memory compare?
The table puts the five side by side, with behavioral memory as a sixth row for contrast.
| Kind | What it stores | Where it lives | How long it lasts | Good for | What it misses |
|---|---|---|---|---|---|
| Context window | Everything sent in this request | The request itself | One call | Reasoning over what is in front of the model | Anything not sent, and detail that blurs as the window fills |
| Conversation history | The messages of one thread | Your database, or a checkpointer | Until the thread is deleted, after the task has ended | Picking a conversation back up | Other threads, unless you carry them over |
| Saved memories | Facts and preferences from past chats | A store or files your application keeps; OpenAI’s systems for ChatGPT | Until edited or deleted | Personal details that should carry across sessions | Anything the user never said |
| Retrieval | Chunks of your documents | A vector index or search engine | Until the documents are reindexed | Answering from your docs, catalog and policies | Facts about this user that are in no document |
| Fine-tuning | Patterns learned from training examples | The model’s weights | Until the next training run | Tone, format and repeated tasks | Anything after the training data, and anything about one user |
| Behavioral memory | What the user did: views, searches, errors, time on a step | A store computed from your product’s events | One session or longer, depending on the key | What the user is doing right now | Why they did it, until they say |
The first five hold text that someone wrote. A user typed it, a writer published it, or someone labeled it for training. Behavioral memory is computed from events your product records, so it includes what a user would not think to mention.
For the tools that implement saved memories and recall, such as Mem0, Zep and Letta, see what is agent memory. Short-term vs long-term agent memory covers how an agent combines the in-session and cross-session kinds.
What gap does LLM memory leave?
Each of the five is filled from language. That makes them good at what the user has said and what your company has written down, and slow to learn what the user is doing. A shopper who has opened the size guide three times does not type “I’m worried about fit.” A customer stuck on a failing payment step may open chat and type only “help.”
The gap is timing. Conversation history and saved memories change when the user writes something. Retrieval changes when someone reindexes. Fine-tuning changes when you train again. What the user did in the last few minutes reaches none of them before the model’s next call, unless something computes it from the events and puts it in the prompt.
That is the job of behavioral memory. What is context engineering treats it as the live state in an agent’s context, and the rest of the AI agents and memory topic covers why it has to be current.
Example: a shopping assistant asked about fit
A shopper on an outdoor clothing store has a rain jacket in their basket. In a chat last spring they mentioned they usually wear a medium. Today, before opening the assistant, they open the jacket’s size guide three times and read the returns policy once. Then they type “does this run small?”
Here is what each kind of memory gives the assistant.
- Context window. The question and the system prompt. It knows “this” means the jacket only if your code put the product in the prompt.
- Conversation history. Today’s thread, which holds one message.
- Saved memories. “Usually wears a medium”, from last spring.
- Retrieval. The jacket’s size chart and the returns policy text.
- Fine-tuning. The store’s tone of voice, if you trained for it.
- Behavioral memory. Three size guide views and one returns page view in this session.
With the first five, the assistant can answer the question it was asked and suggest a medium. With behavioral memory as well, it can tell that this shopper is unsure about fit and has already checked what happens if the jacket does not fit. So it can answer the question and also explain how exchanges work, which speaks to the worry the shopper never typed.
In Snowplow Signals, the behavioral part is an attribute, a value computed from events about one key, here the session. These definitions count views of the size guide and the returns page in the current visit, assuming each has its own URL:
from snowplow_signals import (
Attribute, AtomicProperty, Criteria, Criterion, Event, Signals, StreamAttributeGroup,
domain_sessionid,
)
page_view = Event(name="page_view", vendor="com.snowplowanalytics.snowplow", version="1-0-0")
size_guide_views_session = Attribute(
name="size_guide_views_session",
type="int32",
events=[page_view],
aggregation="counter",
criteria=Criteria(all=[
Criterion.like(AtomicProperty(name="page_url"), "%/size-guide%"),
]),
default_value=0,
)
returns_views_session = Attribute(
name="returns_views_session",
type="int32",
events=[page_view],
aggregation="counter",
criteria=Criteria(all=[
Criterion.like(AtomicProperty(name="page_url"), "%/returns%"),
]),
default_value=0,
)
shop_assistant_fit = StreamAttributeGroup(
name="shop_assistant_fit",
version=1,
attribute_key=domain_sessionid,
owner="you@example.com",
attributes=[size_guide_views_session, returns_views_session],
)
sp = Signals(api_url=api_url, api_key=api_key, api_key_id=api_key_id, org_id=org_id)
sp.publish([shop_assistant_fit])
When the chat opens, your backend reads the group for this session:
fit = sp.get_group_attributes(
name="shop_assistant_fit",
version=1,
attributes=["size_guide_views_session", "returns_views_session"],
attribute_key="domain_sessionid",
identifier=session_id,
)
It gets values such as size_guide_views_session: 3 and returns_views_session: 1. The saved
preference and the size chart come from wherever you keep them now. Your code puts the counts, the saved
preference and the size chart into one prompt. What the assistant says with them is up to the
model.
Where does Signals fit?
Behavioral memory has to be built from events: collected as the user browses, computed into values about a session or a user, and readable at the moment the model is called. You can build that on your own event stream, or use a service for it. Snowplow Signals is one option. Its SDKs collect behavioral events, it computes attributes like the two above, and it serves them over the Python and Node.js SDKs, a REST API or an MCP server. It can also hand an agent the session’s recent activity as JSON or a narrative block, through agentic context. Read times are on the real-time attributes page.
It runs beside whichever LLM memory you use now. Chat history, saved memories and retrieval keep what the user said and what you wrote. Signals supplies what the user did, and your code decides what goes in the prompt. An agentic context covers one session; for what a customer did in earlier visits, your code reads attributes keyed on the user in a separate call. Signals runs as managed SaaS or in your own AWS or GCP account.
See it on your own traffic
Add the SDK, define one attribute, and see an attribute updating against your own traffic in the first session. The free tier runs on the real engine: no card, no sales call.