BingeBuddy is a movie and TV recommendation agent that remembers you between sessions. It was built for the Conversational Agents course at TU Delft, together with a five-person team, and it came with an experiment attached: we wanted to know whether the kind of memory an agent keeps changes how much users trust it.

The question

A recommender that forgets everything after each turn is annoying. The obvious fix is to store what the user said. The less obvious question is what to store.

Two options, borrowed from the psychology of human memory:

• Episodic memory: keep every statement as a separate timestamped event. “On 3 March the user said they like sci-fi.” Nothing is ever merged or overwritten. The store grows without bound and can contain contradictions.
• Semantic memory: keep one consolidated fact per attribute. New information about a user’s taste in sci-fi is merged into the existing sci-fi entry, and the old version is replaced.

Semantic memory is the harder thing to build, because merging requires judgement: is this new statement a refinement, a contradiction, or a separate fact entirely? The hypothesis was that the extra work pays off in user trust, because the agent stops repeating itself and stops holding stale beliefs about you.

So the system was built twice. Same front-end, same conversational model, same prompts for the recommendation itself. The only difference is what happens to the memory after each user message.

System

A user message enters through a Flask front-end, either typed or spoken. Speech is transcribed with Whisper (small.en), and a DistilBERT emotion classifier tags the text before it reaches the agent.

The message then goes two places at once. The conversational agent answers it, using the current memory store and the recent message log as context. Separately, the message is published to a MessageLog, which notifies the memory workflow. That decoupling matters: the user gets a reply without waiting for the memory pipeline, which involves several LLM calls and is slow.

The memory workflow itself is a state graph. We started with LangGraph and ended up writing our own CustomStateGraph, because we needed conditional edges that could loop a node back on itself for repair, and we wanted the state to be a typed object rather than a dictionary.

The memory pipeline

Six agents, each a separate LLM call with its own prompt:

1. Sentinel. A cheap gate. Given the user message, answer TRUE or FALSE: does this contain anything worth remembering? Most conversational turns (“sounds good”, “what else?”) do not, and stopping here saves five downstream calls.
2. Extractor. Pull out every distinct fact, as a list of strings. One message often carries several: a liked genre, a disliked actor, and a streaming platform in the same sentence.
3. Extractor reviewer. Read the extractions against the original message. If something was invented or garbled, set needs_repair and write a repair message explaining what is wrong. The graph routes back to the extractor, which now sees both its previous output and the criticism.
4. Attributor. Assign each fact to one of fourteen attributes: LIKES, DISLIKES, FAVORITE, WANTS_TO_WATCH, PLATFORM, GENRE, PERSONALITY, WATCHING_HABIT, FREQUENCY, AVOID, CHARACTER_PREFERENCES, SHOW_LENGTH, REWATCHER, POPULARITY. A fact with no attribute is dropped.
5. Aggregator. The semantic-only step. For each attribute, merge the new fact into whatever is already stored under that attribute, producing one replacement string.
6. Aggregator reviewer. Same repair loop, applied to the merge. This is where the pipeline catches the aggregator quietly discarding information that should have been kept.

The final memories go to MongoDB, keyed by user and attribute, with upsert.

The episodic variant stops after the attributor. Nothing is merged, and each stored memory carries the timestamp of the message that produced it, so the same attribute accumulates a list rather than a single value.

The reviewer loops are the part I would keep in any future version. A single LLM call asked to extract structured facts from free text will confidently produce facts that are not there. A second call, shown both the source and the extraction and asked only to find the discrepancy, catches a lot of it. It costs one extra call per stage and it is the difference between a memory store you can use and one you have to keep apologising for.

Experiment

Thirty participants, fifteen per group, between-subjects. Each participant had a free conversation with BingeBuddy and then filled in a questionnaire. The experimental group talked to the semantic-memory build, the control group to the episodic one. Neither was told which.

Trust was measured with the twelve-item scale from Jian, Bisantz and Drury (2000), each item on a seven-point Likert scale. Five of the twelve are phrased negatively (“The system is deceptive”, “I am wary of the system”) and were reverse-scored, so that 7 always means maximum trust. Each participant’s score is the mean over the twelve items.

Before comparing the groups, we checked they were comparable. Gender distribution was not significantly different (chi-squared test for homogeneity, \(\chi^2 = 0.15\), \(p = 0.70\)). Neither was age (Kruskal-Wallis, \(p = 0.55\)) nor self-reported familiarity with LLMs (Kruskal-Wallis, \(p = 0.60\)).

Both groups’ trust scores passed a Shapiro-Wilk normality test (control \(W = 0.949\), \(p = 0.51\); experimental \(W = 0.974\), \(p = 0.91\)), and Bartlett’s test found no difference in variance (\(p = 0.40\)), so a Student’s t-test was appropriate.

Result

\(t = -0.894\), \(p = 0.38\). We fail to reject the null hypothesis. Semantic memory did not produce a measurable increase in user trust over episodic memory.

The direction is what the hypothesis predicted, since the semantic group scored slightly higher, and the episodic group’s scores are visibly more dispersed. But with fifteen per group and an effect this small, that is not evidence of anything.

Some honest limitations:

• Underpowered. Thirty participants can only detect a large effect. If the true difference is the half-point it looks like here, this study was never going to find it.
• Sessions were too short. Semantic memory should pay off across many conversations, when contradictions accumulate and the episodic store gets long enough to confuse the agent. A single sitting is close to the best case for episodic memory, since there is nothing yet to contradict.
• Convenience sample. Mostly fellow students, most of whom rated themselves highly familiar with LLMs. Their trust calibration is not the general public’s.
• Trust may be the wrong measure. Users might not notice which memory scheme is running, while still getting better recommendations from one of them. Recommendation quality was never measured directly.

The engineering claim survives the null result: the semantic pipeline works, merges correctly, and runs at conversational speed. What does not survive is the claim that users can tell.

Stack

Python with Poetry, LangChain for prompt composition over both OpenAI and local Ollama models (mistral:7b, deepseek-r1:8b), a hand-rolled state graph for the agent workflow, MongoDB for the memory store, Whisper for transcription, Hugging Face Transformers for emotion classification, Flask for the front-end, and Docker Compose to bring up the database. Analysis in pandas, seaborn, and scipy.stats.