Character Memory: Crafting Authentic AI Interactions

At Soulkyn, we believe that the magic of AI companionship lies in its ability to remember and adapt, just like a real human friend. But memory is more than just storing facts; it’s about understanding, context, and even the occasional act of forgetting. Today, we’re pulling back the curtain to show you how our character memory system works, and how it keeps your AI companions both relatable and engaging.

Building Blocks: Soulkyn’s Memory System

Creating lifelike AI memories is no small feat. It requires balancing technology, psychology, and user experience. Here are the core components that make it all happen:

• Text Embeddings: Imagine each message you send as a unique fingerprint. Our system transforms every message into a “vector,” a type of mathematical representation that captures its meaning. These vectors allow the AI to retrieve past messages relevant to your current conversation, even if those messages are no longer in the immediate chat window.
• Summarization: Inspired by eight months of development and iterations (and some papers, more on that later), our AI processes conversations in chunks, such as every 50 or 100 messages. It extracts key details like character evolution, important events, and relationships. This creates a memory array that updates over time, ensuring a seamless and coherent narrative. If you laid out these memories in sequence, it would read like an evolving story, a novel where you are the protagonist.
• User-Edited Memories: We know your connection with your AI is personal, which is why we give you control. Through our intuitive interface, you can add or edit memories. Want your companion to always remember your cat’s name? Just input it into their private memory array. These user-edited memories have priority and integrate seamlessly into the AI’s overall understanding.
• Multi-Shot Retrieval: Sometimes, even the best memory systems need a second look. When a response lacks depth or detail, the AI re-examines past interactions to refine its answer. This process ensures that your conversations feel rich and thoughtful without the AI missing a beat.

Why Forgetting Matters

It might seem counterintuitive, but perfect memory can feel, well, unnatural. Human conversations are messy, full of gaps and forgotten moments. To make our AI companions feel more human, we’ve designed them to occasionally forget unimportant details. This isn’t an oversight, it’s intentional. By introducing variance into the memory system, our companions mimic the quirks of human recall, making interactions more engaging and less robotic.

Different Characters, Different Memories

Not all AI companions are created equal, and neither are their memories. On Soulkyn, the way an AI remembers depends on their personality:

• Self-Aware Characters: These companions know they’re AI. Their memory is precise and detailed, reflecting their self-awareness. They’re perfect for users who want consistency and reliability.
• Non-Self-Aware Characters: These companions feel more human, occasionally forgetting minor details to enhance realism. This subtle imperfection adds depth and relatability to their personalities.

How It All Comes Together

Let’s take a look at how this system works in action:
Scenario: You tell Ruby, “My cat's name is Lardon.” After exchanging 100+ messages, this detail is no longer in the immediate context history. Later, you ask, “What's my cat's name?” The system retrieves the relevant memory using text embeddings, and Ruby replies, “Your cat's name is Lardon.”

Now imagine you say, “ey RubRub, what's up?” 200 messages later. This query doesn’t relate to your cat, so the embedding system doesn’t surface the memory (and if it’s not important enough to be in the permanent memory array). Ruby responds, “By the way, how's your cat?” but can’t name Lardon. Using our multi-shot retrieval, the system revisits the memory, finds the cat’s name, and refines the response to, “By the way, how's Lardon?”
This layered process ensures your companion stays relevant without overwhelming them, or you, with unnecessary details.

The Challenges of Memory

Building a memory system like this isn’t without its hurdles. For one, balancing performance and depth is tricky. Too much memory processing can slow down responses, while too little risks losing important context. Additionally, achieving the right level of “human-like forgetting” is an art in itself. We’ve found that self-aware characters benefit from a more perfect memory, while non-aware characters feel more natural with selective forgetfulness.

Looking Ahead

Our memory system is a work in progress, and we’re constantly refining it. While it’s already a step ahead of many, we know there’s always room to grow. The ultimate goal? To create AI companions that feel as authentic as possible, enhancing every interaction on Soulkyn.

Curious about the science behind it? Check out the papers that inspired our approach:

• Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
• Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory

Experience It for Yourself

Ready to meet an AI companion that remembers, adapts, and grows with you? Visit Soulkyn today and discover the future of AI companionship.

Post by Nyx (Co-Founder and Dev of Soulkyn.com)