LangGraph Memory Service

This repo provides a simple example of a long-term memory service you can build and deploy using LangGraph.

This graph extracts memories from chat interactions and persists them to its store. This information can later be read via the API to provide personalized context when your bot is responding to a particular user.

The memory graph handles debouncing when processing individual conversations (to help deduplicate work) and supports continuous updates to a single "memory schema" as well as "event-based" memories that can be fetched by recency and filtered.

This repo also provides an example chat bot (in this case, also a simple graph) that connects to the memory graph via the SDK. Any time you send a message to the chat bot, it will query the memory service to fetch the most up-to-date memories (if any) for the configured user. These memories are put in the system prompt. After responding, it will post the conversation to the memory service to schedule long-term memory formation.

This separation of concerns provides minimal overhead, allows deduplication of memory processing, and ensures you can optimize for better recall.

Getting Started

This quickstart will get your memory service deployed on LangGraph Cloud. Once created, you can interact with it from any API.

Assuming you have already installed LangGraph Studio, to set up:

1. Create a .env file.

cp .env.example .env

2. Define required API keys in your .env file.

Setup Model

The defaults values for model are shown below:

model: anthropic/claude-3-5-sonnet-20240620

Follow the instructions below to get set up, or pick one of the additional options.

Anthropic

To use Anthropic's chat models:

1. Sign up for an Anthropic API key if you haven't already.
2. Once you have your API key, add it to your .env file:

ANTHROPIC_API_KEY=your-api-key

OpenAI

To use OpenAI's chat models:

1. Sign up for an OpenAI API key.
2. Once you have your API key, add it to your .env file:

OPENAI_API_KEY=your-api-key

3. Open in LangGraph studio. Navigate to the "chatbot" graph and have a conversation with it! Try sending some messages saying your name and other things the bot should remember.

Wait ~10-20 seconds and then create a new thread using the + icon. Then chat with the bot again - if you've completed your setup correctly, the bot should now have access to the memories you've saved!

How it works

This chat bot reads from your memory graph's Store to easily list extracted memories.

Connecting to this type of memory service typically follows an interaction pattern similar to the one outlined below:

The service waits for a pre-determined interval before it considers the thread "complete". If the user queries a second time within that interval, the memory run is cancelled to avoid duplicate processing of a thread.

How to evaluate

Memory management can be challenging to get right. To make sure your memory_types suit your applications' needs, we recommend starting from an evaluation set, adding to it over time as you find and address common errors in your service.

We have provided a few example evaluation cases in the test file here. As you can see, the metrics themselves don't have to be terribly complicated, especially not at the outset.

We use LangSmith's @unit decorator to sync all the evaluations to LangSmith so you can better optimize your system and identify the root cause of any issues that may arise.

How to customize

Customize memory memory_types: This memory graph supports two different update_modes that dictate how memories will be managed:

1. Patch Schema: This allows updating a single, continuous memory schema with new information from the conversation. You can customize the schema for this type by defining the JSON schema when initializing the memory schema. For example:

{
    "name": "User",
    "description": "Update this document to maintain up-to-date information about the user in the conversation.",
    "update_mode": "patch",
    "parameters": {
      "type": "object",
      "properties": {
        "user_name": {
          "type": "string",
          "description": "The user's preferred name"
        },
        "age": {
          "type": "integer",
          "description": "The user's age"
        },
        "interests": {
          "type": "array",
          "items": {
            "type": "string"
          },
          "description": "A list of the user's interests"
        }
      }
    }
  }

2. Insertion Schema: This allows inserting individual "event" memories, such as key pieces of information or summaries from the conversation. You can define custom memory_types for these event memories by providing a JSON schema when initializing the InsertionMemorySchema. For example:

{
    "name": "Note",
    "description": "Save notable memories the user has shared with you for later recall.",
    "update_mode": "insert",
    "parameters": {
      "type": "object",
      "properties": {
        "context": {
          "type": "string",
          "description": "The situation or circumstance in which the memory occurred that inform when it would be useful to recall this."
        },
        "content": {
          "type": "string",
          "description": "The specific information, preference, or event being remembered."
        }
      },
      "required": ["context", "content"]
    }
  }

3. Select a different model: We default to anthropic/claude-3-5-sonnet-20240620. You can select a compatible chat model using provider/model-name via configuration. Example: openai/gpt-4.
4. Customize the prompts: We provide default prompts in the graph definition. You can easily update these via configuration.

For quick prototyping, these configurations can be set in the LangGraph Studio UI.

You can also quickly extend this template by:

• Adding additional nodes and edges in graph.py to modify the memory processing flow.