Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature]: Introducing async add method aadd (proposal) #1923

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

spike-spiegel-21
Copy link
Collaborator

Motivation:

https://discord.com/channels/1121119078191480945/1279997338944929915/1280570524589625375

Docs:

The async add method aadd can be used for non-blocking of main thread for I/O bound tasks. This help users to reduce latency significantly.

m = Memory.from_config(config)
async def _main():
    await m.aadd("Likes to play cricket on weekends", user_id="alice", metadata={"category": "hobbies"})
 
asyncio.run(_main())

Demo: Sync

import time
start = time.time()
m.add("I love experimenting with new recipes, especially when cooking fusion dishes.", user_id="alex")
m.add("I often spend hours browsing tech blogs and forums to stay updated on the latest programming trends.", user_id="alex")
m.add("I enjoy solving coding challenges online, especially when I find clever solutions to tough problems.", user_id="alex")
print(f"Time taken: {time.time() - start}")

Time taken: 29.755067825317383
Extracted memories:

['Loves experimenting with new recipes', 'Likes finding clever solutions to tough problems', 'Often spends hours browsing tech blogs and forums to stay updated on the latest programming trends', 'Enjoys cooking fusion dishes', 'Enjoys solving coding challenges online']

Demo: Async

import asyncio
async def _main():
    await asyncio.gather(
        # await m.add()
        m.aadd("I love experimenting with new recipes, especially when cooking fusion dishes.", user_id="harry"),
        m.aadd("I often spend hours browsing tech blogs and forums to stay updated on the latest programming trends.", user_id="harry"),
        m.aadd("I enjoy solving coding challenges online, especially when I find clever solutions to tough problems.", user_id="harry")
    )
start = time.time()
asyncio.run(_main())
print(f"Time taken: {time.time() - start}")

Time taken: 11.030951976776123
Extracted memories:

['Enjoys cooking fusion dishes', 'Often spends hours browsing tech blogs and forums to stay updated on the latest programming trends', 'Enjoys solving coding challenges online', 'Loves experimenting with new recipes', 'Likes finding clever solutions to tough problems']

@spike-spiegel-21
Copy link
Collaborator Author

Hi @deshraj, Did only for openai client. Going ahead will be implementing for more I/O bound tasks and wide support.

  1. vector db operations
  2. graph db operations

@spike-spiegel-21 spike-spiegel-21 changed the title [async]: Introducing async add method aadd (proposal) [feature]: Introducing async add method aadd (proposal) Sep 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant