Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Lambda Serverless context reduce Cold Start #78

Closed
hghandri opened this issue Nov 13, 2024 · 0 comments
Closed

Feature request: Lambda Serverless context reduce Cold Start #78

hghandri opened this issue Nov 13, 2024 · 0 comments
Labels

Comments

@hghandri
Copy link
Contributor

Use case

Hi,

I'm experiencing performance issues with cold starts after deploying the project to AWS Lambda using the Python 3.12 runtime on ARM architecture. After investigating, it appears the issue lies in importing modules, which cannot be easily minimized.

I understand that this project needs to be versatile, but it would be beneficial if we could load only the modules required for each specific context.

For example, I only use Bedrock and don’t need to load the entire Anthropic module due to the presence of the Anthropic Classifier. For reference, the Anthropic module takes around 0.7 seconds to load, which significantly impacts the initialization phase during cold starts.

Have you experienced the same thing?

Solution/User Experience

Perhaps into "init.py" on classifiers folder test if module "anthropic" is installed or not and then load or not Anthropic Classifier.

Alternative solutions

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant