Autoscaling Passport under load

[Jkd-eth]

User Story:

As a Passport PM or Dev
I want the system to be able auto scale under load
So that I can handle large partners driving large passports

Acceptance Criteria

GIVEN a partner implements Passport to protect their community
WHEN a large flood a new users comes in
THEN the system can automatically scale to support those users

Product & Design Links:

Tech Details:

Open Questions:

Notes/Assumptions:

[kammerdiener]

@erichfi getting focused on this one and will have some updates by tomorrow.

[kammerdiener]

@Jeremy-Gitcoin // @erichfi

In looking at everything the current scaling parameters are fighting each other. What I would really recommend doing is swapping to just scaling based on CPU and Memory. Drop the ALB scaling metric.

This way we can set CPU and Mem to start scaling up at about 50 percent and set it to start scaling in around 30. This way when it spikes, gains some more space, and recalculates it won't immediately start scaling back in.

[erichfi]

Hi @kammerdiener, thanks for the recommendation. Let's proceed with scaling based on CPU and Memory, dropping the ALB scaling metric. Setting the thresholds to scale up at 50% and scale in around 30% sounds good. Please make the necessary adjustments and keep us posted. If there are any other considerations, let us know.

[kammerdiener]

@erichfi -- I will work on the adjustments to the Pulumi code and get it setup for review.

[kammerdiener]

To provide an update prior to Monday. I am finalizing the changes and should have a PR open Tuesday morning.

[kammerdiener]

PR is avaialble here: passportxyz/passport-scorer#355

I was mistaken on being able to set the scale in so I have slowed it down in an attempt to reduce overall thrashing.