fix(upkeep): Batch deadletter futures into FuturesUnordered

[evanh]

To avoid sequentially sending each deadletter future, put all the futures into a batch using FuturesUnordered to have the entire batch run concurrently.

[john-z-yang]

LGTM, should we refactor this into a helper function?

[john-z-yang]

Also, I noticed that this is using join_all instead of FuturesUnordered like the other PR, is this intentional?

[evanh]

@john-z-yang I changed this to use a helper function, however:

The two loops use different types: retries uses InflightActivation and deadletter uses TaskActivation. I tried to write a generic function to handle both, using closures passed in as arguments. However closures + async move do not work well together, and the only thread I could find about it didn't end up with an answer.

So, instead I changed the retry functionality to deal with TaskActivations and then I could write one function to handle both cases.

Let me know what you think. If this seems too complicated I'll revert it.

[john-z-yang]

@john-z-yang I changed this to use a helper function, however:

The two loops use different types: retries uses InflightActivation and deadletter uses TaskActivation. I tried to write a generic function to handle both, using closures passed in as arguments. However closures + async move do not work well together, and the only thread I could find about it didn't end up with an answer.

So, instead I changed the retry functionality to deal with TaskActivations and then I could write one function to handle both cases.

Let me know what you think. If this seems too complicated I'll revert it.

@evanh Ah I see, I didn't realize that. I think the earlier version seems simpler then. However, either looks fine to me so I'll let you decide!