If a job exceeds its custom retry limit will it be sent to the DLQ?

[mpvosseller]

If a job has a custom retry limit (less than the default) and its limit is exceeded will it be delivered to the DLQ?

Tracing through the code here it appears to me that in this case change_message_visibility returns false and delete_message is called. That I assume will prevent it from being sent to the DLQ.

I was looking at this code because I wondered how you handle sending a job to a DLQ when a job's retry limit is less than the queue's MaxReceiveCount.

[metaskills]

That's a great question! Also, I think your assessment is right. Here is my hot take and I'd love your thoughts.

• Should custom job retries (vs queue) retries be DLQ'able? I'm on the fence about this. But my initial thought is no... and if the anwser is yes, should that be another Queue made in CloudFormation tied to a specific job?
• If we did want them to be in the DLQ without the distinct SQS queue, how can we do that? Is it even possible?
• Would instrumenting the retry_stopped event be helpful here?
• Are there Sidekiq behaviors that we want to mimic?

Once we come up with some ideas and paths. Maybe we can do them and document them. Thoughts?

[mpvosseller]

Thanks for the response Ken. I was really just trying to understand how your library works (and what is even possible with SQS) as I built my Node.js equivalent library https://github.com/mpvosseller/serverless-job

Coming at this fresh I had hoped (and expected) that all failed jobs would be treated identically and each would end up in the DLQ. I had a few wild ideas about how to make it work but they all had drawbacks.

For now my code is just ignoring the concept of DLQs all together. If my library finds a job has reached its try limit then it will delete it (and prevent it from going to the DLQ).

If on the other hand SQS finds a message that has reached the queue's maxReceiveCount or retentionPeriod then SQS will send it to the DLQ.

🤷

Appreciate your library. I learned a lot from your code!

[fabiensebban]

None of my errored messages are stored in the DLQ and I haven't defined a custom retry limit. Is it possible that Lambdakiq does not return the correct error message to SQS so it can put the errored message in the DLQ?
Execution of errored messages is not retried.