[Question] How to debug the activity task startToClose timeout error?

[ArFeRR]

We're receiving the activity startToClose timeout for our activity, written on PHP with the retry policy below:

Timeout 3m
Retry Policy Initial Interval 1s
Retry Policy Backoff Coefficient 3
Retry Policy Maximum Interval 10s
Retry Policy Maximum Attempts 30

The task, which timeouts, does nothing but going to database for selecting some data by indexed field and it actually does it, but after executing the activity code it hangs for no reason until 3 minutes timeout appears. Then retry policy comes to an action and retry task starts execution, it ends with timeout too and so on, and so on. Once it all the task attempts end with timeout is throws an activity exception permanently.

Could you please look through the JSON log from temporal workflow admin for such activity and advice what to do with it?
(We've already checked the deadline exceeded debugging cookbook https://docs.temporal.io/troubleshooting/deadline-exceeded-error ,but everything is alright from our side)

TaskFailed.json
WorkflowFailed.json

[wolfy-j]

1. Check activity status in Temporal UI, was it started?
2. Check logs on worker that runs it, is there a chance that worker is dead?
3. Are you using Temporal cloud or local?
4. Are you using latest versions of RR and SDK?

it feels like your activity worker is crashing, can you try to use mock activity or something that def works first?