feat(engine, engine-rest-core) Add a query criteria to retrieve all executing jobs

[prajwolbhandari1]

related to: #4470

[prajwolbhandari1]

@venetrius any update on this?

[venetrius]

Hi @prajwolbhandari1,

The proposed logic does not exactly query jobs that are currently executing.

select * from act_ru_job where LOCK_EXP_TIME_ is not null and suspension_state_=1 and LOCK_EXP_TIME_ > currenttime

It is possible that LOCK_EXP_TIME_ > currenttime but a job is not executing:
LOCK_EXP_TIME_ is first set when the job executor has acquired the job. However an acquired job might not get executed right away.
If a job failed LOCK_EXP_TIME_ might be updated with an icrement as part of the retry policy

It is also not guaranteed that no job is running if LOCK_EXP_TIME_ <= currenttime. In a case where a job does not return within the LOCK_EXP_TIME_ the job can be acquired by a (different of same) job executor.
I recommend visiting our documentation on the The Job Executor

If the behaviour you want to achieve is still met with this query then one option would be to split executing into two filters.
A query criteria to test if suspension is 1 already exists it is called active
A query criteria that checks if a job has a LOCK_EXP_TIME_ value in the future could be called aquired.

[prajwolbhandari1]

Hi @prajwolbhandari1,

The proposed logic does not exactly query jobs that are currently executing.

select * from act_ru_job where LOCK_EXP_TIME_ is not null and suspension_state_=1 and LOCK_EXP_TIME_ > currenttime

It is possible that LOCK_EXP_TIME_ > currenttime but a job is not executing: LOCK_EXP_TIME_ is first set when the job executor has acquired the job. However an acquired job might not get executed right away. If a job failed LOCK_EXP_TIME_ might be updated with an icrement as part of the retry policy

Once the acquisition thread acquires a job, it sets LOCK_EXP_TIME_ in some future time and put that job in the job queue right? Isn't it mean its currently "executing"? However, I agree there might be other possible outcomes after that but at that point when the LOCK_EXP_TIME_ is in future time and suspension_state_ is active, my understanding is executing. Also, I am considering retries as "executing" since its the part of a job that is being executed.

It is also not guaranteed that no job is running if LOCK_EXP_TIME_ <= currenttime. In a case where a job does not return within the LOCK_EXP_TIME_ the job can be acquired by a (different of same) job executor. I recommend visiting our documentation on the The Job Executor

Can the job be acquired though because the suspension_state_ will in active state not in suspended even though LOCK_EXP_TIME_ <= currenttime? The state has to be suspended for it to be picked by another acquisition thread right?

If the behaviour you want to achieve is still met with this query then one option would be to split executing into two filters. A query criteria to test if suspension is 1 already exists it is called active A query criteria that checks if a job has a LOCK_EXP_TIME_ value in the future could be called aquired.

@venetrius Thank you for your reply. Please review my above comments.

Can you please suggest to add/remove from the query that I came up that would retrieve jobs that are currently executing?

[prajwolbhandari1]

@venetrius any thoughts?

[prajwolbhandari1]

@yanavasileva @venetrius any thoughts on this?

[venetrius]

Hi @prajwolbhandari1,
I did not have the time to dive in and suggest a change to your query. As soon as I have, I will update you here.

[prajwolbhandari1]

Hi @prajwolbhandari1, I did not have the time to dive in and suggest a change to your query. As soon as I have, I will update you here.

@venetrius any thoughts or direction would be helpful.

[venetrius]

Hi @prajwolbhandari1,
I think the solution you proposed has the downside that users have a different expectation of the result set then what is actually guaranteed.

The API already supports the active flag that is half of the executing check.
Do you think it would make sense for your use case to create a new flag acquired that would check that:

        RES.LOCK_EXP_TIME_ IS NOT NULL
        AND
        RES.LOCK_EXP_TIME_  > #{now, jdbcType=TIMESTAMP}

Then when you can call the API with

{
active: true,
acquired: true
}

the same SQL would be executed then what the current changes would introduce with the executing flag.

[prajwolbhandari1]

Hi @prajwolbhandari1, I think the solution you proposed has the downside that users have a different expectation of the result set then what is actually guaranteed.

The API already supports the active flag that is half of the executing check. Do you think it would make sense for your use case to create a new flag acquired that would check that:

        RES.LOCK_EXP_TIME_ IS NOT NULL
        AND
        RES.LOCK_EXP_TIME_  > #{now, jdbcType=TIMESTAMP}

Then when you can call the API with

{
active: true,
acquired: true
}

the same SQL would be executed then what the current changes would introduce with the executing flag.

Perfect @venetrius and it makes sense, let me change the flag from 'executing' to 'acquired'.

[github-actions]

Closing due to lack of requested feedback. If you would like us to look at this, please provide the requested information to re-open the PR.

[prajwolbhandari1]

@venetrius @yanavasileva can you please reopen this PR? I am in process of working on changes. Is it easier to open a new PR together OR reopen this? I am ok with either one.

Thanks!

[venetrius]

Hi @prajwolbhandari1,
I have reopened this PR.

[prajwolbhandari1]

@venetrius Just pushed a new commit. Please take a look at it.

Thanks again @venetrius

[venetrius]

@venetrius Just pushed a new commit. Please take a look at it.

Thanks again @venetrius

Thanks @prajwolbhandari1,
I will be able to take a look at the end of next week.

[prajwolbhandari1]

@venetrius Just pushed a new commit. Please take a look at it.
Thanks again @venetrius

Thanks @prajwolbhandari1, I will be able to take a look at the end of next week.

Sure. Sounds good. Thanks!

[venetrius]

Hi @prajwolbhandari1,
Unfortunately, I was not able to take a look last week, I will do it in the week.

[prajwolbhandari1]

Hi @prajwolbhandari1, Unfortunately, I was not able to take a look last week, I will do it in the week.

Sounds good. Thank you.

[venetrius]

Hi @prajwolbhandari1,
I see you renamed the filter.
The intent of my previous comment was to break out the single filter to two separate filters (active and acquired). I added comments and suggestions to try to make my point clear.
Please take a look and let me know if you have any questions / concerns.

[venetrius]

Suggested change

	date is in future and suspension state is 1. Value may only be `true`, as `false` is the default
	date is in the future. Value may only be `true`, as `false` is the default

To check if suspension state is 1 one can use the existing active filter

[venetrius]

Suggested change

	<if test="acquired">
	AND
	RES.SUSPENSION_STATE_ = 1
	AND
	RES.LOCK_EXP_TIME_ IS NOT NULL
	AND
	RES.LOCK_EXP_TIME_ > #{now, jdbcType=TIMESTAMP}
	<if test="acquired">
	and
	RES.LOCK_EXP_TIME_ is not null
	and
	RES.LOCK_EXP_TIME_ > #{now, jdbcType=TIMESTAMP}

• In the code base SQL keywords or operators are written in lower case.
• RES.SUSPENSION_STATE_ = 1 can be tested by the existing active filter