Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional batch job timestamps #556

Open
wants to merge 7 commits into
base: draft
Choose a base branch
from
Open

Add additional batch job timestamps #556

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1023a30
Added `queued`, `started` and `expires` to the batch job metadata and…
m-mohr Jan 12, 2025
3286739
Apply suggestions from code review
m-mohr Jan 25, 2025
80cd197
Apply suggestions from code review
m-mohr Jan 25, 2025
5f7ca8e
Update according to review, also add to Collection result type
m-mohr Jan 28, 2025
113b1fc
Merge branch 'draft' into job-timestamps
m-mohr Jan 28, 2025
9fe0267
Apply suggestions from code review
m-mohr Jan 28, 2025
2c3815e
Typo
m-mohr Jan 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **New extensions:**
- [Remote Process Definition Extension](./extensions/remote-process-definition/README.md)
- Added `version` property to `GET /processes` [#517](https://github.com/Open-EO/openeo-api/issues/517)
- Added `queued`, `started` and `expires` to the batch job metadata and the corresponding STAC results. [#542](https://github.com/Open-EO/openeo-api/issues/542)

### Fixed

Expand Down
48 changes: 38 additions & 10 deletions openapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4010,13 +4010,12 @@ components:
$ref: '#/components/schemas/created'
updated:
$ref: '#/components/schemas/updated'
queued:
$ref: '#/components/schemas/queued'
started:
$ref: '#/components/schemas/started'
expires:
type: string
format: date-time
description: >-
Time until which the assets are accessible, in UTC. Formatted as
a [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
example: '2020-11-01T00:00:00Z'
$ref: '#/components/schemas/expires'
'openeo:status':
$ref: '#/components/schemas/result_status'
assets:
Expand Down Expand Up @@ -5598,6 +5597,12 @@ components:
$ref: '#/components/schemas/created'
updated:
$ref: '#/components/schemas/updated'
queued:
$ref: '#/components/schemas/queued'
started:
$ref: '#/components/schemas/started'
expires:
$ref: '#/components/schemas/expires'
plan:
$ref: '#/components/schemas/billing_plan'
costs:
Expand Down Expand Up @@ -5660,16 +5665,39 @@ components:
type: string
format: date-time
description: >-
Date and time of creation, formatted as a [RFC
3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
Date and time of creation (for batch jobs: the status 'created' was set),
formatted as a [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
example: '2017-01-01T09:32:12Z'
updated:
type: string
format: date-time
description: >-
Date and time of the last status change, formatted as a [RFC
3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
Date and time of the last status change,
formatted as a [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
If the status is `error`, `canceled` or `finished`,
this is the time when the job has ended.
example: '2017-01-01T09:36:18Z'
queued:
type: string
format: date-time
description: >-
Date and time of queueing the batch job (i.e. when the status 'queued' was set),
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
formatted as a [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
example: '2017-01-01T09:34:00Z'
started:
type: string
format: date-time
description: >-
Date and time when the batch job started processing (i.e. when the status 'running' was set),
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
formatted as a [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
example: '2017-01-01T09:36:18Z'
expires:
type: string
format: date-time
description: >-
Time until which the assets are accessible, in UTC. Formatted as
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Time until which the assets are accessible, in UTC. Formatted as
Time until which the assets (e.g., batch job results) are available for download, in UTC. Formatted as

Copy link
Member Author

@m-mohr m-mohr Jan 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might be a bit misleading.
expires can be used for when the signed URLs expire and also for when the data is deleted, I think. That's why I chose accessible. The addition in brackets is fine though.

Suggested change
Time until which the assets are accessible, in UTC. Formatted as
Time until which the assets (e.g., batch job results) are accessible, in UTC. Formatted as

Maybe we should actually split into expires and unpublished, i.e.

  • expires: signed URL expiry (if relevant here?) but in STAC it's that...
  • unpublished: when the data gets deleted

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok good point.
But indeed, we should separate the expiry of signed URLs (which can be refreshed easily) and the expiry for download availability in general (which is final and non-recoverable).

I think the general availability of results should be a global property on GET /jobs/{job_id} or GET /jobs/{job_id}/results, while the signed URL expiry should be closer to the links/URLs themselves (e.g. different signed URLs within the same resource might have different expiry for some advanced reason)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I've updated the PR accordingly:

  • Clarified usage of expires, removed from the batch job info endoint
  • Added unpublished to batch job info endpoint and batch job results
  • Added all the timestamps also to the Collection-typed result response, previously it was only defined in the Item-typed result response

a [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date-time.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
example: '2020-11-01T00:00:00Z'
description:
type: string
format: commonmark
Expand Down
Loading