Skip to content

Commit

Permalink
new design proposal
Browse files Browse the repository at this point in the history
Signed-off-by: Ido Aharon <[email protected]>
  • Loading branch information
Ido Aharon committed May 1, 2024
1 parent 9100913 commit 777da98
Showing 1 changed file with 8 additions and 10 deletions.
18 changes: 8 additions & 10 deletions design-proposals/data-import-cron-http-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,23 @@ The design will also demonstrate to the user how to operate with such import typ
## Motivation
Currently, the DataImportCron allows the import of Registry Imports only. Recently, there was a demand to also allow HTTP import types.
The problem that arises from HTTP imports is that there is no convention between the different sources, so it's hard to know when the image is updated for each source in a generic way, which will make the polling process more difficult than standard registry sources.
One possible solution If the URL is static is to use a Get request with an If-Modified-Since header where we specify the date from which we want to check if there was a change. If there was a change since the specified date, the request will return with a status of 200OK, and then we know that the image has been updated since that date.

In the current situation we can enable HTTP imports by updating the DesiredDigest label.
The current approach is to support only sources that support the If-Modified-Since header.
If there was a change since the specified date, the request will return with a status of 200OK, and then we know that the image has been updated since that date.

## Goals

* Allow the user to perform http imports manually with dataimportcron
* Create a poller that will cover as many cases as possible for automatic updating
* Allow the user to perform http imports manually with dataimportcron.
* Create a poller that will cover automatic updating with the help of If-Modified-Since header.

## Non Goals

* The poller will probably not cover all import cases and sometimes the user will have to manually update the digest
* The poller will probably not cover all import cases and sometimes the user will have to do manual update.

## User Stories

* As a user, I would like to import images from an HTTP source using DataImportCron
* As a user, I would like the poller automatically update the image when it is needed
* As a user, I would like to manually trigger an HTTP import with DataImportCron
* As a user, I would like to import images from an HTTP source using DataImportCron.
* As a user, I would like the poller automatically update the image when the source is updated.
* As a user, I would like to manually trigger an HTTP import with DataImportCron.

## Repos

Expand All @@ -41,7 +40,6 @@ In the current situation we can enable HTTP imports by updating the DesiredDiges

# Design

If the URL is static:
* Make a GET request with the If-Modified-Since Header starting from the date stored in the AnnLastCronTime annotation
* If the returned status is 200OK, perform import again
* Update AnnLastCronTime to time.Now()
Expand Down

0 comments on commit 777da98

Please sign in to comment.