Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/n calls per hour #381

Merged
merged 17 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ You will need to set the following environmental variables:
- `QUERY_WAIT_SECONDS` - The number of seconds to wait for an on going query
- `CACHE_TIME_SECONDS` - The time in seconds to cache the data is used for
- `DELETE_CACHE_TIME_SECONDS` - The time in seconds to after which the cache is delete
- `LOGLEVEL` - The log level for the application.

Note you will need a database set up at `DB_URL`. This should use the datamodel in [nowcasting_datamodel](https://github.com/openclimatefix/nowcasting_datamodel)

Expand Down
22 changes: 17 additions & 5 deletions src/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,10 @@ def wrapper(*args, **kwargs): # noqa

last_updated, response = remove_old_cache(last_updated, response)

# make route_variables into a string
# make route_variables into a string\
# TODO add url
# TODO sort route variables alphabetically
# url = request.url
route_variables = json.dumps(route_variables)

# use case
Expand Down Expand Up @@ -126,13 +129,20 @@ def wrapper(*args, **kwargs): # noqa
return response[route_variables]
else:
logger.warning(
f"Waited {QUERY_WAIT_SECONDS} seconds but response not "
f"in cache. Setting this route as not running, "
f"and continuing"
"Process finished running but response not "
"in cache. Setting this route as not running, "
"and continuing"
)
currently_running[route_variables] = False
break

logger.warning(
f"Waited {QUERY_WAIT_SECONDS} seconds but response not "
f"in cache. Setting this route as not running, "
f"and continuing"
)
currently_running[route_variables] = False

# 1.1 check if its been called before and not currently running
if (route_variables not in last_updated) and (
not currently_running.get(route_variables, False)
Expand All @@ -149,7 +159,9 @@ def wrapper(*args, **kwargs): # noqa

# 1.2 rerun if cache time out is up and not currently running
now = datetime.now(tz=timezone.utc)
if now - timedelta(seconds=cache_time_seconds) > last_updated[route_variables] and (
if route_variables not in last_updated:
pass
elif now - timedelta(seconds=cache_time_seconds) > last_updated.get(route_variables) and (
not currently_running.get(route_variables, False)
):
logger.debug(
Expand Down
2 changes: 1 addition & 1 deletion src/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ async def add_process_time_header(request: Request, call_next):
start_time = time.time()
response = await call_next(request)
process_time = str(time.time() - start_time)
logger.debug(f"Process Time {process_time} {request.url}")
logger.info(f"Process Time {process_time} {request.url}")
response.headers["X-Process-Time"] = process_time

return response
Expand Down
Loading