Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge dist to prod #183

Merged
merged 61 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
721759b
refactor integrated alerts algo and add dist alerts
solomon-negusse Oct 16, 2024
90d7d4c
use namedtuple both for immutability and access with labels
solomon-negusse Oct 17, 2024
902af0b
refactor to modularize and slim down __call__ method
solomon-negusse Oct 17, 2024
7670bac
encapsulate alpha logic in a method
solomon-negusse Oct 18, 2024
6e5f12a
add unit tests for alerts titiler algorithm
solomon-negusse Oct 18, 2024
6731629
serve encoded rgb to support Flagship date slider
solomon-negusse Oct 21, 2024
08d40ff
reuse alert config named tuple definitions; rename deforestationalert…
solomon-negusse Oct 21, 2024
08b1b0c
move s3 creds/bucket name to env variable
solomon-negusse Oct 21, 2024
bf96b71
fix start date
solomon-negusse Oct 21, 2024
697833f
fix typo
solomon-negusse Oct 21, 2024
6d43d41
fix typo
solomon-negusse Oct 21, 2024
7a379bf
Merge pull request #164 from wri/dist-alert-algo
solomon-negusse Oct 22, 2024
e76a222
merge
solomon-negusse Oct 22, 2024
f5f2d45
Fix typo
solomon-negusse Oct 22, 2024
514d771
add documentation and log message
solomon-negusse Oct 22, 2024
1c70283
Merge branch 'titiler/encoded-alert-tiles' of github.com:wri/gfw-tile…
solomon-negusse Oct 22, 2024
949639d
Merge pull request #166 from wri/titiler/encoded-alert-tiles
solomon-negusse Oct 22, 2024
397763f
Merge branch staging into dev
solomon-negusse Oct 22, 2024
3e8a831
Merge pull request #167 from wri/dev
solomon-negusse Oct 22, 2024
4e9ba8d
Merge branch 'production' into staging
solomon-negusse Oct 23, 2024
6366120
dont use tilefactory for algos for ease of customization; make titile…
solomon-negusse Oct 25, 2024
d0e57d4
make encoded the default render_type
solomon-negusse Oct 25, 2024
d255c24
don't use default alertconfidence enum; create separate alertconf enu…
solomon-negusse Oct 25, 2024
148ae02
update titiler test to point to new endpoint
solomon-negusse Oct 25, 2024
90825e2
remove unused implementation query param from titiler endpoint
solomon-negusse Oct 28, 2024
0343d07
add comment in the order of bands in tile
solomon-negusse Oct 28, 2024
c66efd2
add forest filter layers
solomon-negusse Oct 30, 2024
57d86bb
add DIST alerts forest mask test
solomon-negusse Oct 31, 2024
a8ffd95
add tcl mask that was left behind
solomon-negusse Oct 31, 2024
2401b99
consistent route file naming
solomon-negusse Oct 31, 2024
9463b2c
use the version enum
solomon-negusse Oct 31, 2024
fa2aabb
use consistent import alias
solomon-negusse Oct 31, 2024
bbe05f2
use color/conf and rendertype defaults
solomon-negusse Oct 31, 2024
736e98c
fix import
solomon-negusse Oct 31, 2024
406e29a
test test asset
solomon-negusse Nov 1, 2024
9d53a43
update latest versions in test
solomon-negusse Nov 1, 2024
a7ca04f
fix list order
solomon-negusse Nov 1, 2024
bc349c0
Merge pull request #169 from wri/gtc-3020/update-titile-endpoints
solomon-negusse Nov 1, 2024
78af193
Merge branch 'dev' into gtc-3025/filter-alerts-by-forest
solomon-negusse Nov 1, 2024
32857c6
clean up tree cover loss logic and comment
solomon-negusse Nov 1, 2024
7f968e8
simplify tree cover loss doc
solomon-negusse Nov 1, 2024
6fe4291
fix TCL mask for DIST alert
solomon-negusse Nov 4, 2024
454d840
update docs for clarity; add todo comment for version update in prod
solomon-negusse Nov 5, 2024
693e0c2
get DIST alerts filter dataset/versions
solomon-negusse Nov 7, 2024
2e04a59
Merge pull request #170 from wri/gtc-3025/filter-alerts-by-forest
solomon-negusse Nov 7, 2024
efab786
Merge branch 'staging' into dev
solomon-negusse Nov 7, 2024
bd534de
Merge pull request #171 from wri/dev
solomon-negusse Nov 7, 2024
0b1b1d2
update dataset name; allow test datasets
solomon-negusse Nov 8, 2024
691bd24
add titiler endpoint to cloudfront
solomon-negusse Nov 10, 2024
4a53264
add clarifying comment for the two titiler paths
solomon-negusse Nov 11, 2024
df7b40a
Merge pull request #173 from wri/update-dist-alerts-dataset
solomon-negusse Nov 11, 2024
41b5943
fix tree cover loss year field type; update var name to make mypy happy
solomon-negusse Nov 11, 2024
74e8504
Merge pull request #174 from wri/fix-field-type
solomon-negusse Nov 11, 2024
2692cd6
consistent and descriptive forest filter query params; detailed decod…
solomon-negusse Nov 13, 2024
c35d972
doc styling; add alert decoding example
solomon-negusse Nov 13, 2024
24dbdae
doc updates
solomon-negusse Nov 13, 2024
e8dd996
Merge pull request #175 from wri/field-name-updates
solomon-negusse Nov 13, 2024
e2c71b7
minor doc fix
solomon-negusse Nov 13, 2024
bc6a6c7
:bug: fix: implementation replacement on redirect (#176) (#178)
gtempus Nov 21, 2024
e346d10
Merge all the dist_alerts changes to production
danscales Dec 12, 2024
96c7862
Fix "Tile index out of bounds" error for very north tiles
danscales Dec 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@
from .routes import preview

from .routes.titiler import routes as titiler_routes
from .routes.titiler.gfw_integrated_alerts import router as integrated_alerts_router
from .routes.titiler.umd_glad_dist_alerts import router as dist_alerts_router

gunicorn_logger = logging.getLogger("gunicorn.error")
logger.handlers = gunicorn_logger.handlers
Expand All @@ -55,6 +57,8 @@
burned_areas_tiles.router,
dynamic_vector_tiles.router,
vector_tiles.router,
integrated_alerts_router,
dist_alerts_router,
umd_tree_cover_loss_raster_tiles.router,
umd_glad_landsat_alerts_raster_tiles.router,
umd_glad_sentinel2_alerts_raster_tiles.router,
Expand All @@ -79,10 +83,6 @@
app.include_router(
titiler_routes.mosaic.router, prefix="/cog/mosaic", tags=["Mosaic Tiles"]
)
app.include_router(
titiler_routes.custom.router, prefix="/cog/custom", tags=["Custom Tiles"]
)


#####################
## Middleware
Expand Down
7 changes: 0 additions & 7 deletions app/models/enumerators/alerts_confidence.py

This file was deleted.

17 changes: 17 additions & 0 deletions app/models/enumerators/titiler.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
from enum import Enum


class AlertConfidence(str, Enum):
low = "low"
high = "high"


class IntegratedAlertConfidence(str, Enum):
low = "low"
high = "high"
highest = "highest"


class RenderType(str, Enum):
true_color = "true_color"
encoded = "encoded"
5 changes: 4 additions & 1 deletion app/routes/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@


def to_bbox(x: int, y: int, z: int) -> Bounds:
'''x and y are the tile column and row, z is the zoom level'''
logger.debug(f"Coordinates (X, Y, Z): {x},{y},{z}")
left, bottom, right, top = mercantile.xy_bounds(x, y, z)
logger.debug(f"Bounds (Left, Bottom, Right, Top): {left},{bottom},{right},{top}")
Expand Down Expand Up @@ -232,7 +233,9 @@ def validate_dates(start_date: str, end_date: str, force_date_range) -> None:

def validate_bbox(left: float, bottom: float, right: float, top: float) -> None:
"""Tile should be within WebMercator extent."""

# Extent of whole-world (single) tile at zoom 0, in meters.
min_left, min_bottom, max_right, max_top = mercantile.xy_bounds(0, 0, 0)

if left < min_left or bottom < min_bottom or right > max_right or top > max_top:
if round(left, 0) < round(min_left, 0) or round(bottom, 0) < round(min_bottom, 0) or round(right, 0) > round(max_right, 0) or round(top, 0) > round(max_top, 0):
raise HTTPException(status_code=400, detail="Tile index is out of bounds")
101 changes: 0 additions & 101 deletions app/routes/titiler/algorithms.py

This file was deleted.

Empty file.
175 changes: 175 additions & 0 deletions app/routes/titiler/algorithms/alerts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
from collections import OrderedDict, namedtuple
from typing import Optional

import numpy as np
from fastapi.logger import logger
from rio_tiler.models import ImageData
from titiler.core.algorithm import BaseAlgorithm

from app.models.enumerators.titiler import IntegratedAlertConfidence, RenderType

Colors: namedtuple = namedtuple("Colors", ["red", "green", "blue"])
AlertConfig: namedtuple = namedtuple("AlertConfig", ["confidence", "colors"])


class Alerts(BaseAlgorithm):
"""Decode Deforestation Alerts."""

title: str = "Deforestation Alerts"
description: str = "Decode and visualize alerts"

conf_colors: OrderedDict = OrderedDict(
{
IntegratedAlertConfidence.low: AlertConfig(
confidence=2, colors=Colors(237, 164, 194)
),
IntegratedAlertConfidence.high: AlertConfig(
confidence=3, colors=Colors(220, 102, 153)
),
IntegratedAlertConfidence.highest: AlertConfig(
confidence=4, colors=Colors(201, 42, 109)
),
}
)

record_start_date: str = "2014-12-31"

start_date: Optional[str] = None
end_date: Optional[str] = None
alert_confidence: Optional[str] = None
render_type: RenderType = RenderType.true_color

# metadata
input_nbands: int = 2
output_nbands: int = 4
output_dtype: str = "uint8"

def __call__(self, img: ImageData) -> ImageData:
"""Process the input image and decode deforestation or land disturbance
alert raster data into RGBA format.

Args:
img (ImageData): Input image data with alert date/confidence and intensity
(zoom-level visibility) layers.

Returns:
ImageData: Processed image with RGBA channels either with true colors ready for
visualization or encoding date and confidence for front-end processing.
"""
date_conf_data = img.data[0]

self.intensity = img.data[1]
self.no_data = img.array.mask[0]
self.data_alert_confidence = date_conf_data // 10000
self.alert_date = date_conf_data % 10000

self.mask = self.create_mask()

if self.render_type == RenderType.true_color:
rgb = self.create_true_color_rgb()
alpha = self.create_true_color_alpha()
else: # encoded
rgb = self.create_encoded_rgb()
alpha = self.create_encoded_alpha()

data = np.vstack([rgb, alpha[np.newaxis, ...]]).astype(self.output_dtype)
data = np.ma.MaskedArray(data, mask=False)

return ImageData(data, assets=img.assets, crs=img.crs, bounds=img.bounds)

def create_mask(self):
"""Generate a mask for pixel visibility based on date and confidence
filters, and no data values.

Returns:
np.ndarray: A mask array pixels with no alert or alerts not meeting filter
condition are masked.
"""

mask = ~self.no_data

if self.alert_confidence:
confidence_mask = (
self.data_alert_confidence
>= self.conf_colors[self.alert_confidence].confidence
)
mask *= confidence_mask

if self.start_date:
start_mask = self.alert_date >= (
np.datetime64(self.start_date) - np.datetime64(self.record_start_date)
)
mask *= start_mask

if self.end_date:
end_mask = self.alert_date <= (
np.datetime64(self.end_date) - np.datetime64(self.record_start_date)
)
mask *= end_mask

return mask

def create_true_color_rgb(self):
"""Map alert confidence levels to RGB values for visualization.

Returns:
np.ndarray: A 3D array with RGB channels.
"""
r, g, b = self._rgb_zeros_array()

for properties in self.conf_colors.values():
confidence = properties.confidence
colors = properties.colors
r[self.data_alert_confidence >= confidence] = colors.red
g[self.data_alert_confidence >= confidence] = colors.green
b[self.data_alert_confidence >= confidence] = colors.blue

return np.stack([r, g, b], axis=0)

def create_encoded_rgb(self):
"""Encode the alert date and confidence into the RGB channels, allowing
interactive date filtering and color control on Flagship.

Returns:
np.ndarray: A 3D array with encoded RGB values.
"""
r, g, b = self._rgb_zeros_array()
r = self.alert_date // 255
g = self.alert_date % 255
b = (self.data_alert_confidence // 3 + 1) * 100 + self.intensity

return np.stack([r, g, b], axis=0)

def create_true_color_alpha(self):
"""Set the transparency (alpha) channel for alert pixels based on date,
confidence filters, and intensity input. The intensity multiplier is
used to control how isolated alerts fade out at low zoom levels,
matching the rendering behavior in Flagship.

Returns:
np.ndarray: Array representing the alpha (transparency) channel, where pixel
visibility is adjusted by intensity.
"""
alpha = np.where(self.mask, self.intensity * 150, 0)
return np.minimum(255, alpha)

def create_encoded_alpha(self):
"""Generate the alpha channel for encoded alerts. The default
implementation sets pixel visibility based on date/confidence filters
and intensity input. Can be overridden for specific alert types.

Returns:
np.ndarray: An array representing the alpha channel.
"""
logger.info(
"""Encoded alpha not provided, returning alpha
from input layer and date/confidence mask."""
)
return self.create_true_color_alpha()

def _rgb_zeros_array(self):
r = np.zeros_like(self.data_alert_confidence, dtype=np.uint8)
g = np.zeros_like(self.data_alert_confidence, dtype=np.uint8)
b = np.zeros_like(self.data_alert_confidence, dtype=np.uint8)

return r, g, b
Loading
Loading