Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HARMONY-1878: Make WKT conversion precision configurable #631

Merged
merged 5 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions bin/port-forward
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ function start_port_forwarding() {

stop_port_forwarding "$service_name"

local max_iterations=60
local max_iterations=120
local iteration=0

while [ "$iteration" -lt "$max_iterations" ]; do
Expand All @@ -37,7 +37,7 @@ function start_port_forwarding() {
((iteration++))

if [ "$EXEC_CONTEXT" == "workflow" ]; then
sleep 60
sleep 30
else
sleep 1
fi
Expand Down
28 changes: 14 additions & 14 deletions docs/guides/managing-existing-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,14 @@ The returned JSON response is a map of canonical service names to tags:
"subset-band-name": "latest"
}
```
**Example 2** - Harmony `/service-image-tags` response
**Example 2** - Harmony `/service-image-tag` response

## Get backend service tag (version) information for a specific service

```
curl -Ln -bj https://harmony.uat.earthdata.nasa.gov/service-image-tag/#canonical-service-name
```
**Example 3** - Getting a specific backend service image tag using the `/service-image-tags` API
**Example 3** - Getting a specific backend service image tag using the `/service-image-tag` API

The returned JSON response is a map with a single `tag` field:

Expand All @@ -52,7 +52,7 @@ The returned JSON response is a map with a single `tag` field:
"tag": "1.2.3"
}
```
**Example 4** - Harmony `/service-image-tags` response for a single service
**Example 4** - Harmony `/service-image-tag` response for a single service

## Update backend service tag (version) for a specific service

Expand All @@ -66,17 +66,17 @@ For example:
```
curl -XPUT https://harmony.uat.earthdata.nasa.gov/service-image-tag/#canonical-service-name -H 'Authorization: Bearer <your bearer token>' -d '{"tag": "new-version"}' -H 'Content-type: application/json'
```
**Example 5** - Updating a specific backend service image tag using the `/service-image-tags` API
**Example 5** - Updating a specific backend service image tag using the `/service-image-tag` API

The body of the `PUT` request should be a JSON object with a `tag` field indicating the tag of the updated service image and an optional `test` field with the value of the tag of the regression test docker image, the value 'latest' will be used when `test` field is omitted.
The body of the `PUT` request should be a JSON object with a `tag` field indicating the tag of the updated service image and an optional `regression_test_version` field with the value of the tag of the regression test docker image, the value 'latest' will be used when `regression_test_version` field is omitted.

```JSON
{
"tag": "new-version",
"test": "1.0.0"
"regression_test_version": "1.0.0"
}
```
**Example 6** - Harmony `/service-image-tags` request body for updating a tag
**Example 6** - Harmony `/service-image-tag` request body for updating a tag

The returned JSON response has a tag field indicating the new tag value and a statusLink field with the url for getting the status of the service image deployment.

Expand All @@ -86,7 +86,7 @@ The returned JSON response has a tag field indicating the new tag value and a st
"statusLink": "https://harmony.uat.earthdata.nasa.gov/service-deployment/<deployment-id>"
}
```
**Example 7** - Harmony `/service-image-tags` response for a updating a single service
**Example 7** - Harmony `/service-image-tag` response for a updating a single service

>**Note** this is an asynchronous request, so the status code for the response will be `202 Accepted` - it may take several minutes for the entire update to complete.

Expand Down Expand Up @@ -132,7 +132,7 @@ The returned JSON response has the fields indicating the current status of the s
GET https://harmony.uat.earthdata.nasa.gov/service-deployments-state

```
**Example 10** - Getting the current enable/disable state of the service deployment feature using the `service-image-tag` API
**Example 10** - Getting the current enable/disable state of the service deployment feature using the `service-deployments-state` API

The returned JSON response shows if the service deployment is currently enabled (true) or disabled (false) and any optional message:

Expand All @@ -143,7 +143,7 @@ The returned JSON response shows if the service deployment is currently enabled
}
```
---
**Example 11** - Harmony `service-image-tags` response for enable/disable state
**Example 11** - Harmony `service-deployments-state` response for enable/disable state

## Enable the service deployment feature
The user must have admin permission in order to invoke this endpoint. User can provide an optional message in the JSON body to indicate the reason for enabling. This message will be persisted in database and returned when user retrieves the service deployment state later.
Expand All @@ -154,7 +154,7 @@ For example:
curl -XPUT -H 'Authorization: Bearer <your bearer token>' -H 'Content-type: application/json' https://harmony.uat.earthdata.nasa.gov/service-deployments-state -d '{"enabled": true, "message": "Manually enabled by David"}'
```
---
**Example 12** - Harmony `service-image-tags` request for enabling the service deployment
**Example 12** - Harmony `service-deployments-state` request for enabling the service deployment

The returned JSON response is the same as the get current state of the service deployment feature request above, indicating the current state:

Expand All @@ -164,7 +164,7 @@ The returned JSON response is the same as the get current state of the service d
"message": "Manually enabled by David"
}
```
**Example 13** - Harmony `/service-image-tags` response for enabling the service deployment
**Example 13** - Harmony `/service-deployments-state` response for enabling the service deployment

## Disable the service deployment feature
The user must have admin permission in order to invoke this endpoint. User can provide an optional message in the JSON body to indicate the reason for disabling. This message will be persisted in database and returned when user retrieves the service deployment state later.
Expand All @@ -175,7 +175,7 @@ For example:
curl -XPUT -H 'Authorization: Bearer <your bearer token>' -H 'Content-type: application/json' https://harmony.uat.earthdata.nasa.gov/service-deployments-state -d '{"enabled": false, "message": "Manually disabled by David"}'
```
---
**Example 14** - Harmony `service-image-tags` request for disabling the service deployment
**Example 14** - Harmony `service-deployments-state` request for disabling the service deployment

The returned JSON response is the same as the get current state of the service deployment feature request above, indicating the current state:

Expand All @@ -185,7 +185,7 @@ The returned JSON response is the same as the get current state of the service d
"message": "Manually disabled by David"
}
```
**Example 15** - Harmony `/service-image-tags` response for disabling the service deployment
**Example 15** - Harmony `/service-deployments-state` response for disabling the service deployment

## Getting the Deployment History for All Services
```
Expand Down
6 changes: 2 additions & 4 deletions services/harmony/app/frontends/ogc-edr/get-data-for-point.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,10 @@ import { keysToLowerCase } from '../../util/object';
import { ParameterParseError, mergeParameters, parseWkt, validateWkt } from '../../util/parameter-parsing-helpers';
import { Response, NextFunction } from 'express';
import HarmonyRequest from '../../models/harmony-request';
import env from '../../util/env';
import { RequestValidationError } from '../../util/errors';
import { getDataCommon } from './get-data-common';

// POINT to POLYGON conversion side length, 0.0001 is about 11 meters in precision
const POINT_PRECISION = 0.0001;

/**
* Converts a WKT POINT string to a WKT POLYGON string.
* The polygon is a square centered around the point.
Expand Down Expand Up @@ -82,7 +80,7 @@ function wktMultipointToMultipolygon(wktMultipoint: string, sideLength: number):
* @returns The converted WKT POLYGON or WKT MULTIPOLYGON string.
* @throws RequestValidationError if the WKT string format is invalid.
*/
export function convertWktToPolygon(wkt: string, sideLength: number = POINT_PRECISION): string {
export function convertWktToPolygon(wkt: string, sideLength: number = env.wktPrecision): string {
if (wkt.startsWith('POINT')) {
return wktPointToPolygon(wkt, sideLength);
} else if (wkt.startsWith('MULTIPOINT')) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,10 @@ import { keysToLowerCase } from '../../util/object';
import { ParameterParseError, mergeParameters, parseWkt, validateWkt } from '../../util/parameter-parsing-helpers';
import { Response, NextFunction } from 'express';
import HarmonyRequest from '../../models/harmony-request';
import env from '../../util/env';
import { RequestValidationError } from '../../util/errors';
import { getDataCommon } from './get-data-common';

// LINESTRING to POLYGON conversion side length, 0.0001 is about 11 meters in precision
const LINESTRING_PRECISION = 0.0001;

type Point = { x: number, y: number };

/**
Expand Down Expand Up @@ -162,7 +160,7 @@ function wktMultiLineStringToMultipolygon(
*/
export function convertWktLineToPolygon(
wkt: string,
sideLength: number = LINESTRING_PRECISION): string {
sideLength: number = env.wktPrecision): string {

if (wkt.startsWith('LINESTRING')) {
return convertLineStringToPolygon(wkt, sideLength);
Expand Down
5 changes: 4 additions & 1 deletion services/harmony/app/util/env.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { IsInt, IsNotEmpty, Matches, Min } from 'class-validator';
import { IsInt, IsNotEmpty, IsPositive, Matches, Min } from 'class-validator';
import { HarmonyEnv, memorySizeRegex } from '@harmony/util/env';
import _ from 'lodash';
import * as path from 'path';
Expand Down Expand Up @@ -108,6 +108,9 @@ class HarmonyServerEnv extends HarmonyEnv {
@Min(1)
maxDataOperationCacheSize: number;

@IsPositive()
wktPrecision: number;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: You might want to add a constraint that this is a positive number

locallyDeployedServices: string;
}

Expand Down
3 changes: 3 additions & 0 deletions services/harmony/env-defaults
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,9 @@ MAX_POST_FILE_PARTS=100
# Maximum size (in bytes) of the cache for data operations
MAX_DATA_OPERATION_CACHE_SIZE=512000000

# WKT POINT/LINESTRING to POLYGON conversion side length, 0.0001 is about 11 meters in precision
WKT_PRECISION=0.0001

#############################################################################
# OAuth 2 (Earthdata Login) #
# #
Expand Down
Loading