[Fixes #8148] [Harvesting] Adapt / extend remote service model (#8149)

* Bump urllib3 from 1.26.2 to 1.26.3 (#6908) Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.3. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/1.26.3/CHANGES.rst) - [Commits](urllib3/urllib3@1.26.2...1.26.3) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni <[email protected]> * [Fixes #6880] Circle CI upload tests fail irregulary (#6881) * [Fixes #6880] Circle CI upload tests fail irregulary * CircleCI test fix: sometimes expires due to upload timeout in the test environment * - Avoid infinite loop on upload testing * Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment" This reverts commit 66139fd. Co-authored-by: Alessio Fabiani <[email protected]> Co-authored-by: afabiani <[email protected]> * [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915) * Added malnajdi as contributor * [Fixes #6910] meaningful filename for document download (#6911) * get meaningful document filenames on download * - Strip extension from document title before slugify it (e.g.: image.jpg instead of imagejpg.jpg) Co-authored-by: afabiani <[email protected]> Co-authored-by: Alessio Fabiani <[email protected]> * - CircleCI Upload Tests: trying to reduce more the risk of infinite loop on "wait_for_progress" * [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION (#6913) * [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION * [Fixes #6916] unit test for GEOSERVER_LOCATION * Bump django-cors-headers from 3.6.0 to 3.7.0 (#6901) Bumps [django-cors-headers](https://github.com/adamchainz/django-cors-headers) from 3.6.0 to 3.7.0. - [Release notes](https://github.com/adamchainz/django-cors-headers/releases) - [Changelog](https://github.com/adamchainz/django-cors-headers/blob/master/HISTORY.rst) - [Commits](adamchainz/django-cors-headers@3.6.0...3.7.0) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump amqp from 5.0.3 to 5.0.5 (#6905) Bumps [amqp](https://github.com/celery/py-amqp) from 5.0.3 to 5.0.5. - [Release notes](https://github.com/celery/py-amqp/releases) - [Changelog](https://github.com/celery/py-amqp/blob/master/Changelog) - [Commits](celery/py-amqp@v5.0.3...v5.0.5) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pip from 21.0 to 21.0.1 (#6900) Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1. - [Release notes](https://github.com/pypa/pip/releases) - [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst) - [Commits](pypa/pip@21.0...21.0.1) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump coverage from 5.3.1 to 5.4 (#6903) Bumps [coverage](https://github.com/nedbat/coveragepy) from 5.3.1 to 5.4. - [Release notes](https://github.com/nedbat/coveragepy/releases) - [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst) - [Commits](nedbat/coveragepy@coverage-5.3.1...coverage-5.4) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pytest from 6.2.1 to 6.2.2 (#6907) Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.1 to 6.2.2. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst) - [Commits](pytest-dev/pytest@6.2.1...6.2.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump djangorestframework-gis from 0.16 to 0.17 (#6902) Bumps [djangorestframework-gis](https://github.com/openwisp/django-rest-framework-gis) from 0.16 to 0.17. - [Release notes](https://github.com/openwisp/django-rest-framework-gis/releases) - [Changelog](https://github.com/openwisp/django-rest-framework-gis/blob/master/CHANGES.rst) - [Commits](openwisp/django-rest-framework-gis@v0.16.0...v0.17.0) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * - Algin setup.cfg to requirements.txt * [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923) * [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded * - Add REST APIs test suite to CircleCI * [Fixes #6918] Removal of QGIS support (#6919) * [Cleanup and Refactor] Remove QGIS server backend dependencies * [Cleanup and Refactor] Remove QGIS server backend dependencies * - Fix LGTM issues * allow Basic authenticated requests in LOCKDOWN mode * fix to avoid circular import * flake8 check fix * added tests * [Fixes #6880] Circle CI upload tests fail irregulary (#6881) * [Fixes #6880] Circle CI upload tests fail irregulary * CircleCI test fix: sometimes expires due to upload timeout in the test environment * - Avoid infinite loop on upload testing * Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment" This reverts commit 66139fd. Co-authored-by: Alessio Fabiani <[email protected]> Co-authored-by: afabiani <[email protected]> * [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915) * Added malnajdi as contributor * Bump pip from 21.0 to 21.0.1 (#6900) Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1. - [Release notes](https://github.com/pypa/pip/releases) - [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst) - [Commits](pypa/pip@21.0...21.0.1) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * - Algin setup.cfg to requirements.txt * [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923) * [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded * - Add REST APIs test suite to CircleCI * [Fixes #6918] Removal of QGIS support (#6919) * [Cleanup and Refactor] Remove QGIS server backend dependencies * [Cleanup and Refactor] Remove QGIS server backend dependencies * - Fix LGTM issues * allow Basic authenticated requests in LOCKDOWN mode * fix to avoid circular import * - Align to upstream master branch * [Fixes #7945] Ingest harvested layer data to geonode * Refactor celery tasks in order to allow performing harvesting of selected harvestable resources * Implement copying of remote resources for the GeoNode legacy harvester * Improve harvesting session and the admin * Ensure `extension` is present for harvested GeoNode documents * Add missing default attributes to harvested resources * fixing tests * fixing tests * [Fixes #8148] [Harvesting] Adapt / extend remote service model * - Make sure we set all the "Remote" attributes correctly for a Dataset * - Getting rid of outdated comments * fix migration files conflict * [LGTM] Fix Issues * [CircleCI] Test Fixes * Renamed `geonode` harvester module to `geonodeharvester` Hoping to make lgtm happy * Add migrations to rename geonode harvester module name * Merge branch '7945-harvest-layer-data' of https://github.com/meomancer/geonode into ISSUE_8148 # Conflicts: # geonode/harvesting/harvesters/base.py * Merge branch '7945-harvest-layer-data' of https://github.com/meomancer/geonode into ISSUE_8148 # Conflicts: # geonode/harvesting/harvesters/base.py * [Flake8] Autopep8 * - Fixing migrations * [Dependencies] Align "setup.cfg" to "requirements.txt" * [Remote Services] Harvest a single resource instead of triggering all the marked ones again. * - Few more sanity checks on the 'remote_service' property existence * [Dependencies] Align "setup.cfg" to "requirements.txt" * - Make sure we don't save a dataset twice in case the "final" upload step has been called again * - Using "GeonodeLegacyHarvester" on "Remote Services" of type "GN_WMS" * [Fixes #7945] Ingest harvested layer data to geonode * Refactor celery tasks in order to allow performing harvesting of selected harvestable resources * Implement copying of remote resources for the GeoNode legacy harvester * Improve harvesting session and the admin * Ensure `extension` is present for harvested GeoNode documents * Add missing default attributes to harvested resources * fixing tests * fixing tests * fix migration files conflict * Renamed `geonode` harvester module to `geonodeharvester` Hoping to make lgtm happy * Add migrations to rename geonode harvester module name * Improve support for harvesting raster datasets and disable harvesting maps * - Make sure ll_bbox and default links have been generated for the geonode resources * [Harvesters - WMS] Harvesters should merge remote url parameters if specified Harvester [Harvesters - WMS] Harvesters should use WMS operations to fetch the GetCapabilities Harvester [Harvesters - WMS] GeoNode resources are not correctly defined Harvester [Harvesters - WMS] The GeoNode-WMS service legacy type is missing Harvester [Harvester] GeoNode Legacy Harvester should fetch the metadata and thumbnail from the Remote resources * [Fixes #7945] Ingest harvested layer data to geonode * Improve harvesting session and the admin * fix migration files conflict * Initial work for implementing stoppable harvesting sessions * Implement aborting of harvesting celery tasks Add the `AsynchronousHarvestingSession` model, which is used to implement sessions for both refreshing of a harvester's harvestable resources and for the harvesting of remote resources. Refactor the `admin`, `api` and `tasks` to use this new model. Moved some functions out of `harvesting.utils` module in order to avoid circular imports * fix tests * fix conflicts * Remove accidental duplication of code that crept in during conflict resolution * Add a couple more tests * Update signature of get_resource method in WMS harvester worker * Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used * - Tests Fixes * - Adapt to the new "AsyncHarvestingSession" model changes * - Better management of "Refreshing" and "Harvesting" sessions status from the Remote Service detail page * - Tests Fixes * - General improvements to the PR and fixes to the migrations * - Cleaning up models and getting rid of unused proxy stuff * - Tests fixes * - "upgradeservices" management command * fix filter icon * - Fixes and improvements to the harvester WMS resource discovery * - Tests fixes * - Create thumbs on geonode resources from WMS harvesters * - Update harvester on "rescan" service action * fix filtering with TableFilter * fix filtering views and html * - Create thumbs on geonode resources from WMS harvesters * fix unefined checkbox id * - Create thumbs on geonode resources from WMS harvesters * - Remote Services: create thumbnail uses the original SRID * [Pep8] Fixing Flake8 issues * - Make sure the "thumb" is shown only after the resource has been fully uploaded * - WMS Harvester: make sure we parse the correct CRS and BBOX from GetCapa XML response * - Fix thumbnail generation through the APIs (cherry picked from commit 1c07928) # Conflicts: # geonode/base/api/views.py Co-authored-by: Giovanni Allegri <[email protected]> Co-authored-by: allyoucanmap <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni <[email protected]> Co-authored-by: Florian Hoedt <[email protected]> Co-authored-by: Mohammed Y. Alnajdi <[email protected]> Co-authored-by: biegan <[email protected]> Co-authored-by: Ricardo Garcia Silva <[email protected]> Co-authored-by: meomancer <[email protected]> Co-authored-by: Ricardo Garcia Silva <[email protected]>
GeoNode · Oct 8, 2021 · 4cca7c0 · 4cca7c0
1 parent 0867642
commit 4cca7c0
Show file tree

Hide file tree

Showing 73 changed files with 1,484 additions and 1,731 deletions.
diff --git a/.env b/.env
@@ -37,6 +37,7 @@ GEONODE_DB_CONN_MAX_AGE=0
 GEONODE_DB_CONN_TOUT=5
 DEFAULT_BACKEND_DATASTORE=datastore
 BROKER_URL=amqp://guest:guest@rabbitmq:5672/
+CELERY_BEAT_SCHEDULER=celery.beat:PersistentScheduler
 ASYNC_SIGNALS=True
 
 SITEURL=http://localhost/

diff --git a/.env_dev b/.env_dev
@@ -37,6 +37,7 @@ GEONODE_DB_CONN_MAX_AGE=0
 GEONODE_DB_CONN_TOUT=5
 DEFAULT_BACKEND_DATASTORE=datastore
 BROKER_URL=amqp://admin:admin@localhost:5672//
+CELERY_BEAT_SCHEDULER=celery.beat:PersistentScheduler
 ASYNC_SIGNALS=False
 
 SITEURL=http://localhost:8000/

diff --git a/.env_local b/.env_local
@@ -37,6 +37,7 @@ GEONODE_DB_CONN_MAX_AGE=0
 GEONODE_DB_CONN_TOUT=5
 DEFAULT_BACKEND_DATASTORE=datastore
 BROKER_URL=amqp://admin:admin@localhost:5672//
+CELERY_BEAT_SCHEDULER=celery.beat:PersistentScheduler
 ASYNC_SIGNALS=False
 
 SITEURL=http://localhost/

diff --git a/.env_test b/.env_test
@@ -37,6 +37,7 @@ GEONODE_DB_CONN_MAX_AGE=0
 GEONODE_DB_CONN_TOUT=5
 DEFAULT_BACKEND_DATASTORE=datastore
 BROKER_URL=amqp://guest:guest@rabbitmq:5672/
+CELERY_BEAT_SCHEDULER=celery.beat:PersistentScheduler
 ASYNC_SIGNALS=False
 
 SITEURL=http://localhost:8001/

diff --git a/celery-cmd b/celery-cmd
@@ -3,20 +3,20 @@
 # Luca Pasquali <[email protected]>
 CELERY_BIN=${CELERY_BIN:-"$(which celery||echo celery)"}
 CELERY_APP=${CELERY_APP:-"geonode.celery_app:app"}
-# CELERY__STATE_DB=${CELERY__STATE_DB:-"/mnt/volumes/statics/worker.state"}
+CELERY__STATE_DB=${CELERY__STATE_DB:-"/mnt/volumes/statics/worker.state"}
 # expressed in KB
 CELERY__MAX_MEMORY_PER_CHILD=${CELERY__MAX_MEMORY_PER_CHILD:-"200000"}
 CELERY__AUTOSCALE_VALUES=${CELERY__AUTOSCALE_VALUES:-"2,4"}
 CELERY__MAX_TASKS_PER_CHILD=${CELERY__MAX_TASKS_PER_CHILD:-"10"}
 CELERY__OPTS=${CELERY__OPTS:-"--without-gossip --without-mingle -Ofair -B -E"}
-CELERY__BEAT_SCHEDULE=${CELERY__BEAT_SCHEDULE:-"django_celery_beat.schedulers:DatabaseScheduler"}
+CELERY__BEAT_SCHEDULE=${CELERY__BEAT_SCHEDULE:-"celery.beat:PersistentScheduler"}
 CELERY__LOG_LEVEL=${CELERY__LOG_LEVEL:-"INFO"}
 CELERY__LOG_FILE=${CELERY__LOG_FILE:-"/var/log/celery.log"}
 CELERY__WORKER_NAME=${CELERY__WORKER_NAME:-"worker1@%h"}
 CELERY__WORKER_CONCURRENCY=${CELERY__WORKER_CONCURRENCY:-"4"}
 
 $CELERY_BIN -A $CELERY_APP worker --autoscale=$CELERY__AUTOSCALE_VALUES \
     --max-memory-per-child=$CELERY__MAX_MEMORY_PER_CHILD $CELERY__OPTS \
-    -s $CELERY__BEAT_SCHEDULE \
+    --statedb=$CELERY__STATE_DB --scheduler=$CELERY__BEAT_SCHEDULE \
     --loglevel=$CELERY__LOG_LEVEL -n $CELERY__WORKER_NAME -f $CELERY__LOG_FILE \
     --concurrency=$CELERY__WORKER_CONCURRENCY --max-tasks-per-child=$CELERY__MAX_TASKS_PER_CHILD
diff --git a/celery.sh b/celery.sh
@@ -1,4 +1,4 @@
 #!/bin/bash
 nohup celery -A geonode.celery_app:app beat -l DEBUG -f /var/log/celery.log &>/dev/null &
-nohup celery -A geonode.celery_app:app worker --without-gossip --without-mingle -Ofair -B -E -s django_celery_beat.schedulers:DatabaseScheduler --loglevel=INFO --concurrency=2 -n worker1@%h -f /var/log/celery.log &>/dev/null &
+nohup celery -A geonode.celery_app:app worker --without-gossip --without-mingle -Ofair -B -E --statedb=worker.state --scheduler=celery.beat:PersistentScheduler --loglevel=INFO --concurrency=2 -n worker1@%h -f /var/log/celery.log &>/dev/null &
 nohup celery -A geonode.celery_app:app flower --auto_refresh=True --debug=False --broker=${BROKER_URL} --basic_auth=${ADMIN_USERNAME}:${ADMIN_PASSWORD} --address=0.0.0.0 --port=5555 &>/dev/null &
diff --git a/celery_dev.sh b/celery_dev.sh
@@ -2,4 +2,4 @@ set -a
 . ./.env_dev
 set +a
 
-celery -A geonode.celery_app:app worker --without-gossip --without-mingle -Ofair -B -E -s django_celery_beat.schedulers:DatabaseScheduler --loglevel=DEBUG --concurrency=2 -n worker1@%h
+celery -A geonode.celery_app:app worker --without-gossip --without-mingle -Ofair -B -E --statedb=worker.state --scheduler=celery.beat:PersistentScheduler --loglevel=DEBUG --concurrency=2 -n worker1@%h
diff --git a/geonode/base/api/tests.py b/geonode/base/api/tests.py
@@ -1019,7 +1019,9 @@ def test_set_thumbnail_from_bbox_from_logged_user_for_existing_dataset(self, moc
         response = self.client.post(url, data=payload, format='json')
 
         expected = {
-            "thumbnail_url": "http://localhost:8000/mocked_url.jpg"
+            'message': 'Thumbnail correctly created.',
+            'success': True,
+            'thumbnail_url': 'http://localhost:8000/mocked_url.jpg'
         }
         self.assertEqual(response.status_code, 200)
         self.assertEqual(expected, response.json())
@@ -1044,7 +1046,8 @@ def test_set_thumbnail_from_bbox_from_logged_user_for_not_existing_dataset(self)
         response = self.client.post(url, data=payload, format='json')
 
         expected = {
-            "message": f"Resource selected with id {dataset_id} does not exists"
+            'message': f'Resource selected with id {dataset_id} does not exists',
+            'success': False
         }
         self.assertEqual(response.status_code, 404)
         self.assertEqual(expected, response.json())
@@ -1064,7 +1067,8 @@ def test_set_thumbnail_from_bbox_from_logged_user_for_existing_doc(self):
         response = self.client.post(url, data=payload, format='json')
 
         expected = {
-            "message": "Not implemented: Endpoint available only for Dataset and Maps"
+            'message': 'Not implemented: Endpoint available only for Dataset and Maps',
+            'success': False
         }
         self.assertEqual(response.status_code, 405)
         self.assertEqual(expected, response.json())
@@ -1085,7 +1089,8 @@ def test_set_thumbnail_from_bbox_from_logged_user_for_existing_dataset_raise_exp
         response = self.client.post(url, data=payload, format='json')
 
         expected = {
-            "message": "Some exception during thumb creation"
+            'message': 'Some exception during thumb creation',
+            'success': False
         }
         self.assertEqual(response.status_code, 500)
         self.assertEqual(expected, response.json())
diff --git a/geonode/base/api/views.py b/geonode/base/api/views.py
@@ -553,30 +553,44 @@ def resource_service_permissions(self, request, pk=None):
             IsAuthenticated,
         ])
     def set_thumbnail_from_bbox(self, request, resource_id):
+        import traceback
+        from django.utils.datastructures import MultiValueDictKeyError
         try:
             resource = ResourceBase.objects.get(id=ast.literal_eval(resource_id))
 
             if not isinstance(resource.get_real_instance(), (Dataset, Map)):
                 raise NotImplementedError("Not implemented: Endpoint available only for Dataset and Maps")
 
             request_body = request.data if request.data else json.loads(request.body)
-            bbox = request_body["bbox"] + [request_body["srid"]]
-            zoom = request_body.get("zoom", None)
+            try:
+                bbox = request_body["bbox"] + [request_body["srid"]]
+                zoom = request_body.get("zoom", None)
+            except MultiValueDictKeyError:
+                for _k, _v in request_body.items():
+                    request_body = json.loads(_k)
+                    break
+                bbox = request_body["bbox"] + [request_body["srid"]]
+                zoom = request_body.get("zoom", None)
 
             thumbnail_url = create_thumbnail(resource.get_real_instance(), bbox=bbox, background_zoom=zoom, overwrite=True)
-            return Response({"thumbnail_url": thumbnail_url}, status=200)
+            return Response({"message": "Thumbnail correctly created.", "success": True, "thumbnail_url": thumbnail_url}, status=200)
         except ResourceBase.DoesNotExist:
+            traceback.print_exc()
             logger.error(f"Resource selected with id {resource_id} does not exists")
-            return Response(data={"message": f"Resource selected with id {resource_id} does not exists"}, status=404, exception=True)
+            return Response(
+                data={"message": f"Resource selected with id {resource_id} does not exists", "success": False}, status=404, exception=True)
         except NotImplementedError as e:
+            traceback.print_exc()
             logger.error(e)
-            return Response(data={"message": e.args[0]}, status=405, exception=True)
+            return Response(data={"message": e.args[0], "success": False}, status=405, exception=True)
         except ThumbnailError as e:
+            traceback.print_exc()
             logger.error(e)
-            return Response(data={"message": e.args[0]}, status=500, exception=True)
+            return Response(data={"message": e.args[0], "success": False}, status=500, exception=True)
         except Exception as e:
+            traceback.print_exc()
             logger.error(e)
-            return Response(data={"message": e.args[0]}, status=500, exception=True)
+            return Response(data={"message": e.args[0], "success": False}, status=500, exception=True)
 
     @extend_schema(
         methods=["post"], responses={200}, description="Instructs the Async dispatcher to execute a 'INGEST' operation."

diff --git a/geonode/base/models.py b/geonode/base/models.py
@@ -1022,10 +1022,13 @@ class Meta:
 
     def __init__(self, *args, **kwargs):
         # Provide legacy support for bbox fields
-        bbox = [kwargs.pop(key, None) for key in ('bbox_x0', 'bbox_y0', 'bbox_x1', 'bbox_y1')]
-        if all(bbox):
-            kwargs['bbox_polygon'] = Polygon.from_bbox(bbox)
-            kwargs['ll_bbox_polygon'] = Polygon.from_bbox(bbox)
+        try:
+            bbox = [kwargs.pop(key, None) for key in ('bbox_x0', 'bbox_y0', 'bbox_x1', 'bbox_y1')]
+            if all(bbox):
+                kwargs['bbox_polygon'] = Polygon.from_bbox(bbox)
+                kwargs['ll_bbox_polygon'] = Polygon.from_bbox(bbox)
+        except Exception as e:
+            logger.exception(e)
         super().__init__(*args, **kwargs)
 
     def __str__(self):
@@ -1507,17 +1510,18 @@ def set_center_zoom(self):
         """
         Sets the center coordinates and zoom level in EPSG:4326
         """
-        bbox = self.ll_bbox_polygon
-        center_x, center_y = self.ll_bbox_polygon.centroid.coords
-        center = Point(center_x, center_y, srid=4326)
-        self.center_x, self.center_y = center.coords
-        try:
-            ext = bbox.extent
-            width_zoom = math.log(360 / (ext[2] - ext[0]), 2)
-            height_zoom = math.log(360 / (ext[3] - ext[1]), 2)
-            self.zoom = math.ceil(min(width_zoom, height_zoom))
-        except ZeroDivisionError:
-            pass
+        if self.ll_bbox_polygon and len(self.ll_bbox_polygon.centroid.coords) > 0:
+            bbox = self.ll_bbox_polygon.clone()
+            center_x, center_y = bbox.centroid.coords
+            center = Point(center_x, center_y, srid=4326)
+            self.center_x, self.center_y = center.coords
+            try:
+                ext = bbox.extent
+                width_zoom = math.log(360 / (ext[2] - ext[0]), 2)
+                height_zoom = math.log(360 / (ext[3] - ext[1]), 2)
+                self.zoom = math.ceil(min(width_zoom, height_zoom))
+            except ZeroDivisionError:
+                pass
 
     def download_links(self):
         """assemble download links for pycsw"""

diff --git a/geonode/base/templatetags/base_tags.py b/geonode/base/templatetags/base_tags.py
@@ -29,12 +29,12 @@
 from pinax.ratings.models import Rating
 from guardian.shortcuts import get_objects_for_user
 
-from geonode.base.models import ResourceBase
-from geonode.base.bbox_utils import filter_bbox
-from geonode.layers.models import Dataset
 from geonode.maps.models import Map
+from geonode.layers.models import Dataset
+from geonode.base.models import ResourceBase
 from geonode.documents.models import Document
 from geonode.groups.models import GroupProfile
+from geonode.base.bbox_utils import filter_bbox
 from geonode.base.models import (
     HierarchicalKeyword, Menu, MenuItem
 )
@@ -423,11 +423,3 @@ def display_change_perms_button(resource, user, perms):
         return True
     else:
         return not getattr(settings, 'ADMIN_MODERATE_UPLOADS', False)
-
-
-@register.simple_tag
-def get_dataset_count_by_services(service_id, user):
-    return get_visible_resources(
-        queryset=Dataset.objects.filter(remote_service=service_id),
-        user=user
-    ).count()
diff --git a/geonode/geoserver/helpers.py b/geonode/geoserver/helpers.py
@@ -1483,12 +1483,11 @@ def fetch_gs_resource(instance, values, tries):
                            owner=instance.owner))
     else:
         msg = f"There isn't a geoserver resource for this layer: {instance.name}"
-        logger.exception(msg)
+        logger.debug(msg)
         if tries >= _max_tries:
             # raise GeoNodeException(msg)
             return (values, None)
         gs_resource = None
-        time.sleep(5)
     return (values, gs_resource)
 
 
@@ -1683,20 +1682,6 @@ def sync_instance_with_geoserver(
             except Exception as e:
                 logger.warning(e)
 
-        # Refreshing CSW records
-        logger.debug(f"... Updating the Catalogue entries for Dataset {instance.title}")
-        try:
-            catalogue_post_save(instance=instance, sender=instance.__class__)
-        except Exception as e:
-            logger.exception(e)
-
-        # Refreshing dataset links
-        logger.debug(f"... Creating Default Resource Links for Dataset {instance.title}")
-        try:
-            set_resource_default_links(instance, instance, prune=True)
-        except Exception as e:
-            logger.exception(e)
-
         # Save dataset styles
         logger.debug(f"... Refresh Legend links for Dataset {instance.title}")
         try:
@@ -1711,6 +1696,21 @@ def sync_instance_with_geoserver(
         except Exception:
             pass
 
+    # Refreshing dataset links
+    logger.debug(f"... Creating Default Resource Links for Dataset {instance.title}")
+    try:
+        _prune = (gs_resource is not None)
+        set_resource_default_links(instance, instance, prune=_prune)
+    except Exception as e:
+        logger.exception(e)
+
+    # Refreshing CSW records
+    logger.debug(f"... Updating the Catalogue entries for Dataset {instance.title}")
+    try:
+        catalogue_post_save(instance=instance, sender=instance.__class__)
+    except Exception as e:
+        logger.exception(e)
+
     return instance
 
 

diff --git a/geonode/geoserver/manager.py b/geonode/geoserver/manager.py
@@ -115,7 +115,7 @@ def delete(self, uuid: str, /, instance: ResourceBase = None) -> int:
         if instance and getattr(ogc_server_settings, "BACKEND_WRITE_ENABLED", True):
             _real_instance = instance.get_real_instance()
             if isinstance(_real_instance, Dataset) and hasattr(_real_instance, 'alternate') and _real_instance.alternate:
-                if _real_instance.remote_service is None or _real_instance.remote_service.method == CASCADED:
+                if not hasattr(_real_instance, 'remote_service') or _real_instance.remote_service is None or _real_instance.remote_service.method == CASCADED:
                     geoserver_cascading_delete.apply_async((_real_instance.alternate,))
                     if "geonode.upload" in settings.INSTALLED_APPS:
                         from geonode.upload.models import Upload
@@ -555,7 +555,7 @@ def set_workflow_permissions(self, uuid: str, /, instance: ResourceBase = None,
     def set_thumbnail(self, uuid: str, /, instance: ResourceBase = None, overwrite: bool = True, check_bbox: bool = True) -> bool:
         if instance and not isinstance(instance.get_real_instance(), Document):
             if overwrite or instance.thumbnail_url == static(settings.MISSING_THUMBNAIL):
-                geoserver_create_thumbnail.apply_async((instance.id, overwrite, check_bbox, ))
+                geoserver_create_thumbnail.apply((instance.id, overwrite, check_bbox, ))
             return True
         return False
 

diff --git a/geonode/geoserver/signals.py b/geonode/geoserver/signals.py
@@ -54,7 +54,7 @@ def geoserver_pre_delete(instance, sender, **kwargs):
     # cascading_delete should only be called if
     # ogc_server_settings.BACKEND_WRITE_ENABLED == True
     if getattr(ogc_server_settings, "BACKEND_WRITE_ENABLED", True):
-        if instance.remote_service is None or instance.remote_service.method == CASCADED:
+        if not hasattr(instance, 'remote_service') or instance.remote_service is None or instance.remote_service.method == CASCADED:
             if instance.alternate:
                 geoserver_cascading_delete.apply_async((instance.alternate,))
 

diff --git a/geonode/geoserver/tasks.py b/geonode/geoserver/tasks.py
@@ -29,6 +29,7 @@
     AcquireLock,
     FaultTolerantTask)
 from geonode.base.models import Link
+from geonode.base import enumerations
 from geonode.layers.models import Dataset
 from geonode.base.models import ResourceBase
 
@@ -217,7 +218,7 @@ def geoserver_post_save_datasets(
     expires=30,
     acks_late=False,
     autoretry_for=(Exception, ),
-    retry_kwargs={'max_retries': 3, 'countdown': 10},
+    retry_kwargs={'max_retries': 1, 'countdown': 10},
     retry_backoff=True,
     retry_backoff_max=700,
     retry_jitter=True)
@@ -235,11 +236,15 @@ def geoserver_create_thumbnail(self, instance_id, overwrite=True, check_bbox=Tru
     lock_id = f'{self.request.id}'
     with AcquireLock(lock_id) as lock:
         if lock.acquire() is True:
+            instance.set_processing_state(enumerations.STATE_RUNNING)
             try:
+                instance.set_dirty_state()
                 create_gs_thumbnail(instance, overwrite=overwrite, check_bbox=check_bbox)
                 logger.debug(f"... Created Thumbnail for Dataset {instance.title}")
             except Exception as e:
                 geoserver_create_thumbnail.retry(exc=e)
+            finally:
+                instance.set_processing_state(enumerations.STATE_PROCESSED)
 
 
 @app.task(

diff --git a/geonode/groups/tests.py b/geonode/groups/tests.py
@@ -276,7 +276,7 @@ def test_group_permissions_extend_to_user(self):
         Ensures that when a user is in a group, the group permissions
         extend to the user.
         """
-        layer = Dataset.objects.all()[0]
+        layer = Dataset.objects.first()
         # Set the default permissions
         layer.set_default_permissions()
 

diff --git a/geonode/harvesting/admin.py b/geonode/harvesting/admin.py
@@ -307,13 +307,15 @@ class HarvestableResourceAdmin(admin.ModelAdmin):
         "last_harvested",
         "unique_identifier",
         "title",
+        "abstract",
         "show_link_to_harvester",
         "should_be_harvested",
         "remote_resource_type",
     )
     readonly_fields = (
         "unique_identifier",
         "title",
+        "abstract",
         "harvester",
         "last_updated",
         "last_refreshed",
@@ -331,6 +333,7 @@ class HarvestableResourceAdmin(admin.ModelAdmin):
     )
     search_fields = (
         "title",
+        "abstract",
     )
     list_editable = (
         "should_be_harvested",