Merge pull request 'Master' (#553) from master into Docker

Reviewed-on: #553

Dockerfile

@@ -0,0 +1,36 @@
+# Nutze ein schlankes Python-Image
+FROM python:3.13-slim-bullseye
+
+ENV PYTHONUNBUFFERED 1
+
+WORKDIR /konova
+
+# Installiere System-Abhängigkeiten
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    gdal-bin redis-server nginx \
+    && rm -rf /var/lib/apt/lists/*  # Platz sparen
+
+# Erstelle benötigte Verzeichnisse & setze Berechtigungen
+RUN mkdir -p /var/log/nginx /var/log/gunicorn /var/lib/nginx /tmp/nginx_client_body \
+    && touch /var/log/nginx/access.log /var/log/nginx/error.log \
+    && chown -R root:root /var/log/nginx /var/lib/nginx /tmp/nginx_client_body
+
+# Kopiere und installiere Python-Abhängigkeiten
+COPY ./requirements.txt /konova/
+RUN pip install --upgrade pip && pip install --no-cache-dir -r requirements.txt
+
+# Entferne Standard-Nginx-Site und ersetze sie durch eigene Config
+RUN rm -rf /etc/nginx/sites-enabled/default
+COPY ./nginx.conf /etc/nginx/conf.d
+
+# Kopiere restliche Projektdateien
+COPY . /konova/
+
+# Sammle statische Dateien
+RUN python manage.py collectstatic --noinput
+
+# Exponiere Ports
+#EXPOSE 80 6379 8000
+
+# Setze Entrypoint
+ENTRYPOINT ["/konova/docker-entrypoint.sh"]

README.md

@@ -4,6 +4,7 @@ the database postgresql and the css library bootstrap as well as the icon packag
 fontawesome for a modern look, following best practices from the industry.
 
 ## Background processes
+### !!! For non-docker run
 Konova uses celery for background processing. To start the worker you need to run
 ```shell
 $ celery -A konova worker -l INFO
@@ -18,3 +19,58 @@ Technical documention is provided in the projects git wiki.
 A user documentation is not available (and not needed, yet). 
 
 
+# Docker 
+To run the docker-compose as expected, you need to take the following steps:
+
+1. Create a database containing docker, using an appropriate Dockerfile, e.g. the following
+```
+version: '3.3'
+services:
+  postgis:
+    image: postgis/postgis
+    restart: always
+    container_name: postgis-docker
+    ports: 
+      - 5433:5432
+    volumes:
+      - db-volume:/var/lib/postgresql/data
+    environment:
+      - POSTGRES_PASSWORD=postgres
+      - POSTGRES_USER=postgres
+    networks:
+      - db-network-bridge
+
+networks:
+  db-network-bridge:
+    driver: "bridge"
+
+volumes:
+  db-volume:
+```
+This Dockerfile creates a Docker container running postgresql and postgis, creates the default superuser postgres, 
+creates a named volume for persisting the database and creates a new network bridge, which **must be used by any other
+container, which wants to write/read on this database**. 
+
+2. Make sure the name of the network bridge above matches the network in the konova docker-compose.yml  
+3. Get into the running postgis container (`docker exec -it postgis-docker bash`) and create new databases, users and so on. Make sure the database `konova` exists now!
+4. Replace all `CHANGE_ME_xy` values inside of konova/docker-compose.yml for your installation. Make sure the `SSO_HOST` holds the proper SSO host, e.g. for the arnova project `arnova.example.org` (Arnova must be installed and the webserver configured as well, of course)
+5. Take a look on konova/settings.py and konova/sub_settings/django_settings.py. Again: Replace all occurences of `CHANGE_ME` with proper values for your installation.
+   1. Make sure you have the proper host strings added to `ALLOWED_HOSTS` inside of django_settings.py.  
+6. Build and run the docker setup using `docker-compose build` and `docker-compose start` from the main directory of this project (where the docker-compose.yml lives)
+7. Run migrations! To do so, get into the konova service container (`docker exec -it konova-docker bash`) and run the needed commands (`python manage.py makemigrations LIST_OF_ALL_MIGRATABLE_APPS`, then `python manage.py migrate`)
+8. Run the setup command `python manage.py setup` and follow the instructions on the CLI
+9. To enable **SMTP** mail support, make sure your host machine (the one where the docker container run) has the postfix service configured properly. Make sure the `mynetworks` variable is xtended using the docker network bridge ip, created in the postgis container and used by the konova services.
+   1. **Hint**: You can find out this easily by trying to perform a test mail in the running konova web application (which will fail, of course). Then take a look to the latest entries in `/var/log/mail.log` on your host machine. The failed IP will be displayed there.
+   2. **Please note**: This installation guide is based on SMTP using postfix! 
+   3. Restart the postfix service on your host machine to reload the new configuration (`service postfix restart`)
+10. Finally, make sure your host machine webserver passes incoming requests properly to the docker nginx webserver of konova. A proper nginx config for the host machine may look like this:
+```
+server {
+    server_name konova.domain.org;
+
+    location / {
+        proxy_pass http://localhost:KONOVA_NGINX_DOCKER_PORT/;
+        proxy_set_header Host $host;
+    }
+}
+```

api/models/external_identifier.py

@@ -30,4 +30,22 @@ class ExternalIdentifier(models.Model):
     )
 
     def __str__(self):
-        return f"{self.external_id} -> {self.internal_id}"
+        return f"{self.external_id} -> {self.internal_id}"
+
+    @staticmethod
+    def resolve_external_identifier(external_identifier: str):
+        """ Returns a ExternalIdentifier object, if the given parameter could be resolved as an external identifier.
+
+        Args:
+            external_identifier (str): An external identifier.
+
+        Returns:
+            ExternalIdentifier | None
+        """
+        if external_identifier:
+            try:
+                obj = ExternalIdentifier.objects.get(external_id=external_identifier)
+                return obj
+            except ExternalIdentifier.DoesNotExist:
+                pass
+        return None

api/tests/v1/share/test_api_sharing.py

@@ -31,9 +31,10 @@ class APIV1SharingTestCase(BaseAPIV1TestCase):
     def setUpTestData(cls):
         super().setUpTestData()
 
-    def _run_share_request(self, url, user_list: list):
+    def _run_share_request(self, url, user_list: list, team_list: list):
         data = {
-            "users": user_list
+            "users": user_list,
+            "teams": team_list
         }
         data = json.dumps(data)
         response = self.client.put(
@@ -58,16 +59,22 @@ class APIV1SharingTestCase(BaseAPIV1TestCase):
             self.superuser.username,
             self.user.username,
         ]
+        team_list = [
+            str(self.team.id),
+        ]
 
-        response = self._run_share_request(url, user_list)
+        response = self._run_share_request(url, user_list, team_list)
 
         # Must fail, since performing user has no access on requested object
         self.assertEqual(response.status_code, 500)
         self.assertTrue(len(json.loads(response.content.decode("utf-8")).get("errors", [])) > 0)
 
-        # Add performing user to shared access users and rerun the request
+        # Add performing user to shared access users, switch from team id to team name and rerun the request
         obj.users.add(self.superuser)
-        response = self._run_share_request(url, user_list)
+        team_list = [
+            self.team.name
+        ]
+        response = self._run_share_request(url, user_list, team_list)
 
         shared_users = obj.shared_users
         self.assertEqual(response.status_code, 200)
@@ -84,14 +91,14 @@ class APIV1SharingTestCase(BaseAPIV1TestCase):
         share_url = reverse("api:v1:intervention-share", args=(self.intervention.id,))
         # Expect the first request to work properly
         self.intervention.users.add(self.superuser)
-        response = self._run_share_request(share_url, [self.superuser.username])
+        response = self._run_share_request(share_url, [self.superuser.username], [str(self.team.id)])
         self.assertEqual(response.status_code, 200)
 
         # Change the token
         self.header_data["HTTP_ksptoken"] = f"{self.superuser.api_token.token}__X"
 
         # Expect the request to fail now
-        response = self._run_share_request(share_url, [self.superuser.username])
+        response = self._run_share_request(share_url, [self.superuser.username], [])
         self.assertEqual(response.status_code, 403)
 
     def test_api_intervention_sharing(self):
@@ -144,11 +151,11 @@ class APIV1SharingTestCase(BaseAPIV1TestCase):
         self.assertEqual(self.intervention.users.count(), 1)
 
         # Try to add another user via API -> must work!
-        response = self._run_share_request(share_url, [self.superuser.username, self.user.username])
+        response = self._run_share_request(share_url, [self.superuser.username, self.user.username], [])
         self.assertEqual(response.status_code, 200)
         self.assertEqual(self.intervention.users.count(), 2)
 
         # Now try to remove the user again -> expect no changes at all to the shared user list
-        response = self._run_share_request(share_url, [self.superuser.username])
+        response = self._run_share_request(share_url, [self.superuser.username], [])
         self.assertEqual(response.status_code, 200)
         self.assertEqual(self.intervention.users.count(), 2)

api/utils/serializer/serializer.py

@@ -202,3 +202,33 @@ class AbstractModelAPISerializer:
             obj (Intervention)
         """
         raise NotImplementedError("Must be implemented in subclasses")
+
+
+    def _geometry_conflicts_to_list(self, geometry) -> list:
+        """ Serializes geometry conflict ids into dict
+
+        Args:
+            geometry (Geometry): The geometry to fetch geometry conflicts from
+
+        Returns:
+            ids (list): Serialized geometry conflicts as dict objects inside a list
+        """
+        ids = []
+        conflict_geometries = geometry.get_conflict_geometries()
+        for geom in conflict_geometries:
+            try:
+                data = geom.get_data_objects(["identifier", "id"])
+                if len(data) == 0:
+                    # expected behaviour in case of deleted data object
+                    continue
+                data = data[0]
+            except KeyError:
+                raise AssertionError(f"Geometry {geom.id} is not attached to any entries. Contact an admin!")
+            ids.append(
+                {
+                    "identifier": data["identifier"],
+                    "id": data["id"],
+                }
+            )
+
+        return ids

api/utils/serializer/v1/serializer.py

@@ -54,6 +54,7 @@ class AbstractModelAPISerializerV1(AbstractModelAPISerializer):
             "created_on": self._created_on_to_json(entry),
             "modified_on": self._modified_on_to_json(entry),
             "external_identifiers": ext_ids,
+            "geometry_conflicts": self._geometry_conflicts_to_list(entry.geometry)
         }
         self._extend_properties_data(entry)
         geo_json["properties"] = self.properties_data
@@ -179,13 +180,7 @@ class AbstractModelAPISerializerV1(AbstractModelAPISerializer):
         Returns:
             ExternalIdentifier | None
         """
-        if external_identifier:
-            try:
-                obj = ExternalIdentifier.objects.get(external_id=external_identifier)
-                return obj
-            except ObjectDoesNotExist:
-                pass
-        return None
+        return ExternalIdentifier.resolve_external_identifier(external_identifier)
 
     def _check_external_identifier_on_entry_creation(self, external_identifier):
         """ Special check for POST processing:

api/views/views.py

@@ -6,13 +6,14 @@ Created on: 21.01.22
 
 """
 import json
+import uuid
 
-from django.db.models import QuerySet
+from django.db.models import QuerySet, Q
 from django.http import JsonResponse, HttpRequest
 from django.views import View
 from django.views.decorators.csrf import csrf_exempt
 
-from api.models import APIUserToken
+from api.models import APIUserToken, ExternalIdentifier
 from api.settings import KSP_TOKEN_HEADER_IDENTIFIER, KSP_USER_HEADER_IDENTIFIER
 from compensation.models import EcoAccount
 from ema.models import Ema
@@ -205,6 +206,10 @@ class AbstractModelShareAPIView(AbstractAPIView):
 
         """
         try:
+            external_identifier = ExternalIdentifier.resolve_external_identifier(id)
+            if external_identifier:
+                id = external_identifier.internal_id
+
             users = self._get_shared_users_of_object(id)
             teams = self._get_shared_teams_of_object(id)
         except Exception as e:
@@ -237,6 +242,9 @@ class AbstractModelShareAPIView(AbstractAPIView):
         """
 
         try:
+            external_identifier = ExternalIdentifier.resolve_external_identifier(id)
+            if external_identifier:
+                id = external_identifier.internal_id
             success = self._process_put_body(request.body, id)
         except Exception as e:
             return self._return_error_response(e)
@@ -309,22 +317,36 @@ class AbstractModelShareAPIView(AbstractAPIView):
             raise ValueError("Shared user list must not be empty!")
         new_teams = content.get("teams", [])
 
+        self.__process_user_sharing(new_users, obj)
+        self.__process_team_sharing(new_teams, obj)
+
+        return True
+
+    def __process_user_sharing(self, user_list: list, obj):
+        """ Processes API sharing for user payload
+
+        Args:
+            user_list (list): A list of users to share the obj with
+            obj (BaseObject): The shareable object
+
+        Returns:
+
+        """
         # Eliminate duplicates
-        new_users = list(dict.fromkeys(new_users))
-        new_teams = list(dict.fromkeys(new_teams))
+        new_users = list(dict.fromkeys(user_list))
 
         # Make sure each of these names exist as a user
         new_users_objs = []
         for user in new_users:
-            new_users_objs.append(User.objects.get(username=user))
-
-        # Make sure each of these names exist as a user
-        new_teams_objs = []
-        for team_name in new_teams:
-            new_teams_objs.append(Team.objects.get(name=team_name))
+            try:
+                user_obj = User.objects.get(username=user)
+            except User.DoesNotExist:
+                raise AssertionError(f"User with username {user} does not exist")
+            new_users_objs.append(user_obj)
 
         if self.user.is_default_group_only():
             # Default only users are not allowed to remove other users from having access. They can only add new ones!
+            # So we need to keep the ones that already have access from being removed!
             new_users_to_be_added = User.objects.filter(
                 username__in=new_users
             ).exclude(
@@ -332,17 +354,44 @@ class AbstractModelShareAPIView(AbstractAPIView):
             )
             new_users_objs = obj.shared_users.union(new_users_to_be_added)
 
-            new_teams_to_be_added = Team.objects.filter(
-                name__in=new_teams
-            ).exclude(
-                id__in=obj.shared_teams
-            )
-            new_teams_objs = obj.shared_teams.union(new_teams_to_be_added)
-
         obj.share_with_user_list(new_users_objs)
-        obj.share_with_team_list(new_teams_objs)
-        return True
 
+    def __process_team_sharing(self, team_list: list, obj):
+        """ Processes API sharing for team payload
+
+        Args:
+            team_list (list): A list of teams to share the obj with
+            obj (BaseObject): The shareable object
+
+        Returns:
+
+        """
+        # Eliminate duplicates
+        new_teams = list(dict.fromkeys(team_list))
+
+        # Resolve team names or ids into objects
+        new_team_ids = []
+        for team in new_teams:
+            try:
+                uuid.UUID(team)
+                try:
+                    new_team_ids.append(Team.objects.get(id=team).id)
+                except Team.DoesNotExist:
+                    raise AssertionError(f"Team with id {team} does not exist!")
+            except ValueError:
+                # entry is a name and not a uuid -> try to resolve as name!
+                try:
+                    new_team_ids.append(Team.objects.get(name=team).id)
+                except Team.DoesNotExist:
+                    raise AssertionError(f"Team with name {team} does not exist!")
+
+        new_team_objs = Team.objects.filter(id__in=new_team_ids)
+        if self.user.is_default_group_only():
+            # Default only users are not allowed to remove other users from having access. They can only add new ones!
+            # So we need to keep the ones that already have access from being removed!
+            new_team_objs = obj.shared_teams.union(new_team_objs)
+
+        obj.share_with_team_list(new_team_objs)
 
 class InterventionAPIShareView(AbstractModelShareAPIView):
     model = Intervention

docker-compose.yml

@@ -0,0 +1,21 @@
+services:
+  konova:
+    external_links:
+      - postgis:db
+      - arnova-nginx-server:arnova
+    build: .
+    image: "ksp/konova:x.y"
+    container_name: "konova-docker"
+    command: ./docker-entrypoint.sh
+    restart: always
+    volumes:
+      - /data/apps/konova/uploaded_files:/konova_uploaded_files
+    ports:
+      - "1337:80"
+
+# Instead of an own, new network, we need to connect to the existing one, which is provided by the postgis container
+# NOTE: THIS NETWORK MUST EXIST
+networks:
+  default:
+    name: postgis_nat_it_backend
+    external: true

docker-entrypoint.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+set -e  # Beende Skript bei Fehlern
+set -o pipefail  # Fehler in Pipelines nicht ignorieren
+
+# Starte Redis
+redis-server --daemonize yes
+
+# Starte Celery Worker im Hintergrund
+celery -A konova worker --loglevel=info &
+
+# Starte Nginx als Hintergrundprozess
+nginx -g "daemon off;" &
+
+# Setze Gunicorn Worker-Anzahl (Standard: (2*CPUs)+1)
+WORKERS=${GUNICORN_WORKERS:-$((2 * $(nproc) + 1))}
+
+# Stelle sicher, dass Logs existieren
+mkdir -p /var/log/gunicorn
+touch /var/log/gunicorn/access.log /var/log/gunicorn/error.log
+
+# Starte Gunicorn als Hauptprozess
+exec gunicorn --workers="$WORKERS" konova.wsgi:application \
+    --bind=0.0.0.0:8000 \
+    --access-logfile /var/log/gunicorn/access.log \
+    --error-logfile /var/log/gunicorn/error.log \
+    --access-logformat '%({x-real-ip}i)s via %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

intervention/tests/test_views.py

@@ -36,6 +36,7 @@ class InterventionViewTestCase(BaseViewTestCase):
         self.run_check_url = reverse("intervention:check", args=(self.intervention.id,))
         self.record_url = reverse("intervention:record", args=(self.intervention.id,))
         self.report_url = reverse("intervention:report", args=(self.intervention.id,))
+        self.compensation_remove_url = reverse("intervention:remove-compensation", args=(self.compensation.intervention.id, self.compensation.id))
 
         self.deduction.intervention = self.intervention
         self.deduction.save()
@@ -83,6 +84,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.revocation_new_url: f"{login_redirect_base}{self.revocation_new_url}",
             self.revocation_edit_url: f"{login_redirect_base}{self.revocation_edit_url}",
             self.revocation_remove_url: f"{login_redirect_base}{self.revocation_remove_url}",
+            self.compensation_remove_url: f"{login_redirect_base}{self.compensation_remove_url}",
         }
 
         self.assert_url_success(client, success_urls)
@@ -124,6 +126,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.deduction_new_url,
             self.deduction_edit_url,
             self.deduction_remove_url,
+            self.compensation_remove_url,
         ]
 
         self.assert_url_success(client, success_urls)
@@ -162,6 +165,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.deduction_new_url,
             self.deduction_edit_url,
             self.deduction_remove_url,
+            self.compensation_remove_url,
         ]
         fail_urls = [
             self.run_check_url,
@@ -212,6 +216,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.deduction_new_url,
             self.deduction_edit_url,
             self.deduction_remove_url,
+            self.compensation_remove_url,
         ]
         success_urls_redirect = {
             self.share_url: self.detail_url
@@ -258,6 +263,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.deduction_new_url,
             self.deduction_edit_url,
             self.deduction_remove_url,
+            self.compensation_remove_url,
         ]
         success_urls_redirect = {
             self.share_url: self.detail_url
@@ -304,6 +310,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.deduction_new_url,
             self.deduction_edit_url,
             self.deduction_remove_url,
+            self.compensation_remove_url,
         ]
         success_urls_redirect = {
             self.share_url: self.detail_url
@@ -350,6 +357,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.deduction_new_url,
             self.deduction_edit_url,
             self.deduction_remove_url,
+            self.compensation_remove_url,
         ]
         success_urls_redirect = {
             self.share_url: self.detail_url
@@ -396,6 +404,7 @@ class InterventionViewTestCase(BaseViewTestCase):
             self.deduction_new_url,
             self.deduction_edit_url,
             self.deduction_remove_url,
+            self.compensation_remove_url,
         ]
         # Define urls where a redirect to a specific location is the proper response
         success_urls_redirect = {

intervention/views/compensation.py

@@ -14,7 +14,7 @@ from django.utils.decorators import method_decorator
 from django.views import View
 
 from intervention.models import Intervention
-from konova.decorators import shared_access_required
+from konova.decorators import shared_access_required, default_group_required
 from konova.forms.modals import RemoveModalForm
 from konova.utils.message_templates import COMPENSATION_REMOVED_TEMPLATE
 
@@ -45,10 +45,12 @@ class RemoveCompensationFromInterventionView(LoginRequiredMixin, View):
             redirect_url=reverse("intervention:detail", args=(id,)) + "#related_data",
         )
 
+    @method_decorator(default_group_required)
     @method_decorator(shared_access_required(Intervention, "id"))
     def get(self, request, id: str, comp_id: str, *args, **kwargs) -> HttpResponse:
         return self.__process_request(request, id, comp_id, *args, **kwargs)
 
+    @method_decorator(default_group_required)
     @method_decorator(shared_access_required(Intervention, "id"))
     def post(self, request, id: str, comp_id: str, *args, **kwargs) -> HttpResponse:
         return self.__process_request(request, id, comp_id, *args, **kwargs)

konova/models/geometry.py

@@ -113,7 +113,7 @@ class Geometry(BaseResource):
             objs (list): The list of objects
         """
         objs = []
-
+        stop_searching = False
         # Some related data sets can be processed rather easily
         regular_sets = [
             self.intervention_set,
@@ -125,28 +125,44 @@ class Geometry(BaseResource):
                 deleted=None
             )
             if limit_to_attrs:
-                objs += set_objs.values_list(*limit_to_attrs, flat=True)
+                objs += set_objs.values(*limit_to_attrs)
             else:
                 objs += set_objs
 
-        # ... but we need a special treatment for compensations, since they can be deleted directly OR inherit their
-        # de-facto-deleted status from their deleted parent intervention
-        comp_objs = self.compensation_set.filter(
-            Q(deleted=None) & Q(intervention__deleted=None)
-        )
-        if limit_to_attrs:
-            objs += comp_objs.values_list(*limit_to_attrs, flat=True)
-        else:
-            objs += comp_objs
+            stop_searching = len(objs) > 0
+            if stop_searching:
+                break
+
+        if not stop_searching:
+            # ... but we need a special treatment for compensations, since they can be deleted directly OR inherit their
+            # de-facto-deleted status from their deleted parent intervention
+            comp_objs = self.compensation_set.filter(
+                Q(deleted=None) | Q(intervention__deleted=None)
+            )
+            if limit_to_attrs:
+                objs += comp_objs.values(*limit_to_attrs)
+            else:
+                objs += comp_objs
         return objs
 
-    def get_data_object(self):
+    def get_data_object(self, limit_to_attrs: list = None):
         """
-        Getter for the specific data object which is related to this geometry
+        Getter for the specific data object which is related to this geometry.
+
+        !!! Only returns undeleted entries !!!
+
+        Returns:
+            result (str|None): Returns the desired attributes or None if the data object is marked as deleted
+
         """
-        objs = self.get_data_objects()
+        objs = self.get_data_objects(limit_to_attrs)
         assert (len(objs) <= 1)
-        result = objs.pop()
+        try:
+            result = objs.pop()
+        except IndexError:
+            # If this happens, we just processed a GeometryConflict with an entry which is marked as deleted.
+            # Therefore we return None
+            result = None
         return result
 
     def update_parcels(self):
@@ -436,6 +452,16 @@ class Geometry(BaseResource):
             output_geom.transform(DEFAULT_SRID_RLP)
         return output_geom
 
+    def get_conflict_geometries(self):
+        """ Getter for geometry ids which conflict with this geometry or are conflicted by this one
+
+        Returns:
+            geom_ids (list): List of geometry ids
+        """
+        conflict_geoms_id = GeometryConflict.get_conflict_geometries(self)
+        conflict_geoms = Geometry.objects.filter(id__in=conflict_geoms_id)
+        return conflict_geoms
+
 
 class GeometryConflict(UuidModel):
     """
@@ -459,3 +485,23 @@ class GeometryConflict(UuidModel):
 
     def __str__(self):
         return f"{self.conflicting_geometry.id} conflicts with {self.affected_geometry.id}"
+
+    @staticmethod
+    def get_conflict_geometries(geometry: Geometry):
+        """ Getter for geometries which conflict in one or another way with the given one
+
+        Args:
+            geometry (Geometry): The geometry which shall be checked
+
+        Returns:
+            conflict_geometries (QuerySet): QuerySet of geometries which have conflicts with the given geometry
+        """
+        conflict_geometries = GeometryConflict.objects.filter(
+            affected_geometry=geometry.id,
+        ).values_list("conflicting_geometry__id", flat=True)
+        conflict_geometries = conflict_geometries.union(
+            GeometryConflict.objects.filter(
+                conflicting_geometry=geometry.id,
+            ).values_list("affected_geometry__id", flat=True)
+        )
+        return conflict_geometries

konova/models/object.py

@@ -676,24 +676,17 @@ class GeoReferencedMixin(models.Model):
         if self.geometry is None:
             return request
 
-        instance_objs = []
-        needed_data_object_attrs = [
-            "identifier"
-        ]
-        conflicts = self.geometry.conflicts_geometries.iterator()
+        conflicting_geometries = self.geometry.get_conflict_geometries()
+        data_object_identifiers = []
+        for conflicting_geom in conflicting_geometries:
+            data_obj_id = conflicting_geom.get_data_object(["identifier"])
+            if data_obj_id:
+                data_object_identifiers.append(data_obj_id)
 
-        for conflict in conflicts:
-            # Only check the affected geometry of this conflict, since we know the conflicting geometry is self.geometry
-            instance_objs += conflict.affected_geometry.get_data_objects(needed_data_object_attrs)
-
-        conflicts = self.geometry.conflicted_by_geometries.iterator()
-        for conflict in conflicts:
-            # Only check the conflicting geometry of this conflict, since we know the affected geometry is self.geometry
-            instance_objs += conflict.conflicting_geometry.get_data_objects(needed_data_object_attrs)
-
-        add_message = len(instance_objs) > 0
+        add_message = len(data_object_identifiers) > 0
         if add_message:
-            instance_identifiers = ", ".join(instance_objs)
+            data_object_identifiers = [x["identifier"] for x in data_object_identifiers]
+            instance_identifiers = ", ".join(data_object_identifiers)
             message_str = GEOMETRY_CONFLICT_WITH_TEMPLATE.format(instance_identifiers)
             messages.info(request, message_str)
         return request

konova/sub_settings/django_settings.py

@@ -191,11 +191,10 @@ STATICFILES_DIRS = [
 ]
 
 # EMAIL (see https://docs.djangoproject.com/en/dev/topics/email/)
-
-# CHANGE_ME !!! ONLY FOR DEVELOPMENT !!!
 if DEBUG:
+    # ONLY FOR DEVELOPMENT NEEDED
     EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
-    EMAIL_FILE_PATH = '/tmp/app-messages' # change this to a proper location
+    EMAIL_FILE_PATH = '/tmp/app-messages'
 
 DEFAULT_FROM_EMAIL = env("DEFAULT_FROM_EMAIL")  # The default email address for the 'from' element
 SERVER_EMAIL = DEFAULT_FROM_EMAIL  # The default email sender address, which is used by Django to send errors via mail

konova/tests/test_geometries.py

@@ -85,12 +85,6 @@ class GeometryTestCase(BaseTestCase):
         )
 
     def test_get_data_objects(self):
-        num_objs_with_geom = 0
-        self.assertEqual(
-            len(self.geom_1.get_data_objects()),
-            num_objs_with_geom
-        )
-
         objs = [
             self.intervention,
             self.compensation,
@@ -98,16 +92,13 @@ class GeometryTestCase(BaseTestCase):
             self.ema,
         ]
         for obj in objs:
-            obj.geometry = self.geom_1
-            obj.save()
+            if not obj.geometry:
+                obj.geometry = Geometry.objects.create(geom=self.create_dummy_geometry())
+                obj.save()
 
-            num_objs_with_geom += 1
-            geom_objs = self.geom_1.get_data_objects()
-            self.assertEqual(
-                len(geom_objs),
-                num_objs_with_geom
-            )
-            self.assertIn(obj, geom_objs)
+            data_objs = obj.geometry.get_data_objects()
+            self.assertEqual(len(data_objs), 1)
+            self.assertIn(obj, data_objs)
 
     def test_as_feature_collection(self):
         geometry = self.geom_1.geom

konova/utils/generators.py

@@ -5,22 +5,18 @@ Contact: michel.peltriaux@sgdnord.rlp.de
 Created on: 09.11.20
 
 """
-import random
+import secrets
 import string
-import qrcode
-import qrcode.image.svg
-
-from io import BytesIO
 
 
-def generate_token() -> str:
+def generate_token(length: int = 64) -> str:
     """ Shortcut for default generating of e.g. API token
 
     Returns:
         token (str)
     """
     return generate_random_string(
-        length=64,
+        length=length,
         use_numbers=True,
         use_letters_lc=True
     )
@@ -39,7 +35,7 @@ def generate_random_string(length: int, use_numbers: bool = False, use_letters_l
         elements.append(string.ascii_uppercase)
 
     elements = "".join(elements)
-    ret_val = "".join(random.choice(elements) for i in range(length))
+    ret_val = "".join(secrets.choice(elements) for i in range(length))
     return ret_val
 
 class IdentifierGenerator:

nginx.conf

@@ -0,0 +1,25 @@
+server {
+    listen 80;
+    client_max_body_size 25M;
+
+    location / {
+        proxy_pass http://127.0.0.1:8000;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header Host $host;
+        proxy_redirect off;
+        proxy_cache_bypass $http_upgrade;
+    }
+
+    location /static/ {
+        alias /konova/static/;
+        access_log /var/log/nginx/access.log;
+        autoindex off;
+        types {
+            text/css css;
+            application/javascript js;
+        }
+    }
+
+    error_log /var/log/nginx/error.log;
+}

templates/map/client/libs/jspdf/1.3.2/jspdf.debug.js

@@ -9615,212 +9615,212 @@ Copyright (c) 2012 Willow Systems Corporation, willow-systems.com
     };
 })(jsPDF.API);
 
-/* Blob.js
- * A Blob implementation.
- * 2014-07-24
- *
- * By Eli Grey, http://eligrey.com
- * By Devin Samarin, https://github.com/dsamarin
- * License: X11/MIT
- *   See https://github.com/eligrey/Blob.js/blob/master/LICENSE.md
- */
-
-/*global self, unescape */
-/*jslint bitwise: true, regexp: true, confusion: true, es5: true, vars: true, white: true,
-  plusplus: true */
-
-/*! @source http://purl.eligrey.com/github/Blob.js/blob/master/Blob.js */
-
-(function (view) {
-	"use strict";
-
-	view.URL = view.URL || view.webkitURL;
-
-	if (view.Blob && view.URL) {
-		try {
-			new Blob;
-			return;
-		} catch (e) {}
-	}
-
-	// Internally we use a BlobBuilder implementation to base Blob off of
-	// in order to support older browsers that only have BlobBuilder
-	var BlobBuilder = view.BlobBuilder || view.WebKitBlobBuilder || view.MozBlobBuilder || (function(view) {
-		var
-			  get_class = function(object) {
-				return Object.prototype.toString.call(object).match(/^\[object\s(.*)\]$/)[1];
-			}
-			, FakeBlobBuilder = function BlobBuilder() {
-				this.data = [];
-			}
-			, FakeBlob = function Blob(data, type, encoding) {
-				this.data = data;
-				this.size = data.length;
-				this.type = type;
-				this.encoding = encoding;
-			}
-			, FBB_proto = FakeBlobBuilder.prototype
-			, FB_proto = FakeBlob.prototype
-			, FileReaderSync = view.FileReaderSync
-			, FileException = function(type) {
-				this.code = this[this.name = type];
-			}
-			, file_ex_codes = (
-				  "NOT_FOUND_ERR SECURITY_ERR ABORT_ERR NOT_READABLE_ERR ENCODING_ERR "
-				+ "NO_MODIFICATION_ALLOWED_ERR INVALID_STATE_ERR SYNTAX_ERR"
-			).split(" ")
-			, file_ex_code = file_ex_codes.length
-			, real_URL = view.URL || view.webkitURL || view
-			, real_create_object_URL = real_URL.createObjectURL
-			, real_revoke_object_URL = real_URL.revokeObjectURL
-			, URL = real_URL
-			, btoa = view.btoa
-			, atob = view.atob
-
-			, ArrayBuffer = view.ArrayBuffer
-			, Uint8Array = view.Uint8Array
-
-			, origin = /^[\w-]+:\/*\[?[\w\.:-]+\]?(?::[0-9]+)?/;
-		FakeBlob.fake = FB_proto.fake = true;
-		while (file_ex_code--) {
-			FileException.prototype[file_ex_codes[file_ex_code]] = file_ex_code + 1;
-		}
-		// Polyfill URL
-		if (!real_URL.createObjectURL) {
-			URL = view.URL = function(uri) {
-				var
-					  uri_info = document.createElementNS("http://www.w3.org/1999/xhtml", "a")
-					, uri_origin;
-				uri_info.href = uri;
-				if (!("origin" in uri_info)) {
-					if (uri_info.protocol.toLowerCase() === "data:") {
-						uri_info.origin = null;
-					} else {
-						uri_origin = uri.match(origin);
-						uri_info.origin = uri_origin && uri_origin[1];
-					}
-				}
-				return uri_info;
-			};
-		}
-		URL.createObjectURL = function(blob) {
-			var
-				  type = blob.type
-				, data_URI_header;
-			if (type === null) {
-				type = "application/octet-stream";
-			}
-			if (blob instanceof FakeBlob) {
-				data_URI_header = "data:" + type;
-				if (blob.encoding === "base64") {
-					return data_URI_header + ";base64," + blob.data;
-				} else if (blob.encoding === "URI") {
-					return data_URI_header + "," + decodeURIComponent(blob.data);
-				} if (btoa) {
-					return data_URI_header + ";base64," + btoa(blob.data);
-				} else {
-					return data_URI_header + "," + encodeURIComponent(blob.data);
-				}
-			} else if (real_create_object_URL) {
-				return real_create_object_URL.call(real_URL, blob);
-			}
-		};
-		URL.revokeObjectURL = function(object_URL) {
-			if (object_URL.substring(0, 5) !== "data:" && real_revoke_object_URL) {
-				real_revoke_object_URL.call(real_URL, object_URL);
-			}
-		};
-		FBB_proto.append = function(data/*, endings*/) {
-			var bb = this.data;
-			// decode data to a binary string
-			if (Uint8Array && (data instanceof ArrayBuffer || data instanceof Uint8Array)) {
-				var
-					  str = ""
-					, buf = new Uint8Array(data)
-					, i = 0
-					, buf_len = buf.length;
-				for (; i < buf_len; i++) {
-					str += String.fromCharCode(buf[i]);
-				}
-				bb.push(str);
-			} else if (get_class(data) === "Blob" || get_class(data) === "File") {
-				if (FileReaderSync) {
-					var fr = new FileReaderSync;
-					bb.push(fr.readAsBinaryString(data));
-				} else {
-					// async FileReader won't work as BlobBuilder is sync
-					throw new FileException("NOT_READABLE_ERR");
-				}
-			} else if (data instanceof FakeBlob) {
-				if (data.encoding === "base64" && atob) {
-					bb.push(atob(data.data));
-				} else if (data.encoding === "URI") {
-					bb.push(decodeURIComponent(data.data));
-				} else if (data.encoding === "raw") {
-					bb.push(data.data);
-				}
-			} else {
-				if (typeof data !== "string") {
-					data += ""; // convert unsupported types to strings
-				}
-				// decode UTF-16 to binary string
-				bb.push(unescape(encodeURIComponent(data)));
-			}
-		};
-		FBB_proto.getBlob = function(type) {
-			if (!arguments.length) {
-				type = null;
-			}
-			return new FakeBlob(this.data.join(""), type, "raw");
-		};
-		FBB_proto.toString = function() {
-			return "[object BlobBuilder]";
-		};
-		FB_proto.slice = function(start, end, type) {
-			var args = arguments.length;
-			if (args < 3) {
-				type = null;
-			}
-			return new FakeBlob(
-				  this.data.slice(start, args > 1 ? end : this.data.length)
-				, type
-				, this.encoding
-			);
-		};
-		FB_proto.toString = function() {
-			return "[object Blob]";
-		};
-		FB_proto.close = function() {
-			this.size = 0;
-			delete this.data;
-		};
-		return FakeBlobBuilder;
-	}(view));
-
-	view.Blob = function(blobParts, options) {
-		var type = options ? (options.type || "") : "";
-		var builder = new BlobBuilder();
-		if (blobParts) {
-			for (var i = 0, len = blobParts.length; i < len; i++) {
-				if (Uint8Array && blobParts[i] instanceof Uint8Array) {
-					builder.append(blobParts[i].buffer);
-				}
-				else {
-					builder.append(blobParts[i]);
-				}
-			}
-		}
-		var blob = builder.getBlob(type);
-		if (!blob.slice && blob.webkitSlice) {
-			blob.slice = blob.webkitSlice;
-		}
-		return blob;
-	};
-
-	var getPrototypeOf = Object.getPrototypeOf || function(object) {
-		return object.__proto__;
-	};
-	view.Blob.prototype = getPrototypeOf(new view.Blob());
+/* Blob.js
+ * A Blob implementation.
+ * 2014-07-24
+ *
+ * By Eli Grey, http://eligrey.com
+ * By Devin Samarin, https://github.com/dsamarin
+ * License: X11/MIT
+ *   See https://github.com/eligrey/Blob.js/blob/master/LICENSE.md
+ */
+
+/*global self, unescape */
+/*jslint bitwise: true, regexp: true, confusion: true, es5: true, vars: true, white: true,
+  plusplus: true */
+
+/*! @source http://purl.eligrey.com/github/Blob.js/blob/master/Blob.js */
+
+(function (view) {
+	"use strict";
+
+	view.URL = view.URL || view.webkitURL;
+
+	if (view.Blob && view.URL) {
+		try {
+			new Blob;
+			return;
+		} catch (e) {}
+	}
+
+	// Internally we use a BlobBuilder implementation to base Blob off of
+	// in order to support older browsers that only have BlobBuilder
+	var BlobBuilder = view.BlobBuilder || view.WebKitBlobBuilder || view.MozBlobBuilder || (function(view) {
+		var
+			  get_class = function(object) {
+				return Object.prototype.toString.call(object).match(/^\[object\s(.*)\]$/)[1];
+			}
+			, FakeBlobBuilder = function BlobBuilder() {
+				this.data = [];
+			}
+			, FakeBlob = function Blob(data, type, encoding) {
+				this.data = data;
+				this.size = data.length;
+				this.type = type;
+				this.encoding = encoding;
+			}
+			, FBB_proto = FakeBlobBuilder.prototype
+			, FB_proto = FakeBlob.prototype
+			, FileReaderSync = view.FileReaderSync
+			, FileException = function(type) {
+				this.code = this[this.name = type];
+			}
+			, file_ex_codes = (
+				  "NOT_FOUND_ERR SECURITY_ERR ABORT_ERR NOT_READABLE_ERR ENCODING_ERR "
+				+ "NO_MODIFICATION_ALLOWED_ERR INVALID_STATE_ERR SYNTAX_ERR"
+			).split(" ")
+			, file_ex_code = file_ex_codes.length
+			, real_URL = view.URL || view.webkitURL || view
+			, real_create_object_URL = real_URL.createObjectURL
+			, real_revoke_object_URL = real_URL.revokeObjectURL
+			, URL = real_URL
+			, btoa = view.btoa
+			, atob = view.atob
+
+			, ArrayBuffer = view.ArrayBuffer
+			, Uint8Array = view.Uint8Array
+
+			, origin = /^[\w-]+:\/*\[?[\w\.:-]+\]?(?::[0-9]+)?/;
+		FakeBlob.fake = FB_proto.fake = true;
+		while (file_ex_code--) {
+			FileException.prototype[file_ex_codes[file_ex_code]] = file_ex_code + 1;
+		}
+		// Polyfill URL
+		if (!real_URL.createObjectURL) {
+			URL = view.URL = function(uri) {
+				var
+					  uri_info = document.createElementNS("http://www.w3.org/1999/xhtml", "a")
+					, uri_origin;
+				uri_info.href = uri;
+				if (!("origin" in uri_info)) {
+					if (uri_info.protocol.toLowerCase() === "data:") {
+						uri_info.origin = null;
+					} else {
+						uri_origin = uri.match(origin);
+						uri_info.origin = uri_origin && uri_origin[1];
+					}
+				}
+				return uri_info;
+			};
+		}
+		URL.createObjectURL = function(blob) {
+			var
+				  type = blob.type
+				, data_URI_header;
+			if (type === null) {
+				type = "application/octet-stream";
+			}
+			if (blob instanceof FakeBlob) {
+				data_URI_header = "data:" + type;
+				if (blob.encoding === "base64") {
+					return data_URI_header + ";base64," + blob.data;
+				} else if (blob.encoding === "URI") {
+					return data_URI_header + "," + decodeURIComponent(blob.data);
+				} if (btoa) {
+					return data_URI_header + ";base64," + btoa(blob.data);
+				} else {
+					return data_URI_header + "," + encodeURIComponent(blob.data);
+				}
+			} else if (real_create_object_URL) {
+				return real_create_object_URL.call(real_URL, blob);
+			}
+		};
+		URL.revokeObjectURL = function(object_URL) {
+			if (object_URL.substring(0, 5) !== "data:" && real_revoke_object_URL) {
+				real_revoke_object_URL.call(real_URL, object_URL);
+			}
+		};
+		FBB_proto.append = function(data/*, endings*/) {
+			var bb = this.data;
+			// decode data to a binary string
+			if (Uint8Array && (data instanceof ArrayBuffer || data instanceof Uint8Array)) {
+				var
+					  str = ""
+					, buf = new Uint8Array(data)
+					, i = 0
+					, buf_len = buf.length;
+				for (; i < buf_len; i++) {
+					str += String.fromCharCode(buf[i]);
+				}
+				bb.push(str);
+			} else if (get_class(data) === "Blob" || get_class(data) === "File") {
+				if (FileReaderSync) {
+					var fr = new FileReaderSync;
+					bb.push(fr.readAsBinaryString(data));
+				} else {
+					// async FileReader won't work as BlobBuilder is sync
+					throw new FileException("NOT_READABLE_ERR");
+				}
+			} else if (data instanceof FakeBlob) {
+				if (data.encoding === "base64" && atob) {
+					bb.push(atob(data.data));
+				} else if (data.encoding === "URI") {
+					bb.push(decodeURIComponent(data.data));
+				} else if (data.encoding === "raw") {
+					bb.push(data.data);
+				}
+			} else {
+				if (typeof data !== "string") {
+					data += ""; // convert unsupported types to strings
+				}
+				// decode UTF-16 to binary string
+				bb.push(unescape(encodeURIComponent(data)));
+			}
+		};
+		FBB_proto.getBlob = function(type) {
+			if (!arguments.length) {
+				type = null;
+			}
+			return new FakeBlob(this.data.join(""), type, "raw");
+		};
+		FBB_proto.toString = function() {
+			return "[object BlobBuilder]";
+		};
+		FB_proto.slice = function(start, end, type) {
+			var args = arguments.length;
+			if (args < 3) {
+				type = null;
+			}
+			return new FakeBlob(
+				  this.data.slice(start, args > 1 ? end : this.data.length)
+				, type
+				, this.encoding
+			);
+		};
+		FB_proto.toString = function() {
+			return "[object Blob]";
+		};
+		FB_proto.close = function() {
+			this.size = 0;
+			delete this.data;
+		};
+		return FakeBlobBuilder;
+	}(view));
+
+	view.Blob = function(blobParts, options) {
+		var type = options ? (options.type || "") : "";
+		var builder = new BlobBuilder();
+		if (blobParts) {
+			for (var i = 0, len = blobParts.length; i < len; i++) {
+				if (Uint8Array && blobParts[i] instanceof Uint8Array) {
+					builder.append(blobParts[i].buffer);
+				}
+				else {
+					builder.append(blobParts[i]);
+				}
+			}
+		}
+		var blob = builder.getBlob(type);
+		if (!blob.slice && blob.webkitSlice) {
+			blob.slice = blob.webkitSlice;
+		}
+		return blob;
+	};
+
+	var getPrototypeOf = Object.getPrototypeOf || function(object) {
+		return object.__proto__;
+	};
+	view.Blob.prototype = getPrototypeOf(new view.Blob());
 }(typeof self !== "undefined" && self || typeof window !== "undefined" && window || undefined.content || undefined));
 
 /* FileSaver.js

user/views/propagate.py

@@ -7,9 +7,10 @@ Created on: 10.05.24
 """
 import base64
 import hashlib
+import http
 import json
 
-from cryptography.fernet import Fernet
+from cryptography.fernet import Fernet, InvalidToken
 from django.core.exceptions import ObjectDoesNotExist
 from django.http import HttpRequest, JsonResponse
 from django.utils.decorators import method_decorator
@@ -27,34 +28,77 @@ class PropagateUserView(View):
     proper rights management)
 
     """
+    class PropagateStatus:
+        UNPROCESSED = "unprocessed"
+        UPDATED = "updated"
+        CREATED = "created"
 
     @method_decorator(csrf_exempt)
     def dispatch(self, request, *args, **kwargs):
         return super().dispatch(request, *args, **kwargs)
 
     def post(self, request: HttpRequest, *args, **kwargs):
+        response_data = {
+            "success": None,
+            "status": None
+        }
+
         # Decrypt
         encrypted_body = request.body
         _hash = hashlib.md5()
         _hash.update(PROPAGATION_SECRET.encode("utf-8"))
         key = base64.urlsafe_b64encode(_hash.hexdigest().encode("utf-8"))
         fernet = Fernet(key)
-        body = fernet.decrypt(encrypted_body).decode("utf-8")
-        body = json.loads(body)
-
         try:
-            status = "updated"
-            user = User.resolve_user_using_propagation_data(body)
-            user = user.update_user_using_propagation_data(body)
-        except ObjectDoesNotExist:
-            user = User(**body)
-            status = "created"
+            body = fernet.decrypt(encrypted_body).decode("utf-8")
+            body = json.loads(body)
+        except InvalidToken:
+            response_data["error"] = "Invalid Token"
+            response_data["success"] = False
+            response_data["status"] = self.PropagateStatus.UNPROCESSED
+            return JsonResponse(
+                status=http.HTTPStatus.UNPROCESSABLE_CONTENT,
+                data=response_data
+            )
+        except (json.JSONDecodeError) as e:
+            response_data["error"] = str(e)
+            response_data["success"] = False
+            response_data["status"] = self.PropagateStatus.UNPROCESSED
+            return JsonResponse(
+                status=http.HTTPStatus.UNPROCESSABLE_CONTENT,
+                data=response_data
+            )
+
+        # Process decrypted user data
+        processing_ret_vals = self.__process_user_data(body)
+        response_data["success"] = processing_ret_vals[0]
+        response_data["status"] = processing_ret_vals[1]
+        user = processing_ret_vals[2]
+
         user.set_unusable_password()
         user.save()
 
-        data = {
-            "success": True,
-            "status": status
-        }
+        return JsonResponse(
+            status=http.HTTPStatus.OK,
+            data=response_data
+        )
 
-        return JsonResponse(data)
+    def __process_user_data(self, body: dict) -> (bool, str, User):
+        """ Process decrypted user data
+
+        Args:
+            body:
+
+        Returns:
+            success (bool): Whether the processing was successful
+            status (str): In which way the data was used ('created' | 'updated')
+            user (User): Processed user object
+        """
+        try:
+            user = User.resolve_user_using_propagation_data(body)
+            user = user.update_user_using_propagation_data(body)
+            status = self.PropagateStatus.UPDATED
+        except ObjectDoesNotExist:
+            user = User(**body)
+            status = self.PropagateStatus.CREATED
+        return True, status, user