.gitea/workflows/UnitTests.yaml aktualisiert

Reviewed-on: #483

.gitea/workflows/UnitTests.yaml

@@ -0,0 +1,36 @@
+name: Gitea Actions Demo
+run-name: ${{ gitea.actor }} is testing out Gitea Actions 🚀
+on: [push]
+
+jobs:
+  Run Tests:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository code
+        uses: actions/checkout@v4
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        
+      - name: Set up Python
+        uses: actions/setup-python@v4
+
+      - name: Install dependencies if requirements.txt exists
+        run: |
+          python3 -m venv venv
+          source venv/bin/activate
+          if [ -f requirements.txt ]; then
+            pip install --no-deps -r requirements.txt
+          else
+            echo "No requirements.txt found, skipping installation."
+          fi
+
+      - name: Read .env from server and write to current directory
+        run: |
+          cat /home/jhaeder/.env > .env
+
+
+      - name: Run unit tests
+        run: |
+          source venv/bin/activate
+          python3 manage.py test

Dockerfile

@@ -1,36 +0,0 @@
-# Nutze ein schlankes Python-Image
-FROM python:3.11-slim-bullseye
-
-ENV PYTHONUNBUFFERED 1
-
-WORKDIR /konova
-
-# Installiere System-Abhängigkeiten
-RUN apt-get update && apt-get install -y --no-install-recommends \
-    gdal-bin redis-server nginx \
-    && rm -rf /var/lib/apt/lists/*  # Platz sparen
-
-# Erstelle benötigte Verzeichnisse & setze Berechtigungen
-RUN mkdir -p /var/log/nginx /var/log/gunicorn /var/lib/nginx /tmp/nginx_client_body \
-    && touch /var/log/nginx/access.log /var/log/nginx/error.log \
-    && chown -R root:root /var/log/nginx /var/lib/nginx /tmp/nginx_client_body
-
-# Kopiere und installiere Python-Abhängigkeiten
-COPY ./requirements.txt /konova/
-RUN pip install --upgrade pip && pip install --no-cache-dir -r requirements.txt
-
-# Entferne Standard-Nginx-Site und ersetze sie durch eigene Config
-RUN rm -rf /etc/nginx/sites-enabled/default
-COPY ./nginx.conf /etc/nginx/conf.d
-
-# Kopiere restliche Projektdateien
-COPY . /konova/
-
-# Sammle statische Dateien
-RUN python manage.py collectstatic --noinput
-
-# Exponiere Ports
-#EXPOSE 80 6379 8000
-
-# Setze Entrypoint
-ENTRYPOINT ["/konova/docker-entrypoint.sh"]

README.md

@@ -4,7 +4,6 @@ the database postgresql and the css library bootstrap as well as the icon packag
 fontawesome for a modern look, following best practices from the industry.
 
 ## Background processes
-### !!! For non-docker run
 Konova uses celery for background processing. To start the worker you need to run
 ```shell
 $ celery -A konova worker -l INFO
@@ -19,58 +18,3 @@ Technical documention is provided in the projects git wiki.
 A user documentation is not available (and not needed, yet). 
 
 
-# Docker 
-To run the docker-compose as expected, you need to take the following steps:
-
-1. Create a database containing docker, using an appropriate Dockerfile, e.g. the following
-```
-version: '3.3'
-services:
-  postgis:
-    image: postgis/postgis
-    restart: always
-    container_name: postgis-docker
-    ports: 
-      - 5433:5432
-    volumes:
-      - db-volume:/var/lib/postgresql/data
-    environment:
-      - POSTGRES_PASSWORD=postgres
-      - POSTGRES_USER=postgres
-    networks:
-      - db-network-bridge
-
-networks:
-  db-network-bridge:
-    driver: "bridge"
-
-volumes:
-  db-volume:
-```
-This Dockerfile creates a Docker container running postgresql and postgis, creates the default superuser postgres, 
-creates a named volume for persisting the database and creates a new network bridge, which **must be used by any other
-container, which wants to write/read on this database**. 
-
-2. Make sure the name of the network bridge above matches the network in the konova docker-compose.yml  
-3. Get into the running postgis container (`docker exec -it postgis-docker bash`) and create new databases, users and so on. Make sure the database `konova` exists now!
-4. Replace all `CHANGE_ME_xy` values inside of konova/docker-compose.yml for your installation. Make sure the `SSO_HOST` holds the proper SSO host, e.g. for the arnova project `arnova.example.org` (Arnova must be installed and the webserver configured as well, of course)
-5. Take a look on konova/settings.py and konova/sub_settings/django_settings.py. Again: Replace all occurences of `CHANGE_ME` with proper values for your installation.
-   1. Make sure you have the proper host strings added to `ALLOWED_HOSTS` inside of django_settings.py.  
-6. Build and run the docker setup using `docker-compose build` and `docker-compose start` from the main directory of this project (where the docker-compose.yml lives)
-7. Run migrations! To do so, get into the konova service container (`docker exec -it konova-docker bash`) and run the needed commands (`python manage.py makemigrations LIST_OF_ALL_MIGRATABLE_APPS`, then `python manage.py migrate`)
-8. Run the setup command `python manage.py setup` and follow the instructions on the CLI
-9. To enable **SMTP** mail support, make sure your host machine (the one where the docker container run) has the postfix service configured properly. Make sure the `mynetworks` variable is xtended using the docker network bridge ip, created in the postgis container and used by the konova services.
-   1. **Hint**: You can find out this easily by trying to perform a test mail in the running konova web application (which will fail, of course). Then take a look to the latest entries in `/var/log/mail.log` on your host machine. The failed IP will be displayed there.
-   2. **Please note**: This installation guide is based on SMTP using postfix! 
-   3. Restart the postfix service on your host machine to reload the new configuration (`service postfix restart`)
-10. Finally, make sure your host machine webserver passes incoming requests properly to the docker nginx webserver of konova. A proper nginx config for the host machine may look like this:
-```
-server {
-    server_name konova.domain.org;
-
-    location / {
-        proxy_pass http://localhost:KONOVA_NGINX_DOCKER_PORT/;
-        proxy_set_header Host $host;
-    }
-}
-```

api/tests/v1/update/test_api_update.py

@@ -12,7 +12,7 @@ from django.contrib.gis import geos
 from django.urls import reverse
 
 from api.tests.v1.share.test_api_sharing import BaseAPIV1TestCase
-from konova.sub_settings.lanis_settings import DEFAULT_SRID_RLP
+from konova.models import Geometry
 
 
 class APIV1UpdateTestCase(BaseAPIV1TestCase):
@@ -64,7 +64,7 @@ class APIV1UpdateTestCase(BaseAPIV1TestCase):
 
         put_props = put_body["properties"]
         put_geom = geos.fromstr(json.dumps(put_body))
-        put_geom.transform(DEFAULT_SRID_RLP)
+        put_geom = Geometry.cast_to_rlp_srid(put_geom)
         self.assertEqual(put_geom, self.intervention.geometry.geom)
         self.assertEqual(put_props["title"], self.intervention.title)
         self.assertNotEqual(modified_on, self.intervention.modified)
@@ -94,7 +94,7 @@ class APIV1UpdateTestCase(BaseAPIV1TestCase):
 
         put_props = put_body["properties"]
         put_geom = geos.fromstr(json.dumps(put_body))
-        put_geom.transform(DEFAULT_SRID_RLP)
+        put_geom = Geometry.cast_to_rlp_srid(put_geom)
         self.assertEqual(put_geom, self.compensation.geometry.geom)
         self.assertEqual(put_props["title"], self.compensation.title)
         self.assertNotEqual(modified_on, self.compensation.modified)
@@ -124,7 +124,7 @@ class APIV1UpdateTestCase(BaseAPIV1TestCase):
 
         put_props = put_body["properties"]
         put_geom = geos.fromstr(json.dumps(put_body))
-        put_geom.transform(DEFAULT_SRID_RLP)
+        put_geom = Geometry.cast_to_rlp_srid(put_geom)
         self.assertEqual(put_geom, self.eco_account.geometry.geom)
         self.assertEqual(put_props["title"], self.eco_account.title)
         self.assertNotEqual(modified_on, self.eco_account.modified)
@@ -156,7 +156,7 @@ class APIV1UpdateTestCase(BaseAPIV1TestCase):
 
         put_props = put_body["properties"]
         put_geom = geos.fromstr(json.dumps(put_body))
-        put_geom.transform(DEFAULT_SRID_RLP)
+        put_geom = Geometry.cast_to_rlp_srid(put_geom)
         self.assertEqual(put_geom, self.ema.geometry.geom)
         self.assertEqual(put_props["title"], self.ema.title)
         self.assertNotEqual(modified_on, self.ema.modified)

api/utils/serializer/serializer.py

@@ -13,7 +13,7 @@ from django.contrib.gis.geos import GEOSGeometry
 from django.core.paginator import Paginator
 from django.db.models import Q
 
-from konova.sub_settings.lanis_settings import DEFAULT_SRID_RLP
+from konova.models import Geometry
 from konova.utils.message_templates import DATA_UNSHARED
 
 
@@ -145,8 +145,8 @@ class AbstractModelAPISerializer:
         if isinstance(geojson, dict):
             geojson = json.dumps(geojson)
         geometry = geos.fromstr(geojson)
-        if geometry.srid != DEFAULT_SRID_RLP:
-            geometry.transform(DEFAULT_SRID_RLP)
+        geometry = Geometry.cast_to_rlp_srid(geometry)
+        geometry = Geometry.cast_to_multipolygon(geometry)
         return geometry
 
     def _get_obj_from_db(self, id, user):

compensation/tests/ecoaccount/test_workflow.py

@@ -86,7 +86,7 @@ class EcoAccountWorkflowTestCase(BaseWorkflowTestCase):
         new_title = self.create_dummy_string()
         new_identifier = self.create_dummy_string()
         new_comment = self.create_dummy_string()
-        new_geometry = MultiPolygon(srid=4326)  # Create an empty geometry
+        new_geometry = self.create_dummy_geometry()
         test_conservation_office = self.get_conservation_office_code()
         test_deductable_surface = self.eco_account.deductable_surface + 100
 
@@ -103,7 +103,7 @@ class EcoAccountWorkflowTestCase(BaseWorkflowTestCase):
             "identifier": new_identifier,
             "title": new_title,
             "comment": new_comment,
-            "geom": new_geometry.geojson,
+            "geom": self.create_geojson(new_geometry),
             "surface": test_deductable_surface,
             "conservation_office": test_conservation_office.id
         }

docker-compose.yml

@@ -1,23 +0,0 @@
-version: '3.3'
-
-services:
-  konova:
-    external_links:
-      - postgis:db
-      - arnova-nginx-server:arnova
-    build: .
-    image: "ksp/konova:1.8"
-    container_name: "konova-docker"
-    command: ./docker-entrypoint.sh
-    restart: always
-    volumes:
-      - /data/apps/konova/uploaded_files:/konova_uploaded_files
-    ports:
-      - "1337:80"
-
-# Instead of an own, new network, we need to connect to the existing one, which is provided by the postgis container
-# NOTE: THIS NETWORK MUST EXIST
-networks:
-  default:
-    external:
-      name: postgis_nat_it_backend

docker-entrypoint.sh

@@ -1,27 +0,0 @@
-#!/bin/bash
-
-set -e  # Beende Skript bei Fehlern
-set -o pipefail  # Fehler in Pipelines nicht ignorieren
-
-# Starte Redis
-redis-server --daemonize yes
-
-# Starte Celery Worker im Hintergrund
-celery -A konova worker --loglevel=info &
-
-# Starte Nginx als Hintergrundprozess
-nginx -g "daemon off;" &
-
-# Setze Gunicorn Worker-Anzahl (Standard: (2*CPUs)+1)
-WORKERS=${GUNICORN_WORKERS:-$((2 * $(nproc) + 1))}
-
-# Stelle sicher, dass Logs existieren
-mkdir -p /var/log/gunicorn
-touch /var/log/gunicorn/access.log /var/log/gunicorn/error.log
-
-# Starte Gunicorn als Hauptprozess
-exec gunicorn --workers="$WORKERS" konova.wsgi:application \
-    --bind=0.0.0.0:8000 \
-    --access-logfile /var/log/gunicorn/access.log \
-    --error-logfile /var/log/gunicorn/error.log \
-    --access-logformat '%({x-real-ip}i)s via %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

ema/tests/test_workflow.py

@@ -84,7 +84,7 @@ class EmaWorkflowTestCase(BaseWorkflowTestCase):
         new_title = self.create_dummy_string()
         new_identifier = self.create_dummy_string()
         new_comment = self.create_dummy_string()
-        new_geometry = MultiPolygon(srid=4326)  # Create an empty geometry
+        new_geometry = self.create_dummy_geometry()  # Create an empty geometry
         test_conservation_office = self.get_conservation_office_code()
 
         check_on_elements = {
@@ -99,7 +99,7 @@ class EmaWorkflowTestCase(BaseWorkflowTestCase):
             "identifier": new_identifier,
             "title": new_title,
             "comment": new_comment,
-            "geom": new_geometry.geojson,
+            "geom": self.create_geojson(new_geometry),
             "conservation_office": test_conservation_office.id
         }
         self.client_user.post(url, post_data)

intervention/tests/unit/test_forms.py

@@ -124,7 +124,7 @@ class EditInterventionFormTestCase(NewInterventionFormTestCase):
         self.assertIsNotNone(obj.responsible.handler)
         self.assertEqual(obj.title, data["title"])
         self.assertEqual(obj.comment, data["comment"])
-        self.assertTrue(test_geom.equals_exact(obj.geometry.geom, 0.000001))
+        self.assert_equal_geometries(test_geom, obj.geometry.geom)
 
         self.assertEqual(obj.legal.binding_date, today)
         self.assertEqual(obj.legal.registration_date, today)

konova/forms/geometry_form.py

@@ -72,9 +72,8 @@ class SimpleGeomForm(BaseForm):
         # will be rendered again on failed submit
         self.initialize_form_field("geom", self.data["geom"])
 
-        # Read geojson into gdal geometry
-        # HINT: This can be simplified if the geojson format holds data in epsg:4326 (GDAL provides direct creation for
-        # this case)
+        # Initialize features list with empty MultiPolygon, so that an empty input will result in a
+        # proper empty MultiPolygon object
         features = []
         features_json = geom.get("features", [])
         accepted_ogr_types = [
@@ -102,23 +101,25 @@ class SimpleGeomForm(BaseForm):
                 return is_valid
 
             is_valid &= self.__is_area_valid(g)
-
-            polygon = Polygon.from_ewkt(g.ewkt)
-            is_valid &= polygon.valid
-            if not polygon.valid:
-                self.add_error("geom", polygon.valid_reason)
+            g = Polygon.from_ewkt(g.ewkt)
+            is_valid &= g.valid
+            if not g.valid:
+                self.add_error("geom", g.valid_reason)
                 return is_valid
 
-            features.append(polygon)
+            if isinstance(g, Polygon):
+                features.append(g)
+            elif isinstance(g, MultiPolygon):
+                features.extend(list(g))
 
         # Unionize all geometry features into one new MultiPolygon
-        form_geom = MultiPolygon(srid=DEFAULT_SRID_RLP)
-        for feature in features:
-            form_geom = form_geom.union(feature)
+        if features:
+            form_geom = MultiPolygon(*features, srid=DEFAULT_SRID_RLP).unary_union
+        else:
+            form_geom = MultiPolygon(srid=DEFAULT_SRID_RLP)
 
         # Make sure to convert into a MultiPolygon. Relevant if a single Polygon is provided.
-        if form_geom.geom_type != "MultiPolygon":
-            form_geom = MultiPolygon(form_geom, srid=DEFAULT_SRID_RLP)
+        form_geom = Geometry.cast_to_multipolygon(form_geom)
 
         # Write unioned Multipolygon into cleaned data
         if self.cleaned_data is None:

konova/models/geometry.py

@@ -11,6 +11,7 @@ from django.contrib.gis.db.models import MultiPolygonField
 from django.core.exceptions import ObjectDoesNotExist, MultipleObjectsReturned
 from django.db import models, transaction
 from django.utils import timezone
+from django.contrib.gis.geos import MultiPolygon
 
 from konova.models import BaseResource, UuidModel
 from konova.sub_settings.lanis_settings import DEFAULT_SRID_RLP
@@ -383,6 +384,36 @@ class Geometry(BaseResource):
 
         return complexity_factor
 
+    @staticmethod
+    def cast_to_multipolygon(input_geom):
+        """ If input_geom is not a MultiPolygon, cast to MultiPolygon
+
+        Args:
+            input_geom ():
+
+        Returns:
+            output_geom
+        """
+        output_geom = input_geom
+        if not isinstance(input_geom, MultiPolygon):
+            output_geom = MultiPolygon(input_geom, srid=DEFAULT_SRID_RLP)
+        return output_geom
+
+    @staticmethod
+    def cast_to_rlp_srid(input_geom):
+        """ If input_geom is not of RLP SRID (25832), cast to RLP SRID
+
+        Args:
+            input_geom ():
+
+        Returns:
+            output_geom
+        """
+        output_geom = input_geom
+        if output_geom.srid != DEFAULT_SRID_RLP:
+            output_geom.transform(DEFAULT_SRID_RLP)
+        return output_geom
+
 
 class GeometryConflict(UuidModel):
     """

konova/sub_settings/django_settings.py

@@ -192,10 +192,11 @@ STATICFILES_DIRS = [
 ]
 
 # EMAIL (see https://docs.djangoproject.com/en/dev/topics/email/)
+
+# CHANGE_ME !!! ONLY FOR DEVELOPMENT !!!
 if DEBUG:
-    # ONLY FOR DEVELOPMENT NEEDED
     EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
-    EMAIL_FILE_PATH = '/tmp/app-messages'
+    EMAIL_FILE_PATH = '/tmp/app-messages' # change this to a proper location
 
 DEFAULT_FROM_EMAIL = env("DEFAULT_FROM_EMAIL")  # The default email address for the 'from' element
 SERVER_EMAIL = DEFAULT_FROM_EMAIL  # The default email sender address, which is used by Django to send errors via mail

konova/tests/test_views.py

@@ -469,7 +469,7 @@ class BaseTestCase(TestCase):
         eco_account.save()
         return eco_account
 
-    def assert_equal_geometries(self, geom1: MultiPolygon, geom2: MultiPolygon, tolerance = 0.001):
+    def assert_equal_geometries(self, geom1: MultiPolygon, geom2: MultiPolygon, tolerance=0.001):
         """ Assert for geometries to be equal
 
         Transforms the geometries to matching srids before checking
@@ -491,7 +491,10 @@ class BaseTestCase(TestCase):
             # transformation from one coordinate system into the other, which is valid
             geom1.transform(geom2.srid)
             geom2.transform(geom1.srid)
-        self.assertTrue(geom1.equals_exact(geom2, tolerance) or geom2.equals_exact(geom1, tolerance))
+        self.assertTrue(
+            geom1.equals_exact(geom2, tolerance=tolerance),
+            msg=f"Difference is {abs(geom1.area - geom2.area)} with {geom1.area} and {geom2.area} in a tolerance of {tolerance}"
+        )
 
 
 class BaseViewTestCase(BaseTestCase):

nginx.conf

@@ -1,25 +0,0 @@
-server {
-    listen 80;
-    client_max_body_size 25M;
-
-    location / {
-        proxy_pass http://127.0.0.1:8000;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header Host $host;
-        proxy_redirect off;
-        proxy_cache_bypass $http_upgrade;
-    }
-
-    location /static/ {
-        alias /konova/static/;
-        access_log /var/log/nginx/access.log;
-        autoindex off;
-        types {
-            text/css css;
-            application/javascript js;
-        }
-    }
-
-    error_log /var/log/nginx/error.log;
-}

requirements.txt

@@ -48,7 +48,7 @@ pytz==2024.2
 PyYAML==6.0.2
 qrcode==7.3.1
 redis==5.1.0b6
-requests<2.32.0
+requests==2.32.3
 six==1.16.0
 soupsieve==2.5
 sqlparse==0.5.1

user/admin.py

@@ -15,6 +15,7 @@ class UserNotificationAdmin(admin.ModelAdmin):
 class UserAdmin(admin.ModelAdmin):
     list_display = [
         "id",
+        "sso_identifier",
         "username",
         "first_name",
         "last_name",

user/migrations/0010_user_sso_identifier.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.1.6 on 2025-09-12 06:10
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('user', '0009_user_oauth_token'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='user',
+            name='sso_identifier',
+            field=models.CharField(blank=True, db_comment='Identifies the account based on an unique identifier from the SSO system', max_length=255, null=True),
+        ),
+    ]

user/models/user.py

@@ -6,6 +6,7 @@ Created on: 15.11.21
 
 """
 from django.contrib.auth.models import AbstractUser
+from django.core.exceptions import ObjectDoesNotExist
 
 from django.db import models
 
@@ -32,6 +33,12 @@ class User(AbstractUser):
         db_comment="OAuth token for the user",
         related_name="+"
     )
+    sso_identifier = models.CharField(
+        blank=True,
+        null=True,
+        db_comment="Identifies the account based on an unique identifier from the SSO system",
+        max_length=255,
+    )
 
     def is_notification_setting_set(self, notification_enum: UserNotificationEnum):
         return self.notifications.filter(
@@ -264,4 +271,48 @@ class User(AbstractUser):
             self.oauth_token.delete()
         self.oauth_token = token
         self.save()
-        return self
+        return self
+
+    @staticmethod
+    def resolve_user_using_propagation_data(data: dict):
+        """ Fetches user from db by the given data from propagation process
+
+        Args:
+            data (dict): json containing user information from the sso system
+
+        Returns:
+            user (User): The resolved user
+        """
+        username = data.get("username", None)
+        sso_identifier = data.get("sso_identifier", None)
+        if not username and not sso_identifier:
+            raise AssertionError("No username or sso identifier provided")
+
+        try:
+            user = User.objects.get(username=username)
+        except ObjectDoesNotExist:
+            try:
+                user = User.objects.get(sso_identifier=sso_identifier)
+            except ObjectDoesNotExist:
+                raise ObjectDoesNotExist("No user with this username or sso identifier was found")
+
+        return user
+
+    def update_user_using_propagation_data(self, data: dict):
+        """ Update user data based on propagation data from sso system
+
+        Args:
+            data (dict): json containing user information from the sso system
+
+        Returns:
+            user (User): The updated user
+        """
+        skipable_attrs = {
+            "is_staff",
+            "is_superuser",
+        }
+        for _attr, _val in data.items():
+            if _attr in skipable_attrs:
+                continue
+            setattr(self, _attr, _val)
+        return self

user/views/propagate.py

@@ -44,17 +44,8 @@ class PropagateUserView(View):
 
         try:
             status = "updated"
-            user = User.objects.get(username=body.get('username'))
-            # Update user data, excluding some changes
-            skipable_attrs = {
-                "username",
-                "is_staff",
-                "is_superuser",
-            }
-            for _attr, _val in body.items():
-                if _attr in skipable_attrs:
-                    continue
-                setattr(user, _attr, _val)
+            user = User.resolve_user_using_propagation_data(body)
+            user = user.update_user_using_propagation_data(body)
         except ObjectDoesNotExist:
             user = User(**body)
             status = "created"