Docker setup enhancements #137

New Issue

mpeltriaux · 2022-03-21T13:39:06+01:00

mpeltriaux commented

2022-03-21 13:39:06 +01:00

Status quo

Currently we are running the konova docker setup on a docker-compose orchestration of multiple containers:

konova-docker
1. gunicorn based main application
konova-worker-docker
1. celery based background processing
redis
1. used as message broker between foreground and background processing
konova-nginx
1. serves static web content

There are several things we need to improve on here.

Enhancements

Merge foreground and background service

The split of konova into a foreground and background service might not be the best way for deployment. There is a way to run gunicorn and celery in the same service, as already implemented in the linked branch to this issue: Docker_worker_enhance. This way we can save some resources but need to take a look on the performance anyway.

Explicit network definition

Currently we are simply using the same default network configuration for all orchestrated services, which is based on the network where the postgis container lives:

...
networks:
   default:
      external:
         name: postgis_network
...

This way the following definiton on a service is not needed:

service:
    some_service:
       ...
       networks:
          - default

To simplify future deployments on production server, we should add this default definition on each service. It might be important to link up to more networks, e.g. for communicating with EGON (regularly living in network 'efis'), so any connection can be established into containers living in these networks.

Conflicting redis

We should enhance the redis service definition, so there will be no conflicts (possibly) other redis instances:

services:
  konova-redis-cache:
    image: redis
    container_name: "konova-redis-cache"
    volumes:
      - /redis/data:/bitnami/redis/data
    environment:
      - REDIS_PASSWORD=CHANGE_ME
    networks:
      - default
...

# Status quo Currently we are running the konova docker setup on a docker-compose orchestration of multiple containers: 1. konova-docker 1. gunicorn based main application 1. konova-worker-docker 1. celery based background processing 1. redis 1. used as message broker between foreground and background processing 1. konova-nginx 1. serves static web content There are several things we need to improve on here. # Enhancements ## Merge foreground and background service The split of konova into a foreground and background service might not be the best way for deployment. There is a way to run gunicorn and celery in the same service, as already implemented in the linked branch to this issue: `Docker_worker_enhance`. This way we can save some resources but need to take a look on the performance anyway. ## Explicit network definition Currently we are simply using the same default network configuration for all orchestrated services, which is based on the network where the postgis container lives: ``` ... networks: default: external: name: postgis_network ... ``` This way the following definiton on a service is not needed: ``` service: some_service: ... networks: - default ``` To simplify future deployments on production server, we should add this default definition on each service. It might be important to link up to more networks, e.g. for communicating with EGON (regularly living in network 'efis'), so any connection can be established into containers living in these networks. ## Conflicting redis We should enhance the redis service definition, so there will be no conflicts (possibly) other redis instances: ``` services: konova-redis-cache: image: redis container_name: "konova-redis-cache" volumes: - /redis/data:/bitnami/redis/data environment: - REDIS_PASSWORD=CHANGE_ME networks: - default ... ```

mpeltriaux added the

enhancement

label 2022-03-21 13:39:12 +01:00

mpeltriaux self-assigned this 2022-03-21 13:39:14 +01:00

mpeltriaux commented

2022-05-09 14:06:23 +02:00