Skip to content

Deploy cert-manager on the k8s cluster

Watch out the latest developments on the kubernetes jelastic jps installation manifest. Currently, we are using version 1.16.6, which does not support usage of cert-manager. That is why we have a frontal nginx load-balancer to allow for SSL. This causes several issues, even though it is more interesting financially speaking (we have to define only one IPv4 even if we have many worker nodes). One important issue is that fusionauth cannot work on the cluster. When we update the installation, we will have to take care of a few things:

  • write cluster installation manifest
    • create the necessary subdomains of shopozor.hidora.com (if we really use that domain and not a custom domain)
    • add IPv4 to all worker nodes
    • install cert-manager add-on on master node and let it deal with SSL certificate management on the cluster
    • patch / patch the following ingresses so that they have the cert-manager activated (cf. tasks below)
      • default/kubernetes-api -> shopozor.hidora.com
      • kubernetes-dashboard/kubernetes-dashboard -> shopozor.hidora.com
  • try to install and use fusionauth on the cluster
  • make sure to secure minio too (cf. services#214)
    • run install_assets.sh as part of the cluster installation
    • patch the minio ingress after the cert-manager has been successfully installed
  • adapt devspace configuration
    • change JELASTIC_DOMAIN to DOMAIN_NAME
      • remove var
      • check collateral damage
    • change REMOTE_DOMAIN to DOMAIN_NAME
      • remove var
      • check collateral damage
    • check PRODUCTION_CORS_DOMAINS
    • see if we can remove JELASTIC_ENV_NAME (which is currently set to budzonnerie)
      • remove var
      • check collateral damage
    • staging
      • minio
        • add tls
        • add cert-manager.io/cluster-issuer: letsencrypt-prod annotation
        • fix kubernetes.io/ingress.class: nginx-cert annotation
      • api
        • add tls
        • add cert-manager.io/cluster-issuer: letsencrypt-prod annotation
        • fix kubernetes.io/ingress.class: nginx-cert annotation
      • admin-ui
        • add tls
        • add cert-manager.io/cluster-issuer: letsencrypt-prod annotation
        • fix kubernetes.io/ingress.class: nginx-cert annotation
      • specification
        • add tls
        • add cert-manager.io/cluster-issuer: letsencrypt-prod annotation
        • fix kubernetes.io/ingress.class: nginx-cert annotation
      • webhooks
        • add tls
        • add cert-manager.io/cluster-issuer: letsencrypt-prod annotation
        • fix kubernetes.io/ingress.class: nginx-cert annotation
    • preprod
      • api
        • replace tls hostname
      • admin-ui
        • replace tls hostname
      • specification
        • replace tls hostname
      • webhooks
        • replace tls hostname
    • prod
      • api
        • replace tls hostname
      • admin-ui
        • replace tls hostname
      • specification
        • replace tls hostname
  • adapt gitlab-ci.yaml
    • deploy-consumer-ui
      • add tls
        • add cert-manager.io/cluster-issuer: letsencrypt-prod annotation
        • fix kubernetes.io/ingress.class: nginx-cert annotation
  • make sure to enable the nginx whitelist annotations for the webhooks gateway
   - op: add
     path: /deployments/name=webhooks/helm/values/ingress/annotations
     value:
       # comes from https://checkout.postfinance.ch/fr-ch/doc/webhooks#Public%20IP%20Addresses
       # TODO: get the addresses from an environment variable!
       nginx.ingress.kubernetes.io/whitelist-source-range: 52.211.247.160/32, 52.31.52.106/32, 52.211.171.77/32, 52.211.239.229/32, 52.211.209.173/32, 52.208.210.84/32, 52.212.109.85/32, 52.210.89.1/32, 52.212.185.152/32, 52.212.192.130/32
  • clean up unnecessary configuration
    • remove jelastic/nginx
    • rework jelastic/secure.jps
  • try to deploy actual staging cluster on ameising.com
    • install missing stuff from manifest.jps on shopozor.hidora.com
    • switch cluster integration on gitlab ci
    • run staging pipeline
    • restore DOMAIN_NAME to budzonnerie.com
    • restore STAGING_*_API to a value containing budzonnerie.com
    • define ISSUER_NAME environment variable on gitlab-ci
      • set fusionauth issuer to shopozor.hidora.com
      • use ISSUER_NAME env var to set issuer on hasura
    • restore webhook url on postfinance
  • support DOMAIN_NAMES in ingress configurations (remove and replace env var REMOTE_DOMAINS)
  • see if cookies can be issued properly upon login (cf. this post)
  • see if websockets are working (cf. this post)

Double-check services#192 for more information.

More information on how to install the certification manager on the k8s cluster can be found

Tasks to reconfigure ingresses with cert-manager:

  • add annotation: cert-manager.io/cluster-issuer: letsencrypt-prod
  • modify annotation: kubernetes.io/ingress.class: nginx-cert
  • add host to rule: host: admin.shopozor.hidora.com
  • add tls object to rules:
tls:
- hosts:
  - admin.shopozor.hidora.com
  secretName: hellworld-tls
Edited by Laurent