Add a cluster to APPUiO Cloud

This guide describes the steps required to turn an OpenShift 4 cluster into an APPUiO Zone.

VSHN employees should follow Activate Zone instead of continuing this guide.

Configure Keycloak

  1. Create a new Keycloak Client with the following settings (leave the others at default value)

    Client ID = appuio_<c-cluster-id> (1)
    Access Type = confidential
    Valid Redirect URIs = https://oauth-openshift.apps.cluster-id.tld/oauth2callback/APPUiO
    Base URL = https://console.cluster-id.tld/
    
    [ Authentication Flow Overrides ]
    Browser Flow = APPUiO Browser RBAC (2)
    1 For each enabled APPUiO Zone there shall be its own client using the cluster ID and the prefix appuio_ as name.
    2 See Setup RBAC
VSHN uses id.vshn.net as the IDP.

Configure openshift4-authentication

  1. After adding the cluster as a new client to APPUiO IdP, add the client secret to Vault. The value is being displayed in a grey box in the "Credentials" tab from the Keycloak client settings.

  2. Add component configuration

    parameters:
      openshift4_authentication:
        secrets:
          appuio-cloud-keycloak:
            clientSecret: '?{vaultkv:${cluster:tenant}/${cluster:name}/oidc/appuio-keycloak/clientSecret}' (1)
    
        identityProviders:
          appuio_keycloak:
            name: APPUiO-Cloud
            type: OpenID
            mappingMethod: add
            openID:
              issuer: https://id.appuio.cloud/auth/realms/appuio-cloud
              clientID: ${cluster:name}
              clientSecret:
                name: appuio-cloud-keycloak
              claims: (2)
                preferredUsername:
                  - preferred_username
                name:
                  - name
                email:
                  - email
    1 The Vault path for client secret
    2 See also User object names in the OpenShift cluster

Configure group-sync-operator

The group-sync-operator is required to sync Keycloak groups to each APPUiO Zone.

  1. Add component configuration

    parameters:
      group_sync_operator:
        secrets:
          sync-appuio-keycloak-groups:
            stringData:
              username: '?{vaultkv:${cluster:tenant}/${cluster:name}/oidc/appuio-keycloak-sync/username}'
              password: '?{vaultkv:${cluster:tenant}/${cluster:name}/oidc/appuio-keycloak-sync/password}'
        sync:
          sync-keycloak-groups:
            schedule: '* * * * *'
            providers:
              keycloak:
                keycloak:
                  url: https://id.appuio.cloud
                  credentialsSecret:
                    name: sync-appuio-keycloak-groups
                  loginRealm: master
                  realm: appuio-cloud
                  scope: sub

Configure keycloak-attribute-sync-controller

The keycloak-attribute-sync-controller is required to sync Keycloak user attributes to each APPUiO Zone.

  1. Add component configuration

    parameters:
      keycloak_attribute_sync_controller:
        sync_credentials:
          sync-default-org-credentials:
            stringData:
              username: '?{vaultkv:${cluster:tenant}/${cluster:name}/oidc/appuio-keycloak-sync/username}' (1)
              password: '?{vaultkv:${cluster:tenant}/${cluster:name}/oidc/appuio-keycloak-sync/password}'
    
        sync_configurations:
          sync-default-org:
            url: https://id.appuio.cloud
            loginRealm: master
            credentialsSecret:
              name: sync-default-org-credentials
            realm: appuio-cloud
            attribute: appuio.io/default-organization
            targetAnnotation: appuio.io/default-organization
            schedule: '@every 1m'
    1 The user for syncing attributes is the same as the one used for group-sync-operator.
  2. Compile and push the cluster catalog

  3. Wait for Argo CD to sync the config

Configure metering metrics collection

We collect a small set of metrics from each APPUiO Zone for metering. This is done by deploying a Thanos Receiver on each zone and configuring the zone’s cluster monitoring to send selected metrics to the Thanos Receiver via Prometheus’s remote write feature.

  1. Create an S3 bucket which can be accessed using the same credentials as the cluster’s registry bucket. Use name ${CLUSTER_ID}-appuio-metering for the bucket.

  1. Add configuration for components openshift4-monitoring and thanos

    applications:
      - thanos as appuio-metering-thanos-receiver
    parameters:
      openshift4_monitoring:
        configs:
          prometheusK8s:
            remoteWrite:
              - name: appuio-metering
                url: http://thanos-receive.appuio-metering-thanos-receiver.svc:19291/api/v1/receive
                headers:
                  THANOS-TENANT: '${cluster:name}' (1)
                writeRelabelConfigs:
                  - sourceLabels: ['__name__']
                    regex: '(container_memory_usage_bytes|kube_pod_container_resource_requests|kube_persistentvolumeclaim_resource_requests_storage_bytes)' (2)
                    action: keep
    
      appuio_metering_thanos_receiver:
        namespace: appuio-metering-thanos-receiver
        cluster_kubernetes_version: '${dynamic_facts:kubernetesVersion:major}.${dynamic_facts:kubernetesVersion:minor}' (3)
        commonConfig:
          volumeClaimTemplate:
            spec:
              storageClassName: ssd (4)
          # Override default security context for all components
          securityContext: {}
        objectStorageConfig:
          type: S3
          config:
            bucket: '${cluster:name}-appuio-metering' (5)
            endpoint: objects.lpg.cloudscale.ch (6)
            region: lpg (6)
            access_key: '?{vaultkv:${cluster:tenant}/${cluster:name}/cloudscale/s3_access_key}' (7)
            secret_key: '?{vaultkv:${cluster:tenant}/${cluster:name}/cloudscale/s3_secret_key}' (7)
        query:
          enabled: false # we don't need the querier on the zones
        receive:
          enabled: true
          replicas: 2
    1 Send extra THANOS-TENANT header so that Thanos Receiver sets a useful tenant_id label on the received metrics.
    2 The set of metrics to collect (as an RE2 regex)
    3 The cluster’s Kubernetes version. The example makes use of the kubernetesVersion dynamic fact reported by Steward on the cluster.
    4 The storage class to use for the PVCs for the Thanos Receiver. We highly recommend using a storage class suitable for database workloads.
    5 The name of the bucket created in the previous step
    6 The S3 endpoint and region for the bucket Adjust for zones which are not on cloudscale.ch in region LPG.
    7 The S3 credentials for the bucket. Adjust the secret references so they point to the Vault secret that’s used for the cluster’s registry bucket.

  1. Compile and push the cluster catalog

  2. Wait for ArgoCD to sync the changes

Configure Appcat ObjectStorage provider

Please refer to the dedicated page to setup AppCat ObjectStorage

Add Cluster to Status Page

  1. Login to Statuspal

  2. Select APPUiO Cloud status page

    You need Admin rights on the "APPUiO Cloud" status page to be able to add new services.
  3. Create a service for the APPUiO Zone

  4. Create the following other services and select the previously created APPUiO Zone service as their parent:

    1. OpenShift Console

      1. Description: URL to the OpenShift Console

      2. Select Statuspal monitoring

      3. Method: HEAD

      4. Ping url: URL to the OpenShift Console

      5. Check Automatically create incident

      6. Check Pause monitoring during maintenances

    2. OpenShift / Kubernetes API

      1. Description: URL to the OpenShift / Kubernetes API

      2. Select Statuspal monitoring

      3. Method: GET

      4. Ping url: URL to the OpenShift / Kubernetes API + /healthz

      5. Check Automatically create incident

      6. Check Pause monitoring during maintenances

    3. Image Registry

      1. Description: URL to the Image Registry

      2. Select Statuspal monitoring

      3. Method: HEAD

      4. Ping url: URL to the Image Registry

      5. Check Automatically create incident

      6. Check Pause monitoring during maintenances

    4. Logging

      1. Description: URL to the Logging

      2. Select Statuspal monitoring

      3. Method: HEAD

      4. Ping url: URL to the Logging + /app/kibana

      5. Check Automatically create incident

      6. Check Pause monitoring during maintenances

    5. Networking

    6. Ingress