Uninstallation on cloudscale.ch

Steps to remove an OpenShift 4 cluster from cloudscale.ch.

  • The commands are idempotent and can be retried if any of the steps fail.

  • In the future, this procedure will be mostly automated

Prerequisites

Cluster Decommission

  1. Export the following vars

    export CLOUDSCALE_TOKEN=<cloudscale-api-token> # From https://control.cloudscale.ch/user/api-tokens
    export CLUSTER_ID=<lieutenant-cluster-id>
    export TENANT_ID=<lieutenant-tenant-id>
    export REGION=<region> # rma or lpg (without the zone number)
    export GITLAB_TOKEN=<gitlab-api-token> # From https://git.vshn.net/profile/personal_access_tokens
  2. Setup Terraform

    Prepare terraform

    # Set terraform image and tag to be used
    tf_image=$(\
      yq eval ".parameters.openshift4_cloudscale.images.terraform.image" \
      dependencies/openshift4-cloudscale/class/defaults.yml)
    tf_tag=$(\
      yq eval ".parameters.openshift4_cloudscale.images.terraform.tag" \
      dependencies/openshift4-cloudscale/class/defaults.yml)
    
    # Generate the terraform alias
    alias terraform='docker run -it --rm \
      -e CLOUDSCALE_TOKEN="${CLOUDSCALE_TOKEN}" \
      -e TF_VAR_ignition_bootstrap="${TF_VAR_ignition_bootstrap}" \
      -w /tf \
      -v $(pwd):/tf \
      --ulimit memlock=-1 \
      ${tf_image}:${tf_tag} terraform'
    
    export GITLAB_REPOSITORY_URL=$(curl -sH "Authorization: Bearer ${COMMODORE_API_TOKEN}" ${COMMODORE_API_URL}/clusters/${CLUSTER_ID} | jq -r '.gitRepo.url' | sed 's|ssh://||; s|/|:|')
    export GITLAB_REPOSITORY_NAME=${GITLAB_REPOSITORY_URL##*/}
    export GITLAB_CATALOG_PROJECT_ID=$(curl -sH "Authorization: Bearer ${GITLAB_TOKEN}" "https://git.vshn.net/api/v4/projects?simple=true&search=${GITLAB_REPOSITORY_NAME/.git}" | jq -r ".[] | select(.ssh_url_to_repo == \"${GITLAB_REPOSITORY_URL}\") | .id")
    export GITLAB_STATE_URL="https://git.vshn.net/api/v4/projects/${GITLAB_CATALOG_PROJECT_ID}/terraform/state/cluster"
    
    pushd catalog/manifests/openshift4-cloudscale/

    Initiate terraform

    terraform init \
      "-backend-config=address=${GITLAB_STATE_URL}" \
      "-backend-config=lock_address=${GITLAB_STATE_URL}/lock" \
      "-backend-config=unlock_address=${GITLAB_STATE_URL}/lock" \
      "-backend-config=username=$(whoami)" \
      "-backend-config=password=${GITLAB_TOKEN}" \
      "-backend-config=lock_method=POST" \
      "-backend-config=unlock_method=DELETE" \
      "-backend-config=retry_wait_min=5"
  3. Delete resources from clouscale.ch using Terraform

    terraform destroy
  4. After all resources are deleted we need to remove the bucket

    # Use already exiting bucket user
    response=$(curl -sH "Authorization: Bearer ${CLOUDSCALE_TOKEN}" \
      https://api.cloudscale.ch/v1/objects-users | \
      jq -e ".[] | select(.display_name == \"${CLUSTER_ID}\")")
    
    # configure minio client to use the bucket
    mc config host add \
      "${CLUSTER_ID}" "https://objects.${REGION}.cloudscale.ch" \
      $(echo $response | jq -r '.keys[0].access_key') \
      $(echo $response | jq -r '.keys[0].secret_key')
    
    # delete bootstrap-ignition object
    mc rb "${CLUSTER_ID}/${CLUSTER_ID}-bootstrap-ignition" --force
    
    # delete image-registry object
    mc rb "${CLUSTER_ID}/${CLUSTER_ID}-image-registry --force
    
    # delete cloudscale.ch user object
    curl -i -H "Authorization: Bearer ${CLOUDSCALE_TOKEN}" -X DELETE $(echo $response | jq -r '.href')
  5. Delete vault entries:

    # Vault login
    export VAULT_ADDR=https://vault-prod.syn.vshn.net
    vault login -method=ldap username=<your.name>
    
    # delete token secret
    vault kv delete clusters/kv/${TENANT_ID}/${CLUSTER_ID}/cloudscale
    
    # delete registry secret
    vault kv delete clusters/kv/${TENANT_ID}/${CLUSTER_ID}/registry
    
    # delete ldap secret
    vault kv delete clusters/kv/${TENANT_ID}/${CLUSTER_ID}/vshn-ldap
  6. Delete cluster from Lieutenant API (via portal)

    • Select the Lieutenant API Endpoint

    • Search cluster name

    • Delete cluster entry using the delete button

  7. Delete LDAP service (via portal)

    • Search cluster name

    • Delete cluster entry service using the delete button

  8. Remove IPs from LDAP allowlist

    • Search cluster IPs and remove those lines and any comments related.

    • Create a Merge Request and invite a colleague for a review/approve/merge

  9. Delete all DNS records related with cluster (zonefiles)

  10. Update any related documentation