Uninstallation on Exoscale
Steps to remove an OpenShift 4 cluster from Exoscale.
|
Always follow the 4-eye data deletion (Internal link) principle when decommissioning productive clusters. |
Prerequisites
-
Exoscale API key
-
docker
-
jq
-
yq
yq YAML processor (version 4 or higher) -
exo
>= v1.28.0, Exoscale CLI
Cluster Decommission
-
Export the following vars
export EXOSCALE_ACCOUNT=<exoscale-account> export EXOSCALE_API_KEY=<exoscale-key> export EXOSCALE_API_SECRET=<exoscale-secret> export EXOSCALE_REGION=<cluster-region> export CLUSTER_ID=<cluster-name> # From https://git.vshn.net/-/profile/personal_access_tokens export GITLAB_TOKEN=<gitlab-api-token> export GITLAB_USER=<gitlab-user-name> # For example: https://api.syn.vshn.net # IMPORTANT: do NOT add a trailing `/`. Commands below will fail. export COMMODORE_API_URL=<lieutenant-api-endpoint>
-
Compile cluster catalog to get input variables
commodore catalog compile ${CLUSTER_ID}
-
Configure Terraform secrets
cat <<EOF > catalog/manifests/openshift4-terraform/.env EXOSCALE_API_KEY EXOSCALE_API_SECRET EOF
-
Setup Terraform
Prepare Terraform execution environment# Set terraform image and tag to be used tf_image=$(\ yq eval ".parameters.openshift4_terraform.images.terraform.image" \ dependencies/openshift4-terraform/class/defaults.yml) tf_tag=$(\ yq eval ".parameters.openshift4_terraform.images.terraform.tag" \ dependencies/openshift4-terraform/class/defaults.yml) # Generate the terraform alias base_dir=$(pwd) alias terraform='docker run -it --rm \ -e REAL_UID=$(id -u) \ --env-file ${base_dir}/terraform.env \ -w /tf \ -v $(pwd):/tf \ --ulimit memlock=-1 \ "${tf_image}:${tf_tag}" /tf/terraform.sh' export GITLAB_REPOSITORY_URL=$(curl -sH "Authorization: Bearer $(commodore fetch-token)" ${COMMODORE_API_URL}/clusters/${CLUSTER_ID} | jq -r '.gitRepo.url' | sed 's|ssh://||; s|/|:|') export GITLAB_REPOSITORY_NAME=${GITLAB_REPOSITORY_URL##*/} export GITLAB_CATALOG_PROJECT_ID=$(curl -sH "Authorization: Bearer ${GITLAB_TOKEN}" "https://git.vshn.net/api/v4/projects?simple=true&search=${GITLAB_REPOSITORY_NAME/.git}" | jq -r ".[] | select(.ssh_url_to_repo == \"${GITLAB_REPOSITORY_URL}\") | .id") export GITLAB_STATE_URL="https://git.vshn.net/api/v4/projects/${GITLAB_CATALOG_PROJECT_ID}/terraform/state/cluster" pushd catalog/manifests/openshift4-terraform/
Initialize Terraformterraform init \ "-backend-config=address=${GITLAB_STATE_URL}" \ "-backend-config=lock_address=${GITLAB_STATE_URL}/lock" \ "-backend-config=unlock_address=${GITLAB_STATE_URL}/lock" \ "-backend-config=username=${GITLAB_USER}" \ "-backend-config=password=${GITLAB_TOKEN}" \ "-backend-config=lock_method=POST" \ "-backend-config=unlock_method=DELETE" \ "-backend-config=retry_wait_min=5"
-
Grab location of LB backups and potential Icinga2 satellite host before decommissioning VMs.
declare -a LB_FQDNS for id in 1 2; do LB_FQDNS[$id]=$(terraform state show "module.cluster.module.lb.exoscale_domain_record.lb[$(expr $id - 1)]" | grep hostname | cut -d'=' -f2 | tr -d ' "\r\n') done for lb in ${LB_FQDNS[*]}; do ssh "${lb}" "sudo grep 'server =' /etc/burp/burp.conf && sudo grep 'ParentZone' /etc/icinga2/constants.conf" done
-
Set downtimes for both LBs in Icinga2.
-
Remove APPUiO hieradata Git repository resource from Terraform state
terraform state rm 'module.cluster.module.lb.module.hiera[0].gitfile_checkout.appuio_hieradata'
This step is necessary to ensure the subsequent terraform destroy
completes without errors. -
Delete resources using Terraform
terraform destroy
-
Use Exoscale CLI tool to empty and remove buckets
mkdir -p ~/.config/exoscale cat <<EOF >> ~/.config/exoscale/exoscale.toml [[accounts]] account = "${EXOSCALE_ACCOUNT}" defaultZone = "${EXOSCALE_REGION}" endpoint = "https://api.exoscale.ch/v1" name = "${CLUSTER_ID}" EOF # Bootstrap bucket exo storage delete -r -f "sos://${CLUSTER_ID}-bootstrap/" exo storage rb -f "${CLUSTER_ID}-bootstrap" # OpenShift Image Registry bucket exo storage delete -r -f "sos://${CLUSTER_ID}-image-registry/" exo storage rb -f "${CLUSTER_ID}-image-registry"
-
Decommission Puppet-managed LBs according to the VSHN documentation (Internal link).
Don’t forget to remove the LB configuration in the APPUiO hieradata and the nodes hieradata. -
Remove cluster DNS records from VSHN DNS