Backup and Restore
In our base setup, OpenShift is backed up on various stages: All nodes/masters are backed up, as well as all OpenShift resources. Backups are stored and can be accessed with burp, for further information like retention times and how to restore single files see Backup (BURP).
For a documentation on what is backed up see Backup and OpenShift 3 Backups.
Restore
The pre-backup script on each OpenShift master dumps and leaves all OpenShift objects into /var/lib/openshift-backup
, so if you want to restore something that was deleted/changed since them, check there first.
-
The top level contains one JSON file for each object type (
objects-*.json
). -
The
split/
sub directory contains a folder for each OpenShift namespace, containing all objects for this namespace.
To restore the missing OpenShift objects, simply use oc apply
or oc create
to recreate the object.
oc apply -f /var/lib/openshift-backup/split/cust-app/objects/Route.json
Extracting single objects from backup files
In many cases, only a single object needs to be restored. Use jq
to extract the one you want:
jq -r '.items[] | select(.metadata.name=="my-lost-secret")' \
/var/lib/openshift-backup/split/cust-app/objects/Secret.json \
> /root/restore-my-lost-secret.json
# or pipe to "oc apply -f-" to apply directly
Reattach PVs
If you restore a PVC from a backup, you must manually reattach its PV.
- CAUTION
-
This method only works if the Gluster volume(s) in question has not been recycled and bound by another PV!
If the PV was already recycled, you’ll have to create a new Gluster volume and do a manual file restore from backup using burp.
Check whether the PV is in "Failed" state and therefore save to reattach:
oc get pv/gluster-pvXX
# NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM
# gluster-pvXX 1Gi RWO,RWX Recycle Failed cust-app/data
Get the UID from the new/restored PVC:
oc -n appuio-keycloak get cust-app/data -o json | jq -r .metadata.uid
Patch the PV to associate it with the new PVC:
oc patch pv/gluster-pvXX -p '{"spec":{"claimRef":{"uid": "<UID from last step>"}}}'
This only works if the PVC is in the same namespace and has the same name. If any of those changed, oc edit the PV instead and update spec.claimRef accordingly.
|