How to Manually Perform a Redis Failover
In certain situations two of the three Redis nodes end up in
CrashLoopBackOff because they receive a wrong (old) leader IP address from the running Sentinel.
In the logs this might look like:
sentinel Could not connect to Redis at 10.0.0.20:26379: Connection refused
To resolve this situation, get the IP address of the running leader, connect to a Redis Sentinel Pod, start a
redis-cli and initiate the failover.
kubectl get pod -o wide kubectl exec redis-node-0 -c sentinel -it -- bash redis-cli -a $(< $REDIS_PASSWORD_FILE) -p 26379 SENTINEL REMOVE mymaster SENTINEL MONITOR mymaster <NEW_IP_ADDRESS> 6379 1