Troubleshooting Healing Volumes
Heal
To check for inconsistencies that require healing, run the command below. If there’s no problem, all three should have number of entries equal to zero.
gluster volume heal $name info
To start healing a Gluster volume execute:
gluster volume heal $name
Full heal
To check for inconsistencies that require healing, run the subsequent command. If there’s no problem, all three should have number of entries equal to zero
gluster volume heal $name info
A full heal may help in cases, Gluster still has an inconsistency after a normal heal.
This may produce a lot of IO/Traffic and Load |
gluster volume heal $name full
Transport Endpoint is not Connected
The heal info may give the following for one of the bricks:
# gluster volume heal $name info Brick storage.host123.io:/data/shared_ssd/gluster-pvxx/brick Status: Transport endpoint is not connected Number of entries: -
In this case try to force-start the brick:
gluster volume start gluster-pvxx force
If it is currently running, you may need to stop it first:
gluster volume stop gluster-pvxx
After this operation, you must restart any pod that has mounted this pv as a volume.
Upstream documentation:
Split-Brain
gluster v heal gluster-pvxx split-brain source-brick storagex.lab.zrh.appuio.ch:/data/gluster-pvxx/brick $FILE Healed gfid:53c85f9b-8761-4022-8d9f-afe21517f176. Status: Connected Number of healed entries: 1 gluster v heal gluster-pvxx split-brain latest-mtime $FILE gluster v heal gluster-pvxx split-brain bigger-file $FILE
Upstream documentation:
If you require to mount all the bricks directly to be able to compare them you may use the splitbrain tool.