An infiniband switch died during the holydays, blocking part of Cascade cluster nodes to access /scratch/Cral.
We don't have that kind of replacement part on site, so it will take some time…
Welcome to another year.
EDIT: switch is not dead. There's a glitch between hardware, firmware, kernel and network software. It might need some upgrades and a general reboot. Joy.
EDIT: It was a f*$!§?$! fiber! New year resolution “never trust a network fiber”.