A near-death experience
My FreeNAS based NAS has had an interesting life so far but I thought it had bitten the bullet today!
After experiencing yet another power outage caused by the main fusebox tripping, my various servers appeared to come back up OK with fans whirring and lights blinking. However, a couple of hours later when I attempted to access some files on my NAS, I was greeted with a “can’t connect” type error. I tried accessing the FreeNAS web GUI – no response. Next I tried pinging the box from the shell – dead. So, as a last resort I connected up the monitor and a keyboard… and found the console full of errors relating to mount failures and unrecoverable errors.
To cut a long story short it looks like the power failure corrupted the FreeNAS OS install on the flash drive (and after a bit of Googling it sounds like this happens more often than you would hope!). Given that my RAIDZ array is separate from the OS install being split across the 4 Samsung 2TB drives, I was hopeful that I would be able to reinstall FreeNAS on the flash drive and restore the previously configured ZFS volume. Unfortunately I’d not got round to creating a backup of the FreeNAS configuration (tut tut) so I would have to configure it all up again by hand.
The phoenix rises from the ashes…
After plugging in a brand new 4GB flash drive and running the CD based installer again, I had a fresh vanilla FreeNAS install configured with the same network settings as previously. I was then able to access the web GUI and start to restore what I could remember of the previous configuration. First I created the users and groups required and then performed an auto-import of the ZFS volume – which worked flawlessly! Very nice.
After reconfiguring the missing CIFS and AFP shares (including the one required for my iMac Time Machine backups), enabling SSH and installing my own SSL and SSH keys I was more or less back to the state I was in before. Woo hoo!
So what I have I learnt from this?
Well, several things really, including:
- I must get my NAS box connected to my APC UPS (the reason I haven’t so far is just laziness)
- I must make a backup of my FreeNAS configuration in case I need to do this again
- I have an even greater respect for the resilience of ZFS volumes
- I must get my house electrics sorted once and for all!