Author / Darren

by Darren

RAIDZ resilience put to the test on my new NAS!

In a previous post, I discussed building a NAS device for my home network using an HP Microserver kitted out with 4 x Samsung F4 2TB drives and 8GB of RAM, and running the FreeBSD-based FreeNAS OS. One key objective was to build in a high degree of resilience and so I chose a ZFS RAIDZ2 disk configuration which would tolerate up to 2 concurrent drive failures while still maintaining integrity of the data. At the time, I didn’t fully understand what a real life drive failure would look and feel like… but that was soon to change!

Houston, we have a problem

The NAS had been up and running for a couple of days with no problems at all. I hadn’t started copying real data across to the device but had played with using the web GUI, setting up some ZFS datasets and associated CIFS and AFP shares, some test transfers from different computers and accessing the device using SSH. All seemed very well.

It was while logged onto the box with SSH that I first noticed strange problems. For some reason, the connection would be terminated and at the same time the web GUI would stop responding. Then, a few seconds later I would be able to connect again and the whole cycle would repeat. I also noticed that if I put my ear to the box (it is pretty quiet under normal operation) I could hear what sounded like a drive repeatedly spinning up and down. That noise fills me with dread when it’s not something you specifically want to happen.

I then decided to restart the box and watch the boot sequence from the console. On reboot the first thing that was apparent was that it was taking a long time, and the cause seemed to be detecting the drives. Although the first 3 of the 4 drives were detected OK, albeit slowly, it refused to recognise the 4th drive. This seemed consistent with the unexpected drive noise and suggested the 4th drive was having problems.

RAIDZ to the rescue

At this point I decided to shut the box down again, pop out the 4th drive and reboot to see what happened. On restart the box booted quickly without any problems and seemed stable once up. The web GUI alert indicator stated that the volume was in a degraded state and viewing the disks making up the RAIDZ2 volume confirmed that the 4th drive was missing. However, throughout this all the data was accessible even though the volume was in a degraded state. So, RAIDZ was doing its job!

Testing the suspected failed drive

I didn’t want to believe that a brand new drive could be suffering a failure so I decided to shut the box down again, put the 4th drive back in and then reboot to see the result. At the same time I removed and replaced each drive in its caddy making sure that the connection was solid, and also double checked the connections of all cables onto the motherboard, particularly the single heavy braided cable cluster for the hard drives. On restart, it booted up just fine – and this was with the suspected failed drive back in!

Once the box was back up and stable I decided to run a sequence of S.M.A.R.T. self-tests on the 4th drive. The first short test only took a couple of minutes and came back all OK. The second test was a long one and took several hours to complete. When I checked the results the following morning, everything was reporting OK!

Panic over?

So, could it be that the drive is actually perfectly OK and the problems were down to a different cause? Possibly a dodgy drive caddy connection, or a loose cable connection?

I will definitely keep a very close watch on the system over the next few days.

But one thing positive has come out of this – RAIDZ is doing its job and gives me a fair bit of confidence that my data is safe in the event of a drive failure 🙂

by Darren

Building a NAS using a HP Microserver, FreeNAS and ZFS

I’ve been wanting to setup a home NAS (Network Attached Storage) solution for a while on which to store the masses of media files, documents, backups etc. we have accumulated as a family rather than having it all spread across numerous computers, mobile devices and external hard drives. I’ve toyed with the idea of getting one of the higher spec off-the-shelf BYOD (Bring Your Own Disk) boxes such as a Synology DiskStation, Drobo FS, QNAP or Netgear ReadyNAS and also as an alternative I’ve considered expanding my existing HP ProLiant ML115 G5 server with some more disks and using that for NAS.

After problems in the past with a very near miss of losing gigabytes of irreplaceable data (thank you very much ABC Data Recovery!) I was very keen on a fault tolerant system which could keep my data safe in the event of losing one or maybe two drives out of a multi-drive array. (Of course, this wouldn’t be a replacement for true external backups but it would give me some peace of mind that my data has a relatively high degree of resilience.) This requirement suggests some sort of mirrored RAID setup, and the Drobo implementation of this sounded particularly tempting especially as you can mix and match drives of different sizes, expand the array up to the maximum number of drive bays available and also swap out and add drives on the fly while the NAS is still functioning. But nice as they are, Drobos are still pretty expensive. As a slightly cheaper option I was looking at the Synology DiskStation devices and I liked what I saw (after having first hand experience of setting up a DS211 for a friend). But, I was still leaning towards a more homegrown and probably cheaper solution…

…and along came the HP Microserver!

I’ve been impressed with HP kit for a long time, particularly the business oriented stuff. My trusty ProLiant ML115 server has been running faultlessly under my desk at home for the last few years hosting various mail accounts, low-traffic web sites, development source control repositories and other stuff. So, I happened to be browsing the Ebuyer online shop and saw that HP were doing a £100 cashback offer on the HP ProLiant Athlon II Neo N36L Microserver with 1GB RAM (expandable to 8GB), 250GB hard drive (with a further 3 drive bays unpopulated), on-board NC107i gigabit ethernet and 7 x USB sockets in a well engineered micro tower case. This meant it would cost a mere £124 after cashback – an absolute steal!

After reading lots of happy customer comments on the Ebuyer site saying how well it made a NAS box when running something like FreeNAS, and with the cashback offer deadline fast approaching, I decided it was too good an opportunity to miss and I ordered one along with 8GB (2 x 4GB) of Kingston RAM, 4 x Samsung SpinPoint F4 2TB 5400rpm 32MB cache hard drives and a Sony Optiarc 24x DVD re-writer optical drive.

One other feature worth mentioning is that the motherboard has an easily accessible internal USB socket which is ideal for plugging in a USB flash drive from which the base OS can be run.

FreeNAS & ZFS: The last word in filesystems

FreeNAS is a FreeBSD based platform which is a very popular choice for home built NAS systems. It is built on a solid FreeBSD OS with a handful of services such as CIFS, NFS and AFP sharing, FTP, SSH and features a nice browser based administration GUI. It’s also small enough to be installed on a small USB flash drive which would allow my NAS to boot from the internally installed flash drive leaving the hard drives free for storage only.

One major selling point for FreeNAS / FreeBSD is its support for the ZFS filesystem developed by a very talented team of engineers at Sun Microsystems (now Oracle). If you’ve not heard of ZFS before I recommend you take a look at this presentation by the team who developed it to learn about some of the cool and innovative technology it includes. For me, some of the most compelling features are those related to data integrity, resilience and self-healing in a degraded volume, and by using one of the ZFS custom RAID implementations – RAIDZ1, RAIDZ2 or RAIDZ3 – I would get Drobo-like multi-drive resilience in the event of the loss of 1, 2 or 3 drives respectively.

For my system with 4 x 2TB drives installed, I decided to go with a RAIDZ2 layout which would give me 3.6TB of usable storage with a fault tolerance of 2 drive failures at once. I feel that this gives me a good balance of storage vs fault tolerance given the number and size of drives used.

Installation & setup

After unboxing the HP Microserver I was immediately impressed with its build quality and attention to detail. The server itself is about 210mm (W) x 270mm (H) x 250mm (D) and is of a sturdy construction. The lockable metal front door is perforated to aid airflow and opens to reveal the 4 hard drive bays with removable caddies (the first of which is populated with the 250GB drive included). The spec for the server states that these drives are not hot-swappable but I’ve read that with the correct drivers they can be hot-swapped.

Above the front door is the optical drive bay together with the on/off switch, 4 x USB sockets and LEDs for network and drive activity. There’s also a large HP logo between the front door and optical drive which lights up blue when the server is on, which looks quite cool! The top cover of the server which wraps over and down the top front section of the case slides off to reveal the optical drive bay and is secured with a thumbscrew on the back.

Another small indication of the attention to detail is the inclusion of a set of screws for mounting hard drives in the drive caddies and also for mounting the optical drive which are fixed to the inside of the front door together with a tool for fitting them. A nice touch which means that the server is completely tool-less during set up.

In order to replace the single factory installed 1GB DIMM with the 2 x 4GB DIMMs I had to slide the motherboard out from the base of the server. This was a little tricky involving disconnecting a handful of connectors, some of which are quite stiff, and easing the motherboard out enough to expose the DIMM slots, but it was no more difficult than working on other small form factor PCs.

The final step was to plug in the 4GB USB flash drive onto which I installed FreeNAS 8.0.2-RELEASE (the latest release version available at the time).

After configuring the RAIDZ2 storage (3.6TB usable storage and 2 drive failure resilience) and setting up some CIFS shares and permissions, the NAS was ready for use.

In summary – a poor man’s Drobo FS!

Now that it’s complete, I have a NAS with 3.6TB storage and Drobo-like resilience features for a fraction of the cost. Time will tell how happy I am with this but if the numerous online testimonials are anything to go by, I’m quite optimistic that this will be a good NAS solution.

by Darren

More on the ground source heat pump ground loop installation

Following on from my previous post about the work starting on the installation of the ground loop for our ground source heat pump system, here are a few more timelapse videos of the groundworks.

Now that the work is almost complete, one comment I will make is that the scale of the work involved is much bigger than I had originally expected. I knew the numbers involved – 150m of trench in total, 1m wide by 1m deep, 60 tonnes of sand etc. – but until you actually see how much soil is moved and effort involved, you don’t appreciate the scale of it.

Having said that, the guys doing the work have been very efficient and the paddock is well on its way to being back to its state before, minus the grass of course which will have to be grown from seed again once the work is complete.

by Darren

The ground source heat pump ground loop gets installed!

If you’ve read any of my previous posts, you’ll know that we’re installing a ground source heat pump as part of our barn conversion. Well, the installation of the ground loop collector pipes started in the last couple of days and I siezed the opportunity to shoot some short timelapse videos of the work being done.

The videos linked below show how the 50m x 1m x 1m trenches are dug and how the 200m of pipe loop in each trench is laid as 1m diameter coiled “slinkies” on a bed of 100mm sand. The videos are only short (they were shot with a 15 second interval between each timelapse frame) but are HD so make sure to select the 720p HD option on playback.


by Darren

My journey into the world of green energy microgeneration and ground source heat pumps…

Over the last few months I’ve been researching and planning a reasonably large building project to convert a thatched barn adjoining my house into additional living space. This new living space will obviously require space heating and domestic hot water (DHW) and so part of the project research has been concerned with investigating and choosing the system to be used.

As our village is off the mains gas grid, the main options open to use are:

  • oil
  • gas (LPG)
  • electric
  • some form of renewable such as air/ground source heat pumps (GSHP), solar thermal, solar photovoltaic (PV), biomass or a combination of these

Our current heating and DHW system is based around electrically heated water driving a traditional wet radiator distribution system. Although powered by electricity, this system has operated relatively efficiently over the last 12 years due to it using a special low rate tariff for the majority of the time. However, this is a bespoke system produced by a company that went out of business several years ago so knowledge and spare parts are pretty scarce, and this coupled with the fact that it would not be large enough to satisfy the requirements of the new complete house means that continuing to use this system would not be viable. We could choose to continue using the existing system for the old part of the house and install a completely new, separate system for the barn conversion but I feel the right thing to do would be to install a completely new system to cover the whole house, old and new. Scared by the thought of ever increasing fuel costs, the idea of a green / renewable energy based system started to appeal more and more. When also taking into account the generous benefits of the Renewable Heating Premium Payment (RHPP) and Renewable Heating Incentive (RHI) schemes (more on this later), it starts to sound even more attractive.

The ground source heat pump

Looking at our particular situation the main technology standing out was a ground source heat pump (GSHP). I had vaguely heard of this technology before but after doing a lot of research it became apparent that this would be ideal for our requirements, particularly because we have a large paddock immediately adjacent to the house and garden in which the ground loop collector pipes could be buried. If you are unfamiliar with what a ground source heat pump is, here is a very brief overview…

The ground below us stores energy from the sun as heat and from a depth of around 1m below the surface the temperature remains pretty constant all year round (between around 7°C to 13°C in the UK). It is possible to extract some of this heat energy using liquid filled collector pipes buried in the ground. The liquid is pumped through the pipes and this warms up as the heat from the ground is transferred to the liquid. Only a few degrees difference in temperature between the flow and return is enough to drive a heat pump unit which can be simplistically described as a refrigerator in reverse! It uses a system of compressors, condensers and heat exchangers to convert the small temperature differentials in the liquid coming from and going to the ground collector pipes (low grade heat) into hot water which can be used for space heating and domestic hot water (high grade heat). Once the heat energy has been extracted from the collector pipe liquid, the cooler liquid is pumped back into the ground collector pipes for the whole process to be repeated.

So, the only “fuel” used in this system is the solar energy extracted from the ground (which is free to us!) and the electricity used to drive the heat pump system. For a system sized correctly and operating efficiently you can expect it to be 400% efficient, that is for every one unit of electricity used to power the system you will get four units of heat energy out of it. Sounds good, doesn’t it! The heat pump can generate hot water for both heating and DHW. It works best driving underfloor heating due to the lower maximum temperature of the hot water it produces (up to 65°C) when compared with traditional carbon fuel based water heaters. However, it can drive radiator based distribution systems albeit at a lower temperature than traditional installations.

Ground loop collector pipes

There are different strategies for the installation of ground collector pipes depending on the land available. Possible options are:

  • straight pipe loops in horizontal trenches
  • coils of pipes, also known as “slinkies”, in horizontal trenches
  • “compact collectors” (which look like plastic radiator panels) oriented either horizontally or vertically in horizontal trenches
  • straight pipe loops in deep vertical boreholes (anything up to 200m in depth!)
  • horizontal pipe loops under water (in a lake or river)

The most common approach is to use horizontal trenches of either straight pipe loops, where enough land is available, or slinkies where there is less available space. In our situation, the most cost effective and suitable option is to use slinkies.

The heat pump and cylinder

The heat pump itself is typically a tall fridge/freezer sized unit with various ancillary expansion vessels and pipework. The flow and return pipes to and from the ground collector pipe system connect into the heat pump as do the heating and DHW pipes. A separate highly insulated cylinder stores the heated water ready for its use. One attractive option for the cylinder is that it can incorporate a separate pipework coil internally which can be connected to a solar thermal installation. This means that hot water from the solar thermal panels can feed into the heating/DHW system thereby offsetting the amount of heat required from the GSHP which in turn increases the overall efficiency of the system (remember, the solar thermal heated water is free – ignoring the installation cost – too!)

Choosing a supplier and a specification for the system

Having decided that a GSHP was the technology to go for the next decision was to find a supplier and installer. I was very keen to find a company who had significant experience of this technology, a good reputation and good levels of support as it is vital that the system is sized and installed correctly for it to operate efficiently. The company would also have to be MCS (Microgeneration Certification Scheme) accredited in order for the installation to qualify for the various government grants available.

After a lot of research I decided to go with ICE Energy given their excellent reputation and long history of supplying and installing GSHP systems. My ICE Energy sales contact Richard Floyd was very knowledgeable and helpful throughout the whole process and provided answers to my many (and there were a lot!) and varied questions. After an initial assessment of the requirements, including details of the property, it’s construction, levels of insulation, number of rooms, ratio of space heated by radiators to that to be heated by underfloor heating, land available for ground collector pipes etc. Richard produced a specification for a system comprising an 11kW IVT Greenline HT+ E11 heat pump, 600m of ground collector pipe installed as 3 x 50m trenches of slinkies and an IVT STS280 280 litre cylinder with solar coil. A further site survey was carried out shortly afterwards by a very knowledgeable and friendly engineer Adrian Rawle to ensure that the initial system specification was suitable and appropriate, and to finalise any fine detail specific to my particular installation. ICE Energy are responsible for the specification of the system, the supply of all components, unlimited technical support during installation and final commissioning of the system once installation is complete. The actual installation of the ground collector pipes will be carried out by my main building contractor responsible for the barn conversion and the installation of the heat pump, cylinder and associated plumbing and electrics will be carried out by the plumbing and electrical sub-contractors working on the barn conversion.

Government grants – RHPP and RHI

It’s worth saying something about the government grants associated with the installation of renewable heating systems as it was a big factor taken into consideration when determining what system and technology to go for. The UK government has made a big commitment to green and renewable energy and one aspect of this is encouraging the take-up of such technologies in the commercial and domestic sectors. For domestic installations this translates into providing two grant and subsidy schemes known as the Renewable Heat Premium Payments (RHPP) and the Renewable Heat Incentive (RHI). RHPP is a short term scheme (scheduled to finish in April 2012) which provides a grant towards the installation of qualifying renewable heating systems. Ground source heat pumps are a qualifying technology which attracts a grant of £1,250. More interestingly, RHI is a longer term scheme scheduled to start in October 2012 which will result in guaranteed index linked payments for a period of 23 years (in the case of GSHP installations) based on the amount of heat energy a qualifying system is generating. The final tariffs have yet to be determined but the estimate at this time is that our particular installation would attract payments of between £1,500 – £2,000 a year!

What next?

The barn conversion has already started (look out for details of progress in other blog posts) so at some point in the next few weeks the work will start to install the ground loop collector pipes in the paddock.

Watch this space!

by Darren

The Coach House thatched barn conversion

Last year we were successful in gaining planning approval to convert the early 19th century Grade II listed thatched timber framed barn adjoining our house into additional living space. This will double the size of our home downstairs and give us a new enlarged hall, additional open plan living and dining space, a new kitchen and utility room, and will allow us to convert the existing kitchen to a new office/bedroom with en-suite toilet/show wet room.

One of the conditions attached to the planning approval required the production of a photographic record of the barn (on 35mm black & white film!) before the conversion takes place. I undertook this work with the help of my good friend and neighbour (and resident photography expert) and the results form an excellent showcase of the building and its character. A gallery of images taken from scans of the film negatives can be viewed at the following URL:

http://www.darrenscott.com/galleries/barn

Work is due to start on the conversion very soon so I’ll post more about the work and progress over the next few months.

by Darren

Dazza recommends… Belkin Snap Shield case for Apple iPad 2

If you’re looking for a hard case to protect the rear of your iPad 2 I can highly recommend the Belkin Snap Shield.

It’s a snug fitting hard case made out of a soft touch plastic which clips around the edges of the iPad, providing plenty of protection to the rear of the iPad without getting in the way of using it. It’s also designed to work seamlessly with the Apple Smart Cover and comes in several different semi-translucent colours.

Highly recommended.

by Darren

My private Avro Vulcan XH558 fly-past

So, there I am working away in my office at home when I hear a distant whining coming from the skies to the North.

I’ve been casually watching the stream of tweets from the XH558 crew as the worlds only remaining flying Avro Vulcan delta wing bomber departs from Waddington heading down to Yeovilton for this weekend’s airshows. Could the noise I’m hearing be XH558?

I run outside and stand there with the neighbours looking up at the sky as the mighty Vulcan roars into view – at what must be only 800ft or so height! OMFG!

It’s almost as if I’d phoned the Vulcan operations team and had this conversation:

“Ah hello, is that Avro Vulcan XH558? It is, oh smashing. Is there any chance you could do a low-level fly-past directly over my house please, while you’re en-route to Yeovilton? You can! Really? Oh that is splendid. Thank you very much. Have a safe trip.”

Once she passes overhead I stand there watching her slowly disappear into the distance to the South, black smoke trailing behind her. What a majestic sight.

Long may she fly, Avro Vulcan XH558.

by Darren

PalmerSport 2011

I’ve just come back from my second full day event at Jonathan Palmer’s PalmerSport complex at Bedford Autodrome – and it was absolutely awesome! Even better than the first time I visited 2 years ago.

For anyone who’s interested in driving and motorsport, this has got to be the best track based driving event available to the public. The first time I visited a couple of years ago it was on a corporate hospitality day, but this time round I paid for it myself. It’s not cheap at just short of a grand, but worth every penny in my opinion. It’s not like your run-of-the-mill “track day experience” type of day, this is full-on driving on 4 dedicated tracks in 6 highly tuned race cars (and a Land Rover Defender off-road) accompanied by highly competent instructors (Lewis Hamilton was a former instructor there!) who push you to your limits.

The cars for the 2011 line-up are:

  • Formula Jaguar “slicks and wings” single seater
    (3.0 litre V6, 250bhp, 6 speed paddleshift, 0-60mph 3.1 secs, top speed 170mph)
  • Palmer Jaguar JP-LM 2 seat Le Mans sports car prototype
    (3.0 litre V6, 265bhp, 6 speed paddleshift, 0-6mph 3.4 secs, top speed 167mph)
  • BMW M3 GTP
    (4.0 litre V8, 420bhp, 7 speed paddleshift, 0-6mph 4.6 secs, top speed 185mph)
  • Porsche 911 JP3
    (3.6 litre flat six, 320bhp, 5 speed paddleshift, 0-60mph 4.5 secs, top speed 175mph)
  • Renault Clio Cup racer
    (2 litre 4 cylinder, 205bhp, 6 speed paddleshift, 0-6mph 5.2 secs, top speed 142mph)
  • Caterham 7 Superlight
    (286bhp/tonne, 6 speed manual transmission, 0-60mph 4.8 secs, top speed 130mph)
  • Land Rover Defender

The line-up is almost the same as the last time I visited with one exception – the addition of the BMW M3 GTP to replace the Jaguar XKR I drove last time. I’ve got to say the M3 is a huge improvement over the XKR which I didn’t particularly get on with last time.

Another addition over previous years is that of on-board video recording in several of the cars. At the start of the day you are given a PalmerSport branded USB memory stick which you keep with you throughout the day and hand over to the instructor when it’s your turn to drive one of the cars with video capability, namely the Formula Jaguar, JP-LM, BMW M3 GTP and Caterham 7 Superlight. The video system has forward and inward rear facing cameras and also records telemetry data which is then superimposed over the video when playing it back using some player software provided on the memory stick.

The mix of people on this latest visit was very different to that of my last visit. As the previous event was a corporate hospitality day, there was quite a range of driving skills present, with the bias probably to the less skilled. This time round as the group was significantly smaller with more individuals as opposed to corporate hospitality attendees, there was a higher percentage of “petrol heads” and the skill level was much higher. I felt I’d done OK but maybe not as good as last time, however when I got home and compared my lap times with those from last time I was surprised to find that I’d got higher scores this time round (apart from on the Land Rover Defender, but I can live with that!) I came 4th overall in the JP-LM and I’m pretty sure I was around that ranking in the Formula Jaguar (they don’t do a ranking in that car for some reason but comparing my best lap time and telemetry trace with the other top drivers I was up there with them).

I’ve uploaded a few photos from the day to Flickr – you can take a look at my photostream on this blog site or directly on the Flickr site.

All in all this was another amazing visit and I can’t recommend it highly enough. Hopefully I’ll be visiting again in the future 🙂