Sunday, January 25, 2009

RAID5 Fun

After taking a very long haitus from writing this blog I've come back to relate my RAID5 expansion tale.

Background:

Last year ( June or July I think ). A few of us at (then) work were kicking around the idea of creating dedicated file servers. I have been wanting to create a dedicated NAS+RAID setup for some time. The feeling gets stronger every time I lose a crap-load of data when an HD blows up. We found a really good case on sale at newegg and the buying spree starts. The case, an Antec 300 would do well because of the huge 120MM fans in the front. These are required with the amount of drives I want to eventually have. Fast forward to the end, I turned my old 1.3Ghz Athlon media center into a file server. Current stats are:

1.3Ghz Athlon T-Bird
768MB SDRAM ( A lot for a computer of this age and RAM type ).
1.5TB of RAID5 space ( 3x750GB drives)

Current:

I filled up that 1.5TB pretty quickly. I finally broke down and bought a 4th 750GB drive this weekend. One of great things about software RAID is that you can add drives to it WITHOUT having to reformat ( or technically even unmount the filesystem). By adding a fourth drive, the space efficiency of RAID5 really starts to shine. From 4x750GB drives, I'll have 2.2TB of usable space. Adding the drive to the array was as simple as installing the drive, creating a LINUX RAID paritition ( 0xFD type), and adding it to the raid.

First you add the new drive as a spare drive:

sudo mdadm --add /dev/md0 /dev/sdd1

Then GROW the raid over the new drive

sudo mdadm --grow /dev/md0 --raid-devices=4

This takes a LONG time, depending on the host computer, amount of data, etc.. My 95% full 1.5TB is currently at 1.4% with 9186mins left to go. Yes thats about 6.5 days. So my poor little computer is gonna chug for almost a solid week to grow this RAID.

I should probably explain that this is not typical and that quite a few factors are contributing to the long processing time.

1. Slow processor: While resync'ing the raid is not computationally difficult, because of the slow processor, other processes are backing up and is not helping the situation.
2. Huge amount of data: Because there is just sooo much data to move around, the sync is going to take a while.
3. Drive layout: The layout of my RAID is ... unique. Because this is a very old motherboard, with NO SATA slots ( and all 4d drives being SATA drives), SATA-PCI cards were used, 2 in fact. These two cards are both on the PCI bus, which makes intra-RAID communication slow. This quickly saturates the bus. In fact I'm only getting about 1322K/sec transfer between drives. Thats 1.5MB/s from drives that could theoretically handle 70-80MB/s easy.
4. PCI network card: As if the PCI bus was not saturated enough, this board did not come with on-board LAN. I had to put in a PCI NIC. This further slows the sync because now all network traffic is fighting for bandwidth with the hd transfers.

Edit: I forgot to mention that after the restructuring of the computer, its now called downy. Yes, there is an uppy, its the P2 300MHz laptop that Mike gave me. I was using it as a spare terminal at Space Micro. Since I've changed jobs, and Sony gave me plenty of monitors, uppy is sitting in its dock, waiting to be used again.

No comments: