Anyone tried this before?

[djackson]

Hi,

Have been using the XP RAID-5 mod described on Tomshardware since Feb/Mar. Was using it on an old MSI dual P3 system with 4x80G seagate drives (already had 2x80G and was BIOS limited to 120G anyway). Only prob I had on that system was a dodgy battery and the BIOS forgetting its setting. This resulted in the RAID-5 being rebuilt to often, but otherwise OK. Was not after speed, but storage. OK so 240Gb might not be huge, but it works for me.

Have since transplanted the 4 drive array to my newer P4 system without a hitch. Have 2 drives connected to an on-board IDE channel, and 2 others connected to cheap PCI IDE controller. All worked fine after importing the foriegn disks.

If I can offer any other info, feel free to ask.

[circumflatulator]

djackson, Thanks for the info on your experience with the XP RAID-5 mod. Its comforting to know that you were able to transplant the intact array to a different machine. And like you, I am not after any of the read-speed benefits of RAID-5. I’m after data storage with the ability to recover from a hard drive failure without losing any data. And with HD space becoming so cheap (250GB=$90.00US) a large RAID-5 media file server made out of a standard PC is the budget way to go. No more having to store/index/catalog/find CDs or DVDs or load them into players. Just a couple of mouse clicks and the media is playing right on the big screen. A very convenient setup the wife, grandkids and I have been enjoying.

[shaggy8675]

Good to hear everything seems to work out.

Sorry i didn't respond earlier, but it's my holliday hehe

Let me know if you get it running ok, and if you got problems running ATTO bench contact me, it's verry simple. ( don't forget to set it at 32MB, and it's ok )

Ohh, ps.

Indeed this function is a piece of code identical to the server versions of windows wich does have standard raid 1 and 5.

The hack, just let's windows XP think, it's Windows Server...

Nothing fancy about it, just a small change.

Since much of the source code is identical...

I know real hardware raid is much better, but you can't get it cheaper than this.

Most people don't have the money for real hardware raid like a 400mhz, 1Gb 3Ware card...

But then again, most people use just standart Mainboards, wich do not go over the 133 or 150Mb/sec barrier of their board ( ATA S-ATA etc. )

To get the best out of that you need PCI-X 133 Mainboards, but that's to expensieve for most normal users.

When using raid for speed get 2 fast hdd's like the raptors or Maxtor Plus series, raid0 and you will get your mainboard full at around 120mb/sec it won't go anny faster no matter how expensieve your hardware card is...

I still use software raid on my new Mainboard, only now the S-ATA chips all have raid functions.... so i use them because the setup is so verry easy. ( onboard chips are also software raid...Promise, Via..all S-ATA/ATA chips aboard is Software raid )

But a cheap a** server for music, internet etc. HACK IT

Thanks for the nice replies, ( post them in my guestbook hehe )

Take care for now.

Shaggy,

www.rvandesanden.tk

www.software-raid.tk

www.motorbike-forum.com

and much much more...

Edited August 4, 2005 by shaggy8675

[circumflatulator]

Ok, sorry for the long delay in posting my experience with this. But in case anyone else is interested in doing this, here is what happened:

A friend of mine knows a fellow who works for M$ and he indicated XP is a 32-bit OS and as such can only keep track of 4,294,967,296 (FFFFFFFF base 16) sectors, which at 512 bytes per sector, equals 2,199,023,255,042 (2 TB) of storage space in any single logical drive. Thus the 2TB limit indicated in previous posts. Using 250GB drives, that means the largest RAID-5 array consists of 9 250GB drives (8X250GB for the data, and 1X250GB for the checksums). The checksum storage does not figure in the 32-bit limit calculations. These limitations do not apply with Server 2003 or other 64-bit operating systems, but that doesn’t apply to me as I want to stick with the XP system I have.

Another thing learned from him: You cannot add drives to the RAID-5 array without deleting the volume, adding the additional drive(s), and then reformatting the volume. This means all the data previously on that array would be lost trying to add an additional drive.

With the above in mind, I worked with a friend who needed to buy some more hard drive storage for his computer and he bought 9 new 250GB drives, which he kindly lent to me so I could temporarily copy all my existing data to them. Then I reformatted 9 of my existing 250GB drives into the new RAID-5 array (that took 5 hours) and then copied my data back from the borrowed drives. That whole process took a couple of days.

Observations:

The XP RAID-5 hack works great. It’s been up and in use for a month now and is perfectly stable. It does have a write-speed issue: Writing to the array is about 3 times slower than to a single drive. Read speed appears faster than from a single drive. This is fine for my use (file server for music/video/pictures etc) since I only write a file once and then read it many times when it’s played. I haven’t done the benchmark tests yet, but will post them here when I can get to it.

The CPU overhead (doing the XOR checksum calculations during writing) is about 4%, nothing to worry about. During read operations, CPU is at less than 1%. Memory page utilization and disk que length is less that using a single drive. All good!

The 9 drives that make up the 2TB array are all 250GB Western Digital, but some are IDE, and others are SATA. They are spread over 2 separate PCI controller boards and the built in controllers on the mobo. I was concerned about timing issues because of this setup, but it is working perfectly.

Another nice benefit: I used to have my movies spread over numerous drives. Quite a hassle when you are trying to find a particular film but can’t remember which drive it is on. With one huge logical drive, cataloging all media in self-descriptive categories is a snap and makes searching very intuitive.

I would also recommend that you try to build the largest RAID-5 array you can financially swing because the larger the array, the less “overhead” (seen in Disk Management) you have to tolerate for full redundant backup. With my 9 drives, 8 are holding actual data which shows as only 11% overhead in the Disk Management window. With only 3 drives, that figure would be 33% overhead (2 drives hold data, 1 for checksums).

All in all, I am VERY happy with the performance of XP with this RAID hack, and would highly recommend it to anyone needing large disk storage for media (or other ‘static’ data) files who wants online fault-tolerance without the hassle or expense of tape/CD/DVD backup. Now when one of my disk drives fail (notice I said WHEN, not if) , I WON’T LOSE ANY DATA! You just replace the failed drive with a new one and XP “rebuilds” the data that used to be there. NOTE: If you have a drive fail, be careful you remove and replace ONLY the defective drive. If you accidentally replace the wrong one, when you boot up the system will detect 2 “empty” drives in the array and it can’t recover from that much “lost” data. You then will likely lose ALL your data!

And again, thank you to everyone on this forum who helped me get the courage to try this!

[jaclaz]

It does have a write-speed issue:  Writing to the array is about 3 times slower than to a single drive.  Read speed appears faster than from a single drive. 

<{POST_SNAPBACK}>

It is unfortunately normal, it is the so-called "RAID5 write penalty"see here where it is expalined:

https://www.redhat.com/archives/fedora-list...r/msg01731.html

jaclaz

[circumflatulator]

jaclaz,

Real good info. I kind of understood the reasons for the write speed penalty, but that article explained it in detail. When I read the author’s reasons why he doesn’t like RAID-5, it really concerned me. But on further research I found a work-around for the worst of his issues: the no-checksum calculation for a RAID-5 read operation, which may cause bad data to be propagated throughout the array over time.

What I found is that (at least for Western Digital drives) a lot has changed in the 2 years since Mr. Kagel wrote his article. Specifically, WD has implemented what they call “Data Lifeguard” functions into their drives which does keep track of (among many other things) failed read parity and how many spare sectors are available should used sectors go flakey. When that number drops below 50, the drive is supposed to alert the OS and display (I’m assuming in the Disk Management window on XP) an error message. However, WD also provides a downloadable application (also called Data Lifeguard) which you run manually and it shows you all kinds of diagnostic data, including the number of “Grown-Defects" since the disk was originally manufactured. Presumable, you should run that utility every week or so if you’re not sure your OS supports the error display mentioned above.

The other issue that bothered me, having a second drive fail before you can replace the first failed drive, seems highly improbable in my situation, as all the drives in my array have been in use 24/7 for over a year. I’m assuming that this issue would only be a problem if someone built an array out of a bunch of disks purchased at the same time and from the same manufacturing lot. Manufacturing defects usually show up in the first few days/weeks of use, and are sometime called “infant-mortality” failures. Also, the MTBF (mean-time-before-failure) numbers for hard drives have gone up considerably in the last couple of years.

Another way a user could get hit with a second drive failing before the first is replaced is if he procrastinated too long before replacing the drive. I have a spare 250GB drive that I am using for video-editing (meaning it only will have temporary files on it anyway) that can be used as a “hot spare” when I need to replace a failed drive from the array.

I fully agree with Mr. Kagel’s logic: mirroring is the better way to have reliable failed-disk-data-recovery. But I would need to buy and install 7 more 250GB drives, and besides the cost, there just isn’t room in that rack-mount case (nor cooling or power supply capacity) to do that. I guess that is a tradeoff I'll have to make.

But when all is said and done, the true test of how reliable this array actually is will only be know after I’ve had my first failure and have tried to recover from it. When that happens, I’ll post the results here.

[FatalSaviour]

Serial ATA is much faster then IDE, EIDE, SCSI. Serial ATA go's upto 1.50 MBs per sec or more, that's all i know

<{POST_SNAPBACK}>

Not quite I'm afraid. SATA is slightly faster than IDE, 150Mbs vs 133. External data transfer rates make hardly any real-world difference anyway, I can only begin to see this with the 16mb cache I have on 2 maxtor's I own. As for SCSI, it can own SATA, especially as many SCSI drives are optimzed for servers running at 10k or 15k rpm, which (with the exception of the WD Raptors) is a hell of a lot faster than any SATA drive out there. Anyway, this seems to be digressing.

Is it possible to buy a new mainboard, I know you said it was a rackmount, and I'm not sure on the exact possibilities with, eg. an Nforce 4 Intel Edition, but I pretty sure that it is possible to span arrays across the PATA and SATA controllers integrated on the board. Probably not such a great idea, but just a thought....

[HyperHacker]

XP is a 32-bit OS and as such can only keep track of 4,294,967,296 (FFFFFFFF base 16) sectors, which at 512 bytes per sector, equals 2,199,023,255,042 (2 TB) of storage space in any single logical drive.

<{POST_SNAPBACK}>

That's with 512-byte sectors, though. If you used 1K sectors, you could have 4TB. Of course bigger sectors means more wasted space (no file can physically occupy less than one sector; try saving a one-byte file on a disk with 512-byte sectors, you'll see that Size On Disk is 512 bytes), but also faster access time.

[jaclaz]

@circumflatulator

Yes, I think you have done the most you can do to have data safe in a fairly economic manner.

The only thing on which I somewhat disagree is the following:

Also, the MTBF (mean-time-before-failure) numbers for hard drives have gone up considerably in the last couple of years.

I don't know the exact procedure of "guessing" (as it cannot be defined differently) MTBF, but technology in last years, as I see it, has proceeded in various directions, let's see the + and -

+ Hard drive bearings have improved

+ Hard drive motors have bettered (lower consumption)

- Magnetic density has increased by a ten times multiplier, meaning that "tracks" are more than ten times closer that they were a few years ago, which calls for ten times better precision in driving heads movements

- Rotation speed has increased, posing BIG problems in heating (I remember when hard drives were just slightly warm after working many hours/days) and in head "flying" over the platter surfaces, (this is just a guess on my part, but designing and manufacturing a head arm capable of being supported at a fraction of a millimeter over a platter rotating at 10,000 RPM and NOT landing during spinups/spindowns it's probably more difficult than a head for 4,200 RPM)

So my personal idea is that on "newish" big, fast drives even a fraction of the inevitable wear of mechanical parts will lead to a failure, whilst on "elder" smaller slower ones the same amount of wear is still inside the design tolerances.

Well, only time will tell....

@kurt476

Serial ATA is much faster then IDE, EIDE, SCSI. Serial ATA go's upto 1.50 MBs per sec or more, that's all i know

NO other type of drive is faster than "fast" SCSI ones, this is mainly due to the higher rotation speeds and the fact that the SCSI interface having it's own processor, adds a very negligible overhead on the main system's one.

See this comparison:

http://www.pugetsystems.com/articles.php?id=19

The gap is narrowing, but it's still there, the real problem is whether the much higher costs are justified.

Besides there is the point of reliability, usually SATA drives come with a 1 year warranty, see here:

http://h18004.www1.hp.com/products/servers...answers.html#1c

Q1. What is the warranty period for a SATA drive?

A1. SATA hard drives will have a one year warranty - the equivalent to what is offered on the previous generation Parallel ATA hard drives.

the same manufacturer, on SCSI drives:

http://h18004.www1.hp.com/products/servers...answers.html#4c

Q4. What is the warranty period for a SAS drive?

A4. SAS hard drives offer a three year warranty - the equivalent to what is offered on the previous generation Parallel SCSI hard drives.

As warranty is a cost for the manufacturer, if he gives a three times lasting warranty on a MORE EXPENSIVE product, it either means that you are paying the extra cost in the purchase price, or that the product is "roughly" three times more reliable, even if the above are two "half truths", SCSI comes out as "probably" 1,5 times more reliable than SATA.

(but Maxtor, to name one, has more or less the same warranty periods for both:

http://www.maxtor.com/portal/site/Maxtor/m...anty%20Periods)

So maybe I am wrong and MTBF is really increased, still I have noticed (from friends and forums) that hard drive failure seems to be quite a common occurrence, which was not just a few years ago.

jaclaz

Edited September 9, 2005 by jaclaz

[circumflatulator]

Oh, Crap.

After reading more info on RAID-5, I’m not so sure I have the reliability/backup issue covered as well as I originally thought. I’m even beginning to wonder if I’ve created a monster.

My only purpose in creating this RAID array was to achieve a “cheap automatically-backed-up data storage and serving” device that is very VERY unlikely to suffer any lost data because of a disk drive failure. That was it. Period.

The data stored on that computer is not technically irreplaceable. But for all intents and purposes, it is **** close.

The scanned family photographs (going back to the late 1800s and consisting of over 4,000 flat-photograph scans and over 6,000 35mm hi-res color slide scans) could technically be re-scanned. It only took 4 years to gather/borrow/scan/return those pictures/slides, but it is theoretically possible to re-do it all if the array were to crash.

And I still have all the original music CDs and movie DVDs, so I could re-rip (and re-encode the videos) them all - maybe 300 hours or so should do it.

And all the episodes of Leave It To Beaver and The Andy Griffith Show (now you know my age) and other such stuff from my childhood, that I have digitally recorded off the satellite, could be re-recorded - over time. MUCH much time…

So maybe I should classify my data as “virtually irreplaceable”. It would really be a personal tragedy to lose it.

When it was just stored on individual drives, if one fried – only some of my data disappeared. If the RAID-5 array goes belly-up, it’s ALL gone.

And there it all sits- existing in only ONE place - on a single 2TB RAID-5 array running on an un-impressive XP machine in my basement.

Oh yeah, I’ve done a few things to help out my chances of keeping that data intact.

I’m the only one who has write/modify privilege on the server. So other than my own stupidity or fat-finger mistakes there isn’t any way the users of that data (wife/kids/grandkids playing movies on the networked dedicated-player-computers around the house) can delete or modify it.

The server is not connected to the internet, so viruses and hackers are unable to compromise the data.

I absolutely refuse to load/install/run any 3-rd party software on that server for fear of getting it infected or having a crappy program wipe out a drive or two. Hell, I won’t even put M$ updates on it – it’s working fine as it is.

The rack-mount case has redundant power supplies, oversize triple-redundant cooling fans with 4 temperature sensors and alarms out the wazoo. A true HEPA air filter on the front of the case. A dedicated 3KVA APC UPS with 3 hours of battery-backup run time. And if the power mains to my home go out for over 3 minutes, there is an outside 15KW propane generator that automatically starts and runs most everything in the house (that was already installed for other reasons, like the un-reliable power we have out here in the sticks where I live). Even those hard drives were installed with particular care to assure excellent air flow around all 6 sides of each one.

And the ambient temperature in my basement (here in Utah, almost every home is built with an underground basement. Some folks finish theirs up for additional living space, but mine is only for the HVAC room, water heater, storage area, and shop area) never varies more than 5 degrees from 65F year round. Perfect environment for computer stuff.

And the rack-mount case itself is installed in the top 6U position (in case of water leaks in basement) of a 6-foot tall, 19-inch industrial grade rack cabinet – which itself is bolted to the concrete floor in case of earthquake (we rarely have them out here but they say a big one is overdue).

Jeez, you’d think I’d be happy with this setup and able to sleep at night knowing that my precious data is virtually indestructible.

WRONG!!!!

Starting by following the links jaclaz provided in his last 2 posts, I began finding disturbing information on the unreliability of RAID-5 arrays (the worst is: <http://www.baarf.com/>). Agreed, this information is somewhat dated since its relevance changes as the technology improves, and most of the folks who posted there are database users where the write performance of RAID-5 severely affects their machines. But I still came away with serious concern that this “monster” I’ve created is really not as safe as I had hoped.

Everywhere I read things like “the only truly safe backup method for important data is to do magnetic tape or disk, or optical-media backups on a regular basis”.

How in the hell am I supposed to “back up” 2TB of data that way? Figure the number of CDs or DVDs or tapes to do that! Plus the time commitment. Over time, just the blank media costs would go a long way to pay for a mirrored RAID array (which is supposed to be much safer than RAID-5).

Sorry for my rantings. It’s just that I feel lost – unsure that after all this, my data is really safe enough.

I know there is not a perfect answer to this problem. SysAdmins and others in the IT industry have known this for a long time. I guess if cost were not an issue, RAID-1 looks like the way to go. But I’ve shot my financial wad on the disks I already have virtually filled with data. Not much option for me but to hope this existing array holds out until either I win the Idaho Lottery or the price of disk storage drops a bunch.

But then what? Would I build a 4TB RAID-1 array and then toss and turn all night worrying about that cheap disk power plug splitter I installed last month catching fire inside the server, or a mega-lightning-bolt hitting the power line that blows the whole thing to smitherines?

I think I’m getting psychotic over this. Time to go fishing.

[jaclaz]

Well, sorry if I made you doubt about your settings.

As I said you have achieved a reasonably safe setup, but RAID technology is NOT 100% safe (btw NO techology is).

RAID has been designed as an affordable, resonably priced way to hold data and TO QUICKLY rebuild the working system in case of some partial failure.

It is not intended as a permanently safe storage solutions, not needing backups.

As in RAID technology, redundancy is the word in backup business.

The "theoretically" correct procedure is to have AT LEAST TWO (but three is of course safer) exact backup copies, each phisically located in different places, NOT rooms in the same building, actually in different buildings or, possibly in different cities.(backup media should be stored in "safety boxes" something definitely airtight, waterproof and reasonably fire resistant)

Personally, I do not trust TAPE backups, or at least I have always been very unlucky, every single time I actually needed to access data on tape backups, usually very old data, say ten years old, either the tape, the cassette, the drive otìr the interface failed on me.

Optical (CD or DVD) storage seem to be more reliable, though I cannot say how long the media would last, I have backup CD's from 1994 that still work, but you never know.

Another point when backing up on optical media is the software you use, in my experience I had failures with "common" CD burning softwares, I am not at all connected to this (no advertising intended) but the ONLY software that in my experience gave me 100% valid copies was this "Accuburn":

http://www.infinadyne.com/accuburn-r.html

http://www.infinadyne.com/accuburn-rtech.html

it is slower than other software discs made with it hold a little less data (probably due to some additional error checking or checksums), but if the write for whatever reason fails, you know it.

About the media, I did not found, again in my limited experience, any problem in different brands.

I never used double layered DVD's as, at least when they came out, there were some concerns about their reliability, as compared to single layer ones.

Unfortunately the amount of data you have is impressive, so that any solution will cost much money (and time) with media available today.

In a very near future there are two emerging technologies:

1) blue-ray discs, said to hold as much as 200 GB each or the HD-DVD (seems very much like the old VHS vs. Betamax battle) - as this high capacity involves multi-layers, there could be the same reliability concerns hinted before.

2) Holographic memory, it is due to be released by late 2006, which means that probably will become affordable/reliable not before 2nd half 2007, but having something like this:

http://techon.nikkeibp.co.jp/english/NEWS_...0050608/105586/

would be VERY nice!

Hope I did not worsen again your mood, fishing should help...

....if you catch some of those little sneaky swimming bastards, of course.

jaclaz

[HyperHacker]

Oh sure, the memory cards are a buck for 30GB but the drive costs $10,000.

[circumflatulator]

jaclaz,

Don’t feel bad about pointing me to those web sites. I owe a great debt of gratitude to you and others on this forum who pointed me to the truth! The whole truth may not be in any single site that questions the validity of trusting RAID or other redundant disk setup to store precious data. In fact, for me, a very important thing I learned in this whole endeavor is of a more philosophical versus a technical nature.

Computers are a great tool for many things. We play games on them, surf the web, watch videos, etc. etc. etc. A few of us may even write a program or two! For those tasks, a few tens to hundreds of GB is enough disk space and when a drive crashes, sure it’s a pain in the a$$ but not really a catastrophe. A new drive and a couple hours of reloading the data back and away you go, the memory of the crash quickly fading.

But as disk space gets cheaper, we add more and more. Then we start finding lots of cool things to do with that new-found storage space. For me, it was music, pictures, and movies/tv shows. Before long, you got a ton of stuff that, while most of it might be “technically replaceable”, the effort to reload/rebuild it when a drive fails becomes a massive endeavor. And if you start putting thing on those disks that may not be easily reloaded (like all those scanned family pictures, where the originals were borrowed, scanned, and then returned to various relatives all over the USA), the surety of a disk failure sometime in the future starts to scare the crap out of ya.

That’s where I was when I started this RAID-5 effort. On the surface, it sounded so perfect. Redundancy and backup with very little extra money and no time-consuming weekly removable-media backup sessions. But that’s an illusion. RAID-5 arrays can fail too, with a total loss of data.

So my advice to all (and virtually every one of you will eventually find yourselves in my predicament, say when disk storage gets down to <10 cents/GB) is to plan an affordable, workable and secure redundancy/backup configuration BEFORE you “build a monster” like I did.

RAID-5 isn’t bad. It’s better than no redundant drives/backup at all. But it’s not enough on its own when the quantity and/or value of your data gets beyond a certain point.

And I’m going fishing tomorrow morning with a friend. A few beers and one of them “sneaky swimming bastards” on my line should make everything all-right again!

[jaclaz]

But as disk space gets cheaper, we add more and more.

Yes I find that the problem is that technology is running on two different tracks, one (faster) is hard disk storage capacity (probably due to much greater requests), the other (slower) is backup media capacity.

I remember a small period (in 1994 or 1995) where I had a bunch of Compaq workstations 486 DX66 running DOS+Win3.11 with a 500 Mb hard disk.

I bought (at a very expensive price at the time) an external parallel CD burner, which was slow but held 650 Mb of data, the perfect, easy, 1:1 backup solutions.

Users, each on a fixed day of the week, would "borrow" the burner and leave the PC on at night, imaging the entire disk on the CD.

The morning after they gave back to me the burner and the burned CD.

Simple, easy and effective.

It lasted just a few months, next generation of HD was 1,2 or 2,1 Gb, I kept for a couple of years partitioning those drives in 650 Mb partitions, (the OS at the time was NT 4.00), but finally had to give up when drives crossed the 4 and 8 Gb size, DVD technology was at the beginning, and I had to go "back" to tapes, as said VERY unreliable.

jaclaz

[circumflatulator]

Just a quick update: Everything still working fine. Enjoying the media server immensely. Has worked perfectly without a single glitch since last year when this thread was current. All 2 TB filled and it is a very rare day indeed if at least 2 hours of video is not streamed off those disks. Most days are much more.

That probably indicates the RAID hack for XP is very reliable and capable of flawless operation over a wide range of hard drive and disk controller configurations, as in my setup.

I still don’t know what will happen when the first hardware failure occurs. I try not to think about that. Will post here when it does happen.