Looking for VERY LARGE backup solutions...

[Jaqie Fox]

If you are up for some engineering and coding work, I have an idea Ive been kicking around for years.

Take a couple blu-ray burners and set them up with an autochanger. Make 50 disc disc packs (like say CD reel cakeboxes upside down). think of it like the old disc packs when hard drives were brand new. The system could be set to parallel burns of two discs at once, then do a full verify to make sure the disc burn went well, toss bad ones directly to the trash... you get the drift. As I said it would require engineering work and some serious programming, too, but it would be one heck of a way to do what you ask.

[jcarle]

If you are up for some engineering and coding work, I have an idea Ive been kicking around for years.

Take a couple blu-ray burners and set them up with an autochanger. Make 50 disc disc packs (like say CD reel cakeboxes upside down). think of it like the old disc packs when hard drives were brand new. The system could be set to parallel burns of two discs at once, then do a full verify to make sure the disc burn went well, toss bad ones directly to the trash... you get the drift. As I said it would require engineering work and some serious programming, too, but it would be one heck of a way to do what you ask.

It's an interesting idea, but I refuse to support blu-ray unless one day i absolutely have to. Why? Because it's Sony, and Sony decided to be stubborn with their format, just like they did with BetaMax. Look at the stupid DVD-R / DVD+R mess we have now, Sony just made it worse.

[Jaqie Fox]

True. I meant '~50GB optical media' instead of blu-ray, sorry. I boycot sony as well...

[Maleko]

Blur-ray...another fad from sony, like their mini-disc players, or the media (which i cant remember the name) that wa sonly used in digital camcorders by them which never took off...same old story.

Surely going though a lot of '~50GB optical media' will be a pain in the a** still?

Why not set up a satallite link from your house to my house jcarle? lol j/k

erm, its a tough one with wanting to send it off site to somewhere not locally. with it being so much data.

Invent a water proof/bomb proof/emp proof hard drive cady? lol

Is there any company that lives driving distance from you who can store a NAS station for you? and you ethernet it across?

Dunno, just random thinking atm..

[jcarle]

Even though I'm not a big fan of the idea of backing up to hard drives, aside from tape, which in itself is a very costly solution, I think I found something that could perhaps help.

Western Digital My Book Pro. It's small enough that it could be stored in a lock box if need be, the hard drive is protected, and it stores 500GB via a Firewire 800 port. Since my backups are going to be incremental, I would only need about one per 500GB of backup I do.

Only thing I didn't quite figure out yet is which method is more cost-effective...

[Maleko]

ah, we got one of them here at work, also i want one

didnt tihnk of that tbh.

[CoffeeFiend]

Even though I'm not a big fan of the idea of backing up to hard drives, aside from tape, which in itself is a very costly solution, I think I found something that could perhaps help.

Well, external hard drives are still... hard drives. That didn't really change anything - it's the same media.

Personally, I wouldn't necessarily use one of those though. A LOT of enclosures are notorious for overheating, usually killing their onboard controller, but sometimes the drive too (especially those without fans). Being made of metal is hardly any help. I've given up on these a while ago, and haven't even bothered to RMA the last one.

And if you're going to need several drives for backing up, you could use something like this USB cable/adapter.. It does pretty much all HD types too (PATA/SATA, all sizes). It's surely cheaper to buy one of these than a half dozen [unreliable] enclosures over time - and it really comes in handy. Eventually you could opt for eSATA...

It's certainly going to be FAR cheaper too. 30$ once for a good tool that might be getting a lot of use, plus actual storage costs, using whatever drives currently have the best GB/$ ratio when you need to buy some. That enclosure you pointed to is 350$USD, which works out to 400$CAD (+tax, 1.25GB/$ before tax). For that price, you can buy 3 320GB HDs @~110$ea (2.9GB/$; 960GB total) and have money left, or a couple 500GB WD HDs (if you overlook the price of the cable; 2.39GB/$ - you seem to trust WD drives). As the bigger drives get cheaper, you get bigger ones. Buying those in enclosures is ALWAYS going to be ridiculously expensive (the enclosure is seemingly costing 200$ extra)

As for tape drives, I've looked at that a while ago, and it's definitely not worth it! There's 2 main options:

1)

buy a cheap tape drive (possibly 2nd hand/off ebay or such) - like a DLT IV tape library ("changer"). But then you have to buy a gazillion small tapes. I have a few TBs to backup already, and it keeps expanding (lots of it being stuff that doesn't compress well, like AV). Also, the transfer speeds SUCK. Personally, It would have taken me at least 100 tapes to start with, and my initial backup - assuming the tape library always has empty tapes to use, and works 24/7 - would have taken over a month. And the price of those small tapes isn't much lower than HDs to start with. The only advantage this has (over #2) is a cheaper drive (but then again, how reliable?). Too many tapes (where does one store hundreds of tapes?), and WAY too slow.

2)

Much like #1, but with a newer/nicer tape drive. Like someone mentioned before, say, a LTO3. First off, you'll need a nice SCSI card and cables ($). Then the drive - good luck with that. over 2000$ 2nd hand, and more like 4000$ new (possibly more for a library), e.g. the HP StorageWorks Ultrium 960, which is ~4850$USD + tax and ship. A pretty hefty initial investment for almost anybody! That enough money right now to buy fifty 320GB HDs (16TB of backup). And you haven't bought tapes yet! The tapes cost ~90% of what HDs cost, in terms of $/GB. So buy the time that ~10% difference pays for a 4850$ drive... You'd have to buy 48500$USD of storage to break even, or the equivalent of about 440 320GB'ers (140TB). There are a couple advantages to this solution though - far less tapes than #1, more reliable drive, and faster transfer speeds. To backup ~2GB this way, you'd need 6 tapes and the drive, which is already ~7000$CAD with tax (+ ship, and + the SCSI card and cables).

As for next generation optical media (e.g. beta-ray), it may store more than good ol' DVDs per disc, but the price is ridiculous. Last I checked, 25GB discs were ~25$ (1$/GB), which is more expensive than Hard Drives. And require you to buy a 1000$ drive for a format that could be dead in a year. Hopefully you don't make coasters either! And I'd be worried about scratches and such, and media longevity. Might be worth considering in a couple years, if it's still around, once prices have dropped, and that we know that one format or the other is here to stay. Until then, it's not even worth considering (much like the InPrise holograhic vaporware).

Long story short, hard drives:

-have very good sizes (no need to change media all the time, and store/catalog it all)

-have very good transfer speeds (the best of all solutions)

-has good longevity, unlike optical media in general

-isn't scratch prone (dropping it may not be a good thing though, but rarely happens, unlike scratches)

-don't require a specialized drive/reader/writer that costs a lot of $$$, nor SCSI stuff

-have a very good price for the capacity ($/GB or GB/$, whichever way you want to look at it) - and will remain so (if newer tapes/optical media comes out with better $/GB, you need a new drive/reader/whatever - with HDs, you just buy whatever has the best ratio when you need some)

-doesn't require a hefty initial investment

-doesn't need expensive specialized backup software (like tapes), and is supported by almost all programs (from backup/sync apps, to archivers, etc)

-is accessible almost instantly (tapes has to seek whatever you need to restore first, so even if the speeds are OK for some, there are delays)

-is a simple technology (no SCSI cards, SCSI IDs, termination, drivers, etc), and you can connect it to any old PC with PATA/SATA ports, or via cheap USB/FW adapters and enclosures (ubiquitous)

[IcemanND]

Hey, got an idea was posted here a few weeks ago, paper storage: http://www.msfn.org/comments.php?shownews=18847

Only 5 sheets of low cost paper.

[CoffeeFiend]

Hey, got an idea was posted here a few weeks ago, paper storage: http://www.msfn.org/comments.php?shownews=18847

Only 5 sheets of low cost paper.

Too bad it's a scam? (or alternately, see this instead) When things look too good to be true, it usually is. And it didn't just look improbable, more like totally absurd and impossible. More than 10x the data density of DVDs on wood pulp? Yeah, OK!

A 8.5x11" / A4 sheet, printed @ 600dpi (with no errors or such), 24bit color (not that printers or scanners are anywhere NEAR this accurate), border to border (no margins), gives like 100MB tops. Having this work in the real world is already a fairy tale. And he supposedly gets what... Just 4500x that? Even with 1200dpi (more than the vast majority of scanners/printers can do), that would still be just 400MB. Realistically (especially due to color calibration/gamut/ink fading/etc), you can likely only get 1/3 of that (less if you add some margins). So he's really more like 15000x over a realist expectation.

Him having 45 seconds of video stored on paper isn't exactly a huge accomplishment. @ average xvid/mp3 bitrates (say, ~875 for xvid & 128 for mp3 - or 1mbit total), that's 750KB of data. That would require printing an A4 sheet @ 89DPI (plain black dots, no colors) - let's make it 100dpi with some borders. You could triple the video bitrate (having more or less DVD quality if a decent codec is used) and it would still be easy.

In other news, I've got this really nice bridge for sale, located in Brooklyn.

[jcarle]

@crahak: You make a lot of sense. I guess the most practical and economical solution is the one option I was trying to talk myself out of. At the current moment, technology as it is, it's foolish to try to find a different method as it's just not feasible.

And by the way, that cable? OMG. That's going to become my second favorite tool when I fix client's computers. My favorite right now is my D-Link High Speed USB 2.0 Fast Ethernet Adapter.

[CoffeeFiend]

And by the way, that cable? OMG. That's going to become my second favorite tool when I fix client's computers.

I know! I also have the older PATA only version (was like 9$ on special at one point, still works great for all PATA drives). Well worth the 30$ for sure, and yes, a very handy tool (no need to open case, reboot, etc - just plug and go).

As for making a lot of sense... Let's say I've been down that road before (large video server, music, tons of photos including many thousands of 12MB raw files/panos/large PSD files/etc, many databases, a few SCMs- lots of code, lots of documents and ebooks, etc - a few TBs). Unfortunately, there is no really good/practical solution to backups - HDs being more or less the least worst solution really. Bad enough that even big places are doing this too (70GB RAID array for backup).

And unfortunately there seems to be nothing really better coming up. Newer backup solutions that come out take forever to become afforadable, and by the time they are, the media seems tiny e.g. beta-ray drives are 1000$, and it already would take me like 3 full spindles of the media to backup my stuff, @ 25$/disc no less. By the time they're afforable (2 years or so?), I might need twice that (32 extra blanks needed for every pair of 400GB drives you buy). Having so many bloody HDs does feel somewhat ridiculous though (but then again, thousands of DVDRs or hundreds of tapes would be no better). So many HDs (and associated noise & heat) that I'm dreaming of a nice and expendable iSCSI SAN instead (pretty bad when you wish coolermaster stackers had more space for drives, I need like, shelving or something).

I had high hopes for InPhase's holographic stuff a while ago, but the way it's going, we'll still be waiting in 2015. By the time it's out with its big discs (big by today's standards) and affordable (the drives are supposed to cost like 10k$), the 500$ Dells will likely ship with 2TB HDs (using perpendicular recording or even flash).

[Jaqie Fox]

I know! I also have the older PATA only version (was like 9$ on special at one point, still works great for all PATA drives). Well worth the 30$ for sure, and yes, a very handy tool (no need to open case, reboot, etc - just plug and go).

$30!?!?!?

you guys pay way too much for stuff.

http://www.geeks.com/details.asp?invtid=20...&cpc=CBLbsc

[Maleko]

I know! I also have the older PATA only version (was like 9$ on special at one point, still works great for all PATA drives). Well worth the 30$ for sure, and yes, a very handy tool (no need to open case, reboot, etc - just plug and go).

$30!?!?!?

you guys pay way too much for stuff.

http://www.geeks.com/details.asp?invtid=20...&cpc=CBLbsc

So do you it seems!

We can get them for as little as £3.99 here! hehe, (tis without a power supply mind, but thats easily worked round)

Edited December 8, 2006 by Maleko

[Jaqie Fox]

So do you it seems!

We can get them for as little as £3.99 here! hehe, (tis without a power supply mind, but thats easily worked round)

Not if you wanna take it with you!

there are cheaper ones without the psu, but it's a lot more trouble to have one without and have to rig something (which will almost always be larger and harder to take with you say in a laptop bag)

[jcarle]

@crahak: That's a hell of a nice system they put together. A shame that they chose Maxtor hard drives. I could only imagine using WD RE2 drives that it would last ages. Re-confirms my beliefs that 3ware makes some of the best RAID hardware though.

@Jaqie Fox: Cheaper ain't always better. I dunno, that nGear one seems a little more reliable. Plus $30 CAD vs $15 USD, not a huge difference.