Flash Drives Wearing Out ?

[Monroe]

Like most everybody today I have come to rely on quick backups of various programs using several USB Flash Drives. I have various sizes ... 4 GB, 2 GB and smaller. The smaller sizes are older and as the prices dropped on larger flash drives I bought the 2 and 4 GB size. I haven't gone any higher in size since I didn't want to get "lazy" and just have so much GB space available that I totally rely on the flash drive never failing. As it is now, I load so many things on a flash drive and then take time every so often to burn most everything to a CD for a permanent save. Then I delete what was burned from the flash drive and start all over again till the flash drive fills up again. I was curious about whether a flash drive should ever be defragged and hit the Google search trail today. I came across this article indicating that flash drives should never be defragged and a related article, Can a USB thumbdrive "wear out"? ... I am wondering what others from this forum have to say on the subject or are you pretty much in agreement with the "never defrag a flash drive" statement? I do remember there was talk that computers in the future would probably have flash memory instead of a spinning hard drive. Not sure if that is happening yet ... the small Net Books may have the flash memory since they are so small but I'm not sure about that. I guess we all expect a flash drive to die some day but I would hate to have a really large flash drive loaded with hard to replace files. Just curious for any input from you guys who understand all this better than myself. Having older computers with Windows 98SE and smaller hard drives (40 GB) compared to the large hard drives of today, I really like the speed and convenience of a flash drive and being able to transfer things between computers. thanks ...

From the first article:

You've hit one nail squarely on the head: flash devices (or any "solid state" devices) don't gain a performance benefit from being defragmented.

But in reality things get worse. Much, much worse.

You should never defragment a flash drive.

If you're regularly defragging a flash drive, you're adding thousands upon thousands of write operations each time you do so. Whatever the expected lifespan of the device, you could easily be cutting it in half or worse.

Should I defragment my USB Flash drive?

http://ask-leo.com/should_i_defragment_my_...lash_drive.html

Can a USB thumbdrive "wear out"?

http://ask-leo.com/can_a_usb_thumbdrive_wear_out.html

Edited December 13, 2009 by duffy98

[jaclaz]

USB sticks (or more generally FLASH based devices) do have a finite amount of write cycles.

They are NOT suited as permanent/safe storage as they also have a limited data retention threshold - actually unknown for sure - but estimated in 10 years:

http://www.allmemorycards.com/glossary/reliability.htm

Check these:

http://www.freescale.com/files/microcontro...letin/EB618.pdf

http://www.atmel.com/dyn/resources/prod_do...nts/doc2546.pdf

About defragmenting, it is NOT a good idea.

During a typical defragmenting operation, data is usually moved back and forth several times.

An "offline" defragmentation", like creating an image of the stick, defragmentng the image and then restore the defragmented image is better suited.

But again, and just as it is perfectly unneeded in a number of times on normal hard disks, do not overdo it:

http://www.msfn.org/board/does-frequent-fo...dd-t134982.html

Also read these, about the opportunity of NOT using NTFS (or other semi-journaled or journaled filesystem on Flash devices):

http://www.msfn.org/board/help-remove-file...84-page-17.html

http://www.msfn.org/board/usb-stick-dead-t137512-page-6.html

SSD, like used in netbooks use a different kind of Flash:

http://en.wikipedia.org/wiki/Flash_memory#...for_hard_drives

which has presumably a much greater life expectancy, BUT read these too:

http://www.boot-land.net/forums/index.php?showtopic=8757

http://www.boot-land.net/forums/index.php?showtopic=9615

jaclaz

[Monroe]

jaclaz ... thanks for all the info and links in your message. I will read each one today. I have never heard of "offline defragmentation" so I will also check that out.

Edited December 13, 2009 by duffy98

[jaclaz]

An interesting source, focused on data retention and durability of DATA records:

http://preservationmatters.blogspot.com/

jaclaz

[Queue]

It's worth noting that even when a flash drive wears out, you can still retrieve the data from it, just no longer write to it.

But yeah, don't defragment it, provides no worthwhile benefit, and just wears it out a little. Also, it takes a pretty **** long time under normal use, even with defragmenting, to actually wear out a flash drive. I mean years and years.

Queue

Edited December 13, 2009 by Queue

[LoneCrusader]

Also read these, about the opportunity of NOT using NTFS (or other semi-journaled or journaled filesystem on Flash devices):

http://www.msfn.org/board/help-remove-file...84-page-17.html

http://www.msfn.org/board/usb-stick-dead-t137512-page-6.html

jaclaz

I read over those threads, good useful info!

I have a question though - is there a non-journaling file system that can be used to format a flash disk that can support files larger than 4GB, and still be read by Windows 98? I saw someone mention/suggest formatting disks with UDF in another thread, but I am unfamiliar with it.

It would be handy to have such a filesystem, that way it would be possible to save a DVD image on a flash drive (even if you were only going to actually USE it on an XP/NTFS system) but still have access to any other smaller files on the flash disk in Windows 98.

[dencorso]

UDF is the best possibility. Multibooter is experimenting with it. ext2 (or ext2fs, one of the linux fs) is another possibility, supposing DiskInternals Map Drive works. Neither seem to me very good options, though.

[LoneCrusader]

ext2 (or ext2fs, one of the linux fs) is another possibility

Most (someone correct me if I'm wrong) Linux file systems are journaling file systems - including ext2, so that would just lead back to the same problem as NTFS in this situation.

[jaclaz]

It's worth noting that even when a flash drive wears out, you can still retrieve the data from it, just no longer write to it.

This is debatable.

Actually you only "know" that a fleash has worn out when you try writing on it.

It is NOT a "binary switch" ON/OFF, there is a progressive degradation, it is very likely that a number of cells will not take the new info, while some will, and the result would be a mix-up of old and new information.

@LoneCrusader

Remember, you asked for it :

Most (someone correct me if I'm wrong)

You are WRONG.

Ext2 is NOT journaled:

http://en.wikipedia.org/wiki/Ext2

Ext3 can be read as "journaled Ext2":

http://en.wikipedia.org/wiki/Ext3

jaclaz

Edited December 14, 2009 by jaclaz

[Queue]

It's worth noting that even when a flash drive wears out, you can still retrieve the data from it, just no longer write to it.

This is debatable.

I've intentially worn out a drive. I cannot write to it whatsoever. I can still read data off the drive. None of the data on the drive is corrupt aside from the file that was being written to when it stopped letting me write. I had assumed this was standard for all flash drives, but maybe it varies per manufacturer?

In addition, what I'd read about flash drives is they know when a write fails and will dynamically swap in a backup block in place of the dud, until they run out of backups, in which case the whole drive becomes unwritable. Is that not the case?

Queue

[dencorso]

I don't know of drivers for Win 9x/ME, but for the NT OS Family there are, at least, Ext2Fsd and Ext2 IFS for Windows, both freeware, and WDM.

[jaclaz]

I've intentially worn out a drive. I cannot write to it whatsoever. I can still read data off the drive. None of the data on the drive is corrupt aside from the file that was being written to when it stopped letting me write. I had assumed this was standard for all flash drives, but maybe it varies per manufacturer?

That is exactly the debatable part, you cannot recover that file, and depending on how the file was written to the device you may not have the original anymore.

And additionally, yes, if a given manufacturer uses a wear-leveling algorithm it is possible that it goes "beserk" and you also lose something else.

And, as mentioned there is the problem of data leakage from the cells, for which AFAIK, apart "generic" assumptions there are no reputable "field tests" confirming the actual data retention period on a "sound" stick, let alone on a "worn out" one.

Problem is that apparently some wear-leveling algorithms do operate even when reading only:

http://www.forensicfocus.com/index.php?nam...opic&t=3542

In addition, what I'd read about flash drives is they know when a write fails and will dynamically swap in a backup block in place of the dud, until they run out of backups, in which case the whole drive becomes unwritable. Is that not the case?

Yes and no.

Not all USB controllers for USB stick have wear-leveling capbilities, and of course each of them uses it's own algorithm, all of them being rigorously UNpublicated.

The theory is exactly like that, but we don't know (or we don't know specifically for "each" device) if the writing (and swapping) is done in "bursts" at a 512 byte sector "granularity", at filesystem cluster "granularity" or what.

@dencorso

JFYI:

http://www.msfn.org/board/vxds-and-related...lp-t135527.html

jaclaz

Edited December 14, 2009 by jaclaz

[dencorso]

Well, thanks to jaclaz (and eidenk), I now know of an ext2 driver for Win 9x/ME:

FSDEXT2, by Peter van Sebille, latter revised by Gerald Schnabel.

The last revision of the driver is here: fsdext2_0163.zip and its source is here: fsdext2_0163source.zip.

It claims explicitly to support Win 98. So, please, do test it.

I'll do it myself as soon as a proper opportunity presents itself.

[LoneCrusader]

@LoneCrusader

Remember, you asked for it :

Most (someone correct me if I'm wrong)

You are WRONG.

Ext2 is NOT journaled:

http://en.wikipedia.org/wiki/Ext2

Ext3 can be read as "journaled Ext2":

http://en.wikipedia.org/wiki/Ext3

jaclaz

Ah, stupid of me. I already knew that ext3 was journaled, and when I was posting that I looked too quickly at the beginning of that ext2 Wikipedia page and saw "journaling file system" and then didn't read the rest. My apologies dencorso.

Edited December 15, 2009 by LoneCrusader

[Chozo4]

Well, thanks to jaclaz (and eidenk), I now know of an ext2 driver for Win 9x/ME:

FSDEXT2, by Peter van Sebille, latter revised by Gerald Schnabel.

The last revision of the driver is here: fsdext2_0163.zip and its source is here: fsdext2_0163source.zip.

It claims explicitly to support Win 98. So, please, do test it.

I'll do it myself as soon as a proper opportunity presents itself.

Pity, there doesn't seem to be a DOS version of the EXT2 driver unless I'm missing something. Would anyone know of such a solution for read/write use via DOS?

Since all the other questions have been answered, just thought I'd share my experiences with using flashdrive based setups. While being good for access, just be wary of the ones you get as alot of budget line ones will in fact suffer greatly with impaired performance due to the controller. It's best to check reviews for flash drives to determine which would be best for the needs (read/write limitations, speed, etc).

I've been using an /Optima-Pro Attache' 4gb/ (got for $7) for nearly 2 years on my laptop as the primary drive (no HD inside anymore) to run Win98se off of for web development, games, and running as a web server 24/7. However, even though it runs fine and hasn't had any read/write issues (yet... knock on wood); the speed is severely lacking. With the use of SmartDrive and the USB2 DOS drivers loaded off a boot cd for example, the max attainable speeds seem to be ~680kb/write + ~7.8mb/read (tested using rawspeed with a 50mb test file) for this model. It gets incredibly sluggish on write speeds thus requiring all temporary file locations, caches, etc be thrown into a ramdrive for performance reasons.

So, they do take some time to wear out even with excessive amounts of accesses under a FAT32 filesystem. You should be pretty safe at least for 2 years from experiences here if you consistantly use it on a daily basis for heavy amounts of transfer.