Fat32 -> NTFS - highly debating

[rjisinspired]

I am stuck on to whether or not to convert from Fat32 to NTFS in XP. I have always been fond of Fat32. Off and on I would read information on both the pros and cons from converting from Fat32 to NTFS. Reading information about it is one thing, I am curious to know if there's anybody here who had done the conversions and how the experience ended up for you?

In my situation the only real and logical reason to do this conversion is toward video encoding. It is a pain having to segment capture files, correct each segment for consistent frame rates, join them, etc. There are times where a 6GB, uncompressed, video enocde would make things much easier from a starting off standpoint with editing. Later I would use a codec for compression with the least amount of quality loss along with keyframes per frame adjustments for decompressing back to re-edit video for future times.

I am highly thinking of converting to NTFS but I am hesitating for the following reasons:

1. I read it's better to start completely fresh and not perform an upgrade from Fat32 to NTFS - Makes sense to me but right now I don't feel like reformatting from fresh because I have other things I am tending to but later on after doing my work and backing those pieces up, this may be a workable solution to start off fresh. I always back up important stuff frequently like data and stuff in the event of a malfunction but what I'm working on now is videos.

2. Smaller cluster sizes, more writings to drive. - Read about this but I am fuzzy on concepts of it.

3. File compression, possibly leading to a slower performance with programs and system. - if this effects video captuing timings or overall performance in general then maybe converting wouldn't be a good idea? I have a capable machine: 2gb ram, 3.0 Ghrz processor, custom built machine.

4. Defragmentation issues - In Fat32 I don't need to bother with extra defragging issues, nor an MFT altogether.

A few years ago there was a project towards win9x systems called "Extended FAT" or "FAT64" which was designed to get by the limits of the 4GB file barrier but I haven't seen or heard about it so I'm guess the idea might had been dropped altogether?

Thanks for your time and for any experiences anyone may have regarding NTFS conversions.

[CharlotteTheHarlot]

I am highly thinking of converting to NTFS but I am hesitating for the following reasons: <snip>

Like myself, lots of members here will no doubt address these points, but before doing that let me offer an outside the box thought. Rather than stress over potential data loss and an un-reversible modification, let's make it reversible! Get another HDD (the exact same model could be useful, more later). Now clone the current FAT32 C: HDD to the new one as FAT32 C: (cloning methodology to explained later if you need it to be). Now swap out the old one for the new one. Now do your FAT32 > NTFS. This is what we do in forensics, you have a failsafe backup and can in fact reverse the irreversible conversion by swapping out the new one for the old one. You can safely experiment without losing any sleep. Now, by obtaining the same model HDD you get the added benefit of a bonafide performance comparison between the two different File Systems on identical hardware for your specific work. If the new HDD is different, just remember to clock some processes under FAT32 before you convert and do the same later under NTFS.

(1) I agree about starting fresh, especially when its a newer, faster HDD. Partition/Format with WinXP install is as common a chore as they come. But this is not necessary for you yet, because if you clone the drive and then convert you can actually see if NTFS is worth it for you first. Nothing should stop you from later re-installing fresh on the new HDD if you like (that old drive will still be sitting there waiting if things go FUBAR).

(2) Cluster sizes are a bookkeeping affair to the file system (i.e., in a phone book you could store every phone number on its own page or put lots of numbers on each page to reduce the total number of pages). On FAT32 there is a maximum amount of storage units (clusters) that can be addresed by the OS/RAM/Drivers. If that number gets too big the math fails. So they clamp down on the total number of clusters and just increase the amount of sectors per cluster as the drive gets bigger (keeping the phone book small). The consequence is that at the end of a file there can be slack space. For example a file that is 100 bytes of data ( 00,100 bytes ) will actually have a footprint of 32 KB ( 32,768 bytes ) on a typical modern large FAT32 HDD. Slack/waste maximum is theoretically 1 byte less than the cluster size ( 1 byte file stores as 32,768 bytes ). If the HDD has many many tiny little files a lot of space is wasted (because none are usually full clusters). NTFS on modern drives use 4 KB cluster size ( 8 absolute sectors instead of 64 for the FAT32 example ). There will be less slack waste on NTFS ( 1 byte file stores as 4,096 bytes ), but at some other, often debated cost.

(3) Think of NTFS file compression like DRVSPACE. But it is both optional and automatic. I would never use it on purpose myself because it absolutely costs some CPU processing. On WinXP when the HDD gets close to full, the system begins to NTFS compress files to save space. One should never let it get to this point! Buy a new HDD with lots of space. Compression in the archive sense, as in large groups of files in a ZIP/RAR does make sense because many files are stored as one. BTW, this is a real oldtime argument, stretching way back to Pklite for files on floppies, and lately with UPX. You can always find people that think they magically SPEED UP things rather than slowing them down. I assure you they are wrong. Avoid it. Besides, the Wikipedia NTFS page says Microsoft partially agrees with me: "NTFS can compress files using a variant of the LZ77 algorithm (also used in the popular ZIP file format).[17] Although read-write access to compressed files is transparent, Microsoft recommends avoiding compression on server systems and/or network shares holding roaming profiles because it puts a considerable load on the processor.[18]". But then it goes on "Single-user systems with limited hard disk space will probably use NTFS compression successfully.[citation needed] The slowest link in a computer is not the CPU but the speed of the hard drive, so NTFS compression allows the limited, slow storage space to be better used, in terms of both space and (often) speed.[19]". Yeah, well, I'll decide what my CPU is spending its time doing, thank you very much. Not.

(4) Defragging on FAT32 was critical, and not only because of performance gains. It was critical because if the HDD had any Boot Sector problem or the OS was FUBAR, the files that were defragged (clusters were made contiguous) are much more easily retrievable (by those with knowledge and experience). To make a long story short, the drive was as safe as the last defrag. NTFS does not need contiguous clusters for files to be retrieved in an emergency because the file pointers are stored differently, in a MFT. By design it is almost bombproof. In short, you do not have to defrag on NTFS unless you want to (IMHO naturally).

About FAT64, I wondered the exact same thing in this post. But then later I saw the WIKI about FAT64 and found out it already exists as exFAT.

EDIT: minor oops, don't ask. EDIT: again. That's weird. When going in for the 2nd edit, even though the 1st edit was definitely shown, the page that is served after clicking FULL EDIT is the original non-edited page.

Edited December 23, 2008 by CharlotteTheHarlot

[jaclaz]

I may add that generally speaking SERVER machines have by design very fast disks and comparatively low CPU power.

On the other hand Common END USER PC's have relatively slow disks and usually very fast CPU's, that are often not used to their full capabilities.

Consequently, since:

TANSTAAFL

http://en.wikipedia.org/wiki/TANSTAAFL

Compression is a trade-off, you are exchanging some CPU cycles for faster data access/transfer.

It's up to you to find out if the "deal" is advantageous in your particular case.

Also, compression, even if not as much as encryption, adds a layer of complexity that can make, should disaster strike, recovering data more difficult or impossible.

jaclaz

[E-66]

Re: cluster sizes - in XP you can choose the cluster size you want when you format a drive/partition. If you have a dedicated partition for your large video files just choose the largest cluster size.

[paraffin]

Just get with the times and use NTFS, at the very least you get a lot more security.

Without thinking about this to much the compromises of FAT32 are not worth the hassle i.e. largest file size....

Rebuild your PC it takes less than an hour including installing essential apps, you have a modern File Structure, fast OS as its fresh and no file size restrictions....

It's a no brainer....

[jaclaz]

Hey, this must be a record , this thread got as far as 4 (four) posts before someone came out to say that NTFS is better.

If I were you I would also abandon XP and double boot Linux and Vista , this way you'll save a lot of of other posts.....

Back to topic, cannot really say, I never personally experienced ANY problem using the CONVERT.EXE utility, but again it never happened to me to use it on a fully configured SYSTEM volume.

The most logical thing to do, if you are living happily with FAT32 and the only problem you have is with biggish video files, would be to get another hard disk, and make on it a partition NTFS only for DATA where to store the bigger than 4 Gb files. (or use a partition utility to resize your current partition(s) and create a NTFS DATA one on your current HD).

Then, when you have another occasion and time to re-install, try re-installing the system to a NTFS volume.

jaclaz

[DJ MyRinX]

[offtopic] CharlotteTheHarlot nice explanation, thanks! [/offtopic]

As a user myself i would go for a fresh install on a NTFS formatted partition/hdd.

why?

There are times where a 6GB, uncompressed, video enocde would make things much easier from a starting off standpoint with editing

The size of the (visual related) files are getting bigger and bigger, and you probably will be having more and more problems with the 4 GB limitation in the (nearby) future.

Paraffin's post sounds kind of crude, but a fresh install takes about an hour or two to be completed (if you got the right drivers and such)

"Charley's" suggestion sounds like a nice project though, if you got some spare time.

jaclaz suggestion sounds like a winner as well.

edit: didn't see jacklaz post

Edited December 23, 2008 by DJ MyRinX

[paraffin]

LOL, I don't think I actually said that NTFS was better I just pointed out the benefits of NTFS in a limited fashion.

To be honest I doubt anyone would actually notice that they were using FAT32 or NTFS unless they hit the file size limit. As the file size limit is the problem here or one of them then I would say get onto NTFS and forget all your troubles and workarounds for FAT32.

My method might be crude but I love a freshly built machine...

Also my method is one of many and the best one for you is the one that matters.

[rjisinspired]

Thanks for all your advice and suggestions. I like the cloning idea of the drive. I'll get my hands on a second driive some time soon.

The 4GB limit will also become more of an issue if I ever get into HD video someday.

I see what you mean about the slack. I see it when I sometimes save full web pages containing small bits of files with images and the file contents could read say: 200K but end up being 2.4MB something on disk. That is a bit ridiculous.

A while ago I ran a LiveCD of Ubuntu and really liked what I saw. Though there was a problem with the grub loader not being able to install at the time so I couldn't get it working. Maybe I'll try the Ubuntu route again someday.

I think win7 will be far better than Vista, as long as MS doesn't bloat it up with a bunch of stuff.

[Zxian]

I would simply suggest taking the plunge and converting the disk. Run a complete chkdsk on the drive beforehand, and then convert it to NTFS. The only time I've seen the conversion process fail is when there were several corrupt sectors that hadn't been caught by chkdsk earlier.

I don't want to open up this box of worms again, but Vista is already a considerable improvement on XP, which was a step up from 2000 (judging by your OS choice in your profile). The main problem to date has been that OEM's have been installing it on underpowered hardware. Ever seen the Mojave Experiment?

@paraffin - There are several reasons to use NTFS over FAT32. Performance on larger drives has proven to be faster on NTFS partitions, and there are countless other benefits to the system. For one, NTFS is a journaling file system which leads to reduced fragmentation problems compared to FAT.

[WwTIPPYwW]

this is a relevant thread....like 10 years ago. NTFS offers much more improvements over FAT32 to make this debate nonexistant.

You can use tools to convert to NTFS w/o formating.

You can use tools to read NTFS partitions.

This used to be a lot more complicated years ago.

Before doing anything - BACKUP you vital data!

[CoffeeFiend]

this is a relevant thread....like 10 years ago. NTFS offers much more improvements over FAT32 to make this debate nonexistant.

That pretty much sums it up. Most people made the switch with Win2k almost 10 years ago and didn't look back since then (in fact, many of us sort of already had before that -- with Win NT or HPFS on OS/2)

FAT32 is a lowest common denominator thing -- everything reads and writes it, including embedded devices. Besides that, it's about the worst filesystem still in use these days (e.g. compared to NTFS, ext2/3, ZFS, etc).

NTFS has ACLs, is journaled (WAY more fault tolerant), supports encryption (EFS/bitlocker), supports larger files (e.g. large multimedia files, large disc images, large VHD/VDMK files, etc -- TONS of uses), isn't commonly limited to 32GB partitions, supports having more files, supports unicode file names, has the same [default] cluster size as memory page size (faster), directory junctions, soft & hard links, more attributes, alternate data streams, volume mount points, volume shadow copy, transactions, quotas, sparse files, reparse points, etc. And the list goes on. LOTS of useful stuff in there.

FAT32 performs poorly on anything but tiny partitions, has extra overhead as the Fast FAT file system driver keeps a copy of the entire FAT in RAM (~8MB extra per partition IIRC) and that evens slows down the boot process a little bit (having to read it, and it also has to calculate the free space), is "commonly" limited to small 32GB partitions, is more error prone (all these scandisk errors, .chk files and all), wastes a lot of disk space (bigger clusters), etc.

There's simply no contest here.

Smaller cluster sizes, more writings to drive

Nah. Same amount of writes to the drive (a cluster is like a group of several 512 byte disk sectors). Just wastes significantly less disk space.

Vista works great BTW

[CharlotteTheHarlot]

@rjisinspired: although you're hearing NTFS is better and even Vista is great, I'll bet this is something we can all agree on:

Don't convert your one-and-only system drive without a backup!

That was my main point above. To sharpen this point further, an internal ATA drive is $50-$100, far less cost than any of the probable video tools you use (I'm guessing Vegas/Pinnacle/Avid/AfterEffects/etc), not to mention any video hardware. I have seen this before, clients invest heavily in their primary field (Audio/Video) equipment but cut corners on the thing thats holds their valuable and very perishable data: the HDD and a fallback plan. Put it another way, don't worry if the camera or the DVD burner dies, but do worry about the data and its container. It is the only thing that is irreplaceable, thus it is the only thing that matters.

As to the merits of the two file systems, each has their own strengths and weaknesses. Interestingly, you are well-positioned to report back later with a near-scientific comparison of both file systems on identical hardware for specific tasks. So please, if you manage to test with careful controls, your experiences would help others (and as an added bonus: fan the wonderful flames of NTFS vs. FAT32! ).

[rjisinspired]

Great. After all the videos are finished up I'll backup all the data I have and do the conversion. My videos are either from an Aiptek cam or Sony Digital8 so I have hardcopies.

I also have tapes from over 20+ years ago from VHS-C when I was 13 and one on the last day of Junior High, lol. Tapes are in excellent condition up to this day.

I'll keep updated on how things go after this conversion. Thanks for all the help.

[CharlotteTheHarlot]

Great. After all the videos are finished up I'll backup all the data I have and do the conversion. My videos are either from an Aiptek cam or Sony Digital8 so I have hardcopies.

I also have tapes from over 20+ years ago from VHS-C when I was 13 and one on the last day of Junior High, lol. Tapes are in excellent condition up to this day.

I'll keep updated on how things go after this conversion. Thanks for all the help.

Im guessing that you are doing things like editing in Vegas and/or AVI>MPG for DVD burning. If you have a stopwatch you might do the same exact operation under both file systems to obtain potentially useful info for other members. Clicking on a transition effect would NOT be useful (mostly CPU driven) but clicking save after an edit in Vegas (with stopwatch) will punish the file system. Another idea, time your cold boot (power off to desktop), warm boot (restart), and shutdown (desktop to power off). Post them in this thread for both FAT32 and NTFS.

The key thing to do to create an effective control, is to do the same operations under FAT32, and then later under NTFS after the conversion.

Edited December 24, 2008 by CharlotteTheHarlot