Vista SP1 File Copy Performance

[CoffeeFiend]

I've been investigating the Vista SP1 file copy performance a bit. The speed is perfectly fine as far as I'm concerned, but I felt like having a look anyways, especially copying large files on the same disk.

To quote Mark Russinovich:

The other case where SP1 might not perform as well as original Vista is for large file copies on the same volume. Since SP1 issues smaller I/Os, primarily to allow the rest of the system to have better access to the disk and hence better responsiveness during a copy, the number of disk head seeks between reads from the source and writes to the destination files can be higher, especially on disks that don’t avoid seeks with efficient internal queuing algorithms.

While copying a file (I've tried sizes ranging from 15MB to 700MB) onto the same disk, explorer.exe:

-reads 4x 1MB chunks (initial reads)

-writes 64x 64KB chunks (4MB)

then it repeats:

-reads 1x 1MB chunk

-writes 16x 64KB chunks (1MB too)

until it reaches the end of the file, where it reads the remainder (not quite 1MB),

then writes the rest of the file, still in 64KB chunks

The exact same happens when I copy from cmd.exe too.

Teracopy works drastically different, it uses a 10MB buffer for both reading and writing. It repeats a "3 consecutive 10MB reads, 3 consecutive 10MB writes" cycle.

While I didn't spent time yet timing those exact scenarios (I soon will), I've been doing some tests, using yet another tool I just might share: VFCGB (the name just rolls of the tongue, don't you think? ), or "Vista File Copy Ghetto Benchmark" for those not into over-complicated nonsensical acronyms.

You tell it which file to use to make its tests, and how many times to repeat it (e.g. vfcgb.exe somelargefile.ext 10). It will copy the file as many times as you told it, using different sizes of buffers: 1KB, 2KB, 4KB, 8KB, 16KB, 32KB, 64KB, 128KB, 256KB, 512KB, 1024KB and finally 2048KB (12 tests total, each repeated as often as you told it to). And it displays how long it took to copy the file you mentioned, for each pass, and average time (plain old arithmetic mean) + % of difference in speed for each pass compared to the average (e.g. pass 1/10 5% slower then the calculated average)

It's fairly basic right now. I'll be adding Vista-style "asymmetric" buffer sizes (1MB read/64KB write) and a test with Teracopy-sized buffers too (10MB), and likely csv exporting or such, for easier analysis of the output data in apps like excel (beats typing the times by hand for sure). I'll also add separate read and write benches too (reading a large file into a buffer of different sizes, and writing a differently sized buffer many times to create a large file). Oh, and eventually some "larger" reads at the beginning, like Vista does to "trigger the Cache Manager read-ahead thread to issue large I/Os" (as Mr Russinovich puts it).

Anyways. Here are some preliminary results, on 3 different hard drives -- all SATA with NCQ. I'll be trying an IDE (PATA) drive eventually, and perhaps do some test runs on XP SP3 box just to see how it differs.

On the left

Explanations:

Each curve represents how long it took to complete the file copy operation at different buffer sizes (each curve is actually the average of 10 copy passes) Different curves represent different tests done with files of different sizes, on different drives.

I know, it can be hard to follow some of them, but there's 9 curves and I don't do miracles.

The red curve, is the average of the 7 tests.

The yellow curve is a curve that shows the number of read/write operations required to copy the file. It was added just to see how closely the "average" curve would follow it.

Observations:

It's no surprise, with tiny buffer sizes (1KB to 8KB or so), copying files is excruciatingly slow. As you increase the buffer size, the file copies happen quite a bit faster. It peaks @ 128KB (fastest). @ 256KB there's not much changed yet. But as you hit 512KB -- OUCH!!!1!one! Copying files using a 512KB buffer is nearly as slow as copying files using a tiny 1KB buffer! And that's the case on every copy I've done, no matter what file size, no matter what hard drive. Perhaps it's the cache manager (write behind?) that's having a hard time to cope? Not sure yet, but the performance is truly awful -- you'd think it would be better than at 256KB really. And once you go past that (bigger buffer sizes again), then the problem starts to go away for some reason. I'll definitely be looking into this!

Another thing you can notice is, when you copy bigger files, you tend to need a somewhat bigger buffer to get decent speed (just look at the top 2 curves, they're the two 700MB files). With smaller files, copy performance is still acceptable @ 16Kb, but with them large files, you really want 64KB or more.

On the right

This is just another representation of the "average" curve (on the left, in red). All the file copy passes (10), for every file, on every drive has been averaged. You can clearly see that file copy operation speed peaks @ 128KB (24%), that means overall it's about 4x faster using that, than with a 1KB buffer, and still nearly 4x as fast as with using a 512KB buffer.

It does make me wonder. Vista reading 1MB chunks doesn't initially seem like the best setting for max copy speed, but I'll be looking into that further. Also, I'm not sure why there's that incredible dip in performance @ 512KB. My current guess would be the write-behind cache having issues to keep up with it, and that might explain why Vista is doing 64KB writes (not 1MB) while it does 1MB reads. Since I've been using the same size for read & write buffers so far, we can't be sure yet of which one is causing the slowdown (read? write? both?). And I'll also have to try 10MB buffers like Teracopy.

I'll post a follow up later

Edited July 17, 2008 by crahak

[CoffeeFiend]

Just a very quick update about that big peak @ 512KB.

I did try testing 3 different combinations:

1. 1024KB reads + 1024 writes (was pretty slow, nearly as bad as 512/512)
   
2. 1024KB reads + multiple smaller 64KB writes (just like Vista does)
   
3. 10MB read + 10MB write (like teracopy does)
   

The first is very slow like I found out earlier (same figures)

The second (1024/64) is a great deal faster -- from about 60% faster (on 700MB file transfers) to nearly 4x faster (on 50MB files). So it must be the larger writes that Vista isn't liking (again, probably the write-behind cache filling up). That nasty peak in the curve pretty much vanishes if you keep the write size smaller seemingly (larger reads don't seem to be problematic)

And the third (10MB/10MB), which ranges from not very fast at all (only 15% faster than the 1024/1024 fiasco for 50MB files) to about the same as 1024/64 on very large files (60% faster on 700MB files), so I'm not really seeing any benefits to their approach yet (however, I'm repeating one 10MB read + one 10MB write, and not 3 reads then 3 writes like them, so that might change things a little bit). I haven't benchmarked teracopy either, but it's not looking like it would be any faster than vista's built-in file copy right now.

So perhaps I'll break the benchmark down in 4 distinct parts:

1. multiple small reads for one 64KB write, to see how reading smaller chunks affects performance -- seeking increased lots, likely much slower
   
2. one large read for multiple 64KB writes, to see how reading bigger chunks affects performance -- seems to help so far, 1MB seems like the sweet spot too
   
3. one 64KB read for multiple small writes, to see how writing smaller chunks affects performance -- seeking increased lots, likely much slower
   
4. multiple 64KB reads for one large write, to see how writing bigger chunks affects performance -- 512KB seems to hurt performance a LOT
   

More details, and more graphs later on.

[eyeball]

Interesting read, keep up the good work!

[jaclaz]

Maybe the "strategies" used by this:

http://www.ipmsg.org/tools/fastcopy.html.en

are of interest?

jaclaz

[CoffeeFiend]

@jaclaz: I'll have a look.

Haven't had time to play with this lately. But I've been thinking about using async I/O perhaps (probably won't change anything, still worth looking at), or skipping the FileStream class altogether (maybe it's not "tuned" for 64KB writes?), and calling ReadFile/WriteFile (kernel32.dll) directly instead, or possibly even calling CreateFile "by hand", specifying the FILE_FLAG_NO_BUFFERING flag... Bufferred/unbufferred I/O likely makes some difference (again, depending on the block size and all).

Operations on flash drives are likely affected quite differently by block size too (they tend to use blocks of like 256KB, vs plain old tiny 512 byte sectors for HDs). Similarly, speeds would likely vary quite a bit when I change my RAID0 stripe size or NTFS block size. There's a LOT of factors involved.

And we're still only comparing operations on the same drive, copying between 2 drives would likely affect the results too (besides overall speed).

Edited July 29, 2008 by crahak

[HexiumVII]

Very interesting. I've been very interested in hard drive performance ever since i got my first comps. I was pretty appalled with Vista's copy performance, but Sp1 seems to optimized it a bit better. Win7 seems much better. I'm currently going through Mark Russinovich's Inside Windows to get a better knowledge of whats going on deep in windows. I'm going to try do a few tests myself with XP, Vista, and 7beta. I have a huge bunch of hard drives including Fujitsu 15k MAUs, Seagate 15k.3s, 7200.9, 7200.10, 7200.11, Hitachi 1TB, WD 1TB Green, WD Blacks and Scorpio. Would be interesting if i can get some consistent results. Any tips on what kind of tools i should use? I've been using Teracopy for a while, but 2.0 doesn't let you change buffer sizes easily anymore. I've tried all the other copy tools, but always go back to teracopy.

[CoffeeFiend]

Any tips on what kind of tools i should use? I've been using Teracopy for a while, but 2.0 doesn't let you change buffer sizes easily anymore. I've tried all the other copy tools, but always go back to teracopy.

Tools for what? Testing? I haven't really found anything great for that (easier to write your own).

As far as Teracopy goes, I never saw any actual speed gains, perceived or measured. I tried it like a several times, and always went back to Windows' built-in copy.