Optimal Page File Setting

[NoelC]

Thanks.  Those seem reasonable, but the concept still bothers me.

 

What happens when the RAMDISK isn't big enough?  There WILL be times.

 

Bear in mind that if you set up something like Intel RST (Rapid Storage Technology) to drive the disk subsystem, you DO have write-back RAM cache between the low-level I/O and the actual disk.  So essentially you have an on-demand RAMDISK.  That's why well-configured systems using RST can achieve near 1 GB/second I/O transfer rates with 4K byte operations.

 

Also, are RAMDISK drivers 100% reliable?  I'm assuming they must be pretty good, but that would inject another variable into the system that's not normally there, and could cause a system to be less stable than one with a sufficiently large pagefile.sys in its default location.

 

The virtual memory OS design is originally by the guy who did it for Digital Equipment Corp. before doing it for Microsoft.  Dave Cutler possessed True Genius (he may still), and I would think twice (or maybe 256 times) before trying to second-guess his design decisions.  I think it's also somewhat irresponsible even to try to even outguess the design goals of "some stupid program".  While theoretically your RAMDISK should just be a faster version of a drive, it seems to me "some stupid program" would just use the RAM if it could.  It's awfully easy to allocate RAM (PointerToBuffer = malloc(SizeNeeded)) as compared to writing to the pagefile.

 

But I'm glad it works for you.  Please don't take the above as a criticism.  I'm really striving to learn here and very much appreciate hearing about how alternate configuration choices operate in the real world.

 

I'd be interested in hearing specifics...  Do you know of a particular implementation of "some stupid program" that writes to the pagefile?

 

I can tell you this:  With a LOT of RAM and very high speed SSD-based storage, I HAVE pushed my system (e.g., when working with huge Photoshop files) into virtual memory, where both Photoshop and the system are writing HUNDREDS of gigabytes to scratch and page files.  It's actually somewhat difficult to tell when the point is reached where the system goes virtual, and I consider that a success.

 

-Noel

[TELVM]

... What happens when the RAMDISK isn't big enough?  There WILL be times ...

 

Windows will start bitching about 'low on memory':

 

 

That will happen even when there are still several GB of RAM free.

 

 

... Do you know of a particular implementation of "some stupid program" that writes to the pagefile?

 

Photoshop, Prime95, and the ArmA game come to mind. There are others.

[vinifera]

thats windows for you

some say it preserves part of memory for kernel and shit

but 30% overall is too much hahahah

[NoelC]

I'm not having any problems with Photoshop, even when working with multiple gigapixel images.  I don't see how faking a pagefile on a RAM partition could help that be any better.  It's already perfectly responsive.

 

A virtual memory type operating system requires sufficient resources to run.  Those resources include both RAM and disk.

 

Anecdotally on a 48 GB system that's settled into a long-term groove, after a week or two of run time (heavy daily use, plus nightly backups) I only see about 10% of the RAM used at any given time when no applications are running.  The important parts of the file system are cached, things that need to be loaded are loaded, and it just hums along, responsive to my needs.  I couldn't care less whether it writes to the pagefile as long as it's reliable, and I honestly don't see how it could perform any more smoothly.

 

That made me think...  Goals should figure into this discussion...

 

For me, priorities are:

 

• Reliability - MUST not crash, MUST not lose data.
   
• Performance - must provide a highly interactive experience.
   
• Multitasking - must have the reserve capacity to run things in whatever combination needs demand.
   
• Low maintenance - must allow me to concentrate on what I'm using it for with minimal maintenance.

 

I can imagine others might alter the order of the top few items, and spread or narrow the relative importance levels.

 

-Noel

Edited October 4, 2015 by NoelC

[TELVM]

...  I don't see how faking a pagefile on a RAM partition could help that be any better.  It's already perfectly responsive ...

 

 

OK, that's your prerogative.

[jaclaz]

Goals should figure into this discussion...

I don't see any actual "discussion" here. 

We all know how your settings, hardware and tuned OS's are perfect for your uses  (which is a good thing).

The fact that a system with 48 Gb (i.e. some 24 - twentyfour - times the minimum requirements for the OS) runs smoothly is not particularly surprising.

 

Let's take the old (maximum) 3x suggestion for the pagefile (though as seen current recommendations are much lower, more in the range of  0.5-1.1x).

On a 2 Gb machine that would be 2 Gb+6Gb=8Gb (RAM+pagefile)

On a 4 Gb machine that would be 4 Gb+12Gb=16Gb (RAM+pagefile)

On  a 8 Gb machine that would be 8 Gb+24 Gb=32 Gb (RAM+pagefile)

So you have (all in RAM) 1.5 to 6 times the maximum amount of memory available on -say - 89.37% of running non-server machines in the world.

 

OT, but anecdotally, when in the 80's people transplanted  Volkswagen Porsche 914 2 liter motors on Fiat 500's they were unanimous in reporting that the car became much faster and ran a lot smoother.

jaclaz

[NoelC]

Yes, I have a big workstation.  I purposefully overprovisioned it so it would serve all my needs for years to come (and in fact that's turning out to be the case, as it's now 2-1/2 years old and I haven't craved more yet).  I believe I've only ever seen a "low on memory" message maybe once in several years of hard use.

 

As one who has achieved reasonably good performance, should I not involve myself in discussions regarding system performance?

 

I hope I'm not coming across as confrontational here - that is NOT my intent (and I'm sorry to anyone whose viewpoints I've challenged if they feel I have offended).  I am actually trying to spark real discussion on this subject.  Practical Windows performance interests me greatly.

 

My pagefile, on the 48 GB system, by the way is 16 GB.  Seemed like pushing the virtual address space up to a nice round number was a good idea at the time.  How much it gets used is a mystery to me, and that Pagefile Usage Monitor you've posted, TELVM, seems interesting.  I also ran across this:

 

http://blogs.technet.com/b/clinth/archive/2013/10/16/tracking-page-file-reads-and-writes.aspx

 

It seems to me that getting the setup right when the pagefile actually WILL need to be used will be even more important than on a machine with big RAM. 

 

I'm going to leave a Performance Monitor running and try some stuff, such as running Photoshop on big datasets.  Here's what it shows with nothing much happening:

 

 

Note that this is from Win 8.1.  I don't think there will be a fundamental difference between 8.1 and 10, though it would be interesting to gather stats from both and compare.

 

-Noel

Edited October 5, 2015 by NoelC

[351837]

@NoelC

​http://www.standards.com/ThisAndThat/PageFileUsageMonitor.html

[jaclaz]

@NoelC

It is not about confrontation of any kind, you are simply exposing your successes in piloting a F-14 Tomcat to people that own and fly Piper's and Cessna's, all interesting but rarely of practical use.

I presume that your photoshop uses on astronomical images implies loading files that are a few zillion bytes, possibly much larger that what a "normal" professional photographer deals with daily.

 

jaclaz

[Nerwin]

I have a 800mb pagefile set on my SSD and it Windows has yet to use it even though I use photoshop and lightroom all day. So I guess its working fine for me. 

[Ridrok]

If that can be useful to anyone, I have one of the first I7 ever made, 12gb DDR3 RAM, Win 8.1 x64 on a 256gb Plextor PCI-E SSD with tonnes of applications mainly to develop.

The applications I develop sometime require a bit more memory than the free one I have with VS/sygwin loaded, so I did setup Windows to use a fixed 4gb pagefile on the SSD to accomplish this. Simple. -- My Linux Gentoo on the same PC also has a 4gb swap partition for same use on Linux --

 

So my advice is simple: 1) set it fixed size to be sure the place is allocated once for all and on a fast drive if you really require this memory space (and make it contiguous if you use a HDD) 2) Set the size to what the programs you run may require, no need to match physical memory size which became irrelevant with the memory we can now put on motherboards. Do you care if Windows cannot dump its full memory in case of crash? I am not, I don't develop FOR Microsoft, I develop WITH Microsoft products.

 

Ridrok

[jaclaz]

So my advice is simple: 1) set it fixed size to be sure the place is allocated once for all and on a fast drive if you really require this memory space (and make it contiguous if you use a HDD) 2) Set the size to what the programs you run may require, no need to match physical memory size which became irrelevant with the memory we can now put on motherboards. Do you care if Windows cannot dump its full memory in case of crash? I am not, I don't develop FOR Microsoft, I develop WITH Microsoft products.

Yep , and as said before:

http://www.msfn.org/board/topic/174520-optimal-page-file-setting/?p=1109062

 the newish Windows NT 6 OS's allow to have a dedicated file for crashdumps (that can of course be set on a "slow" device).

 

While I am at it, good luck to anyone that wishes to (or needs to) analyze a 16 or 48 Gb memory dump . 

 

jaclaz

[TELVM]

... you are simply exposing your successes in piloting a F-14 Tomcat to people that own and fly Piper's and Cessna's ...

 

"There is no little enemy" (Ben Franklin)

 

 

https://en.wikipedia.org/wiki/Carl_Gustaf_von_Rosen#Biafran_War

Edited October 7, 2015 by TELVM

[NoelC]

While I am at it, good luck to anyone that wishes to (or needs to) analyze a 16 or 48 Gb memory dump . 

 

If I ever generate one, I'll let you know how it goes. 

 

-Noel

[jmonroe0914]

I have a 800mb pagefile set on my SSD and it Windows has yet to use it even though I use photoshop and lightroom all day. So I guess its working fine for me. 

If the pagefile isn't being used, it's because it's been disabled by Samsung Magician.  The pagefile serves an extremely important service and should be set to a minimum of 16GB [at least equivalent to the amount of RAM installed, since it's there for RAM usage].  The 840 Pro carries a 5 year warranty, so writes shouldn't be of much importance.  I frequently use 10 - 11GB page file sizes with Windows 10, and if you're using a high quality SSD, you'll notice a more responsive system over time by ensuring it has at least 16GB allocated to the pagefile (to understand why, do some reading on google to understand what the pagefile is, why it's there, and how it affects system performance).  I utilize Hyper-V quite frequently. so my pagefile is set to the same size as my RAM [32GB] on my 850 Evo, and has been known to climb above 20GB of usage.  

 

Either allowing windows to manage the size, or setting a minimum 1024mb / maximum 16384MB would be best.  RAM transfers occur at 2GB/s+, whereas PageFile transfers on HDDs occur at a max of 160MB/s [generally far less, < ~100MB/s; whereas the 840 Pro transfers at 520MB/s, probably not dipping below 400MB/s for pagefile read/writes

Edited December 6, 2015 by jmonroe0914