Take advatage of 4GB+ RAM in Windows 7 32-bit

[BoardBabe]

This question is theoretically for educationally use ONLY.

The 4GB RAM limit of Windows 7 (as in vista) is known to be software limited from MS. However there is a "workaround" using certain legit(!) files from Windows Server 2008 if you own both versions, replaced with Vista system files for Windows Vista to take advantage of more than 4GB of RAM. As Server 2008 can handle up to 64GB RAM.

Is this possible also in the Windows 7 kernel?

[spacesurfer]

The 4GB RAM limit of Windows 7 (as in vista) is known to be software limited from MS.

I thought it was a limitation of the architecture and hence a hardware limit.

[cluberti]

This question is theoretically for educationally use ONLY.

The 4GB RAM limit of Windows 7 (as in vista) is known to be software limited from MS. However there is a "workaround" using certain legit(!) files from Windows Server 2008 if you own both versions, replaced with Vista system files for Windows Vista to take advantage of more than 4GB of RAM. As Server 2008 can handle up to 64GB RAM.

Is this possible also in the Windows 7 kernel?

The 4GB limit is hardware based (well, almost - I'll explain the workaround later that allows up to 128GB) - the x86 architecture can really only address memory addresses at 2^32, or 4,294,967,296 (4,294,967,296/1024x1024=4096, or 4GB). So, no executable code can be at any address higher than the 4GB "boundary". However, there were 2 workarounds (hacks) that Intel put in place awhile ago to try and extend the life of the 32bit platform until it could get it's 64bit Itanium platform mainstream, and these were PAE and PSE (read here for more on why Microsoft chose to implement PAE rather than PSE).

The hack you speak of, used in Windows, is called Physical Address Extension (PAE), which was added by Intel in the Intel Pentium Pro and Pentium II lines of CPUs (and continued subsequently to this day) to allow data (and data ONLY) to be mapped into RAM above the 4GB boundary - however, there is a set of limitations on this window. The first limitation is that this window to the RAM above 4GB *must* be mapped into a contiguous memory address range in the application's original 2GB (or 3GB, with /3GB) process space, because again the x86 architecture can only address memory up to the 4GB range (so mapping it into this window allows it to be addressed, with the application handling the management overhead). That brings us to the second limitation, specifically that this window must be managed entirely by the app, as I mentioned previously - there is no using the OS's memory manager, as the OS does not know how to handle memory above the 4GB boundary, thus placing an overhead on the app. The last limitation is that the memory window *must* not contain executable code (again, the CPU's eip register doesn't understand addresses above 0xFFFFFFFF, the address limit of the x86 architecture).

It's a hack, and if you want to actually *really* use memory above 4GB, you *must* use a CPU architecture that supports it (Intel's Itanium ia64 or the AMD/Intel x86-64, or x64).

Consider reading:

http://blogs.technet.com/markrussinovich/a...21/3092070.aspx

http://blogs.technet.com/askperf/archive/2...e-s-my-ram.aspx

http://www.brianmadden.com/blogs/brianmadd...eally-mean.aspx

[BoardBabe]

Thanks for the informative reply.

[cluberti]

This is actually good to sticky, because it's only currently in the NT/2K/2K3 forum as a sticky, where people might not see it.

[Nihility]

I'm interested in knowing which programs support PAE.

Any idea which do?

I understand RAMDISKS are great with PAE, anything else?

[mats]

Applications wich uses large caches like databases for example. However since the mapping introduces a performace penalty it's better to got to true 64 bit soloutions if possible

[frozen78]

(...) However since the mapping introduces a performace penalty (...)

Hello everyone :-)

Mats, what do You mean by that? Does this "trick" decrease the overall performance of the system and applications?

I'd like to know how to enable this in Windows 7 beta. I'm just curious - as BoardBabe said earlier - just "for educational use ONLY".

Recently, when I was testing many Windows "tweaking" programs, I found one (which name I can't remember right now...) that had an option to "force" 32-bit Vista to see the memory above 4 GB. So maybe there is any registry hack, that would make it happen. Now I'm using 64-bit version of Win 7 beta (build 7000), but I'd like to install 32-bit version of the build 7022 and enable the option, which would allow the system to take advantage of all my memory. Just for my curiosity.

Any help on how to do that (step by step) would be appreciated.

Greetings.

[CoffeeFiend]

frozen78: read post #3 by cluberti.

Nothing will give you > 4GB on a 32 bit OS (besides the only one exception, on server OS'es, hardware permitting, which requires software to be specifically written for it like mats was referring to, and even then no code can run in there, just data).

You want 4GB or more, you move on to x64. It's that simple.

[Queue]

How is virtual memory handled when you have 4GB of RAM (well, when you've used up all addressable memory space)? I'd assume it's still used, so doesn't that give you more potential space to work with than the limit of 4GB? The main reason I ask this, is RAM above the 4GB limit has been made usable by RAMDISKs, so couldn't you put the swap file on said RAMDISK and, sort've, make use of all your RAM?

Queue

Edited February 19, 2009 by Queue

[CoffeeFiend]

How is virtual memory handled when you have 4GB of RAM

Virtual memory doesn't give you a bigger address space. It merely lets you swap pages in and out to/from disk.

RAM disks don't change anything to that picture at all. They can only be created inside RAM you can address in the first place. Unless it fiddles with the MMU directly (I wouldn't really want that personally), and even then it's pretty much an ugly hack if anything.

[cluberti]

How is virtual memory handled when you have 4GB of RAM (well, when you've used up all addressable memory space)? I'd assume it's still used, so doesn't that give you more potential space to work with than the limit of 4GB? The main reason I ask this, is RAM above the 4GB limit has been made usable by RAMDISKs, so couldn't you put the swap file on said RAMDISK and, sort've, make use of all your RAM?

Queue

Don't confuse virtual memory with actual, physical (RAM) memory. Virtual memory is just that - a virtual construct to map a virtual page that the OS and apps see, and the OS memory manager does all the work of mapping that virtual page address to a *real* memory address backed by *real* pages, either in physical RAM or in the swapfile. The RAMDISK binary is using a driver to handle this, not an application. Drivers can access anything they can be programmed to access, and "hide" things to fool windows on the other end. This is not the same.

[Queue]

Ok, I grasp that virtual memory isn't addressed in addition to RAM then, as in, it's not an extension, just, what, temporary storage? So, a single program couldn't use more than 4GB of RAM, regardless of how much virtual memory you have allocated, correct? For multi-tasking purposes though, multiple programs could all make use of (not realistically, but hypothetically) all 4GB of RAM each, if you had enough virtual memory to page out the RAM for each program as they take turns using the RAM?

Regardless of how crude or elegant a hack it is to make a RAMDISK out of RAM beyond the 4GB limit, wouldn't this still provide much faster handling of virtual memory? Obviously it couldn't be as good as being able to use the RAM above 4GB as real RAM, but doesn't it still provide interesting potential?

Queue

[cluberti]

Ok, I grasp that virtual memory isn't addressed in addition to RAM then, as in, it's not an extension, just, what, temporary storage? So, a single program couldn't use more than 4GB of RAM, regardless of how much virtual memory you have allocated, correct? For multi-tasking purposes though, multiple programs could all make use of (not realistically, but hypothetically) all 4GB of RAM each, if you had enough virtual memory to page out the RAM for each program as they take turns using the RAM?

Regardless of how crude or elegant a hack it is to make a RAMDISK out of RAM beyond the 4GB limit, wouldn't this still provide much faster handling of virtual memory? Obviously it couldn't be as good as being able to use the RAM above 4GB as real RAM, but doesn't it still provide interesting potential?

Queue

It's interesting, yes, and yes you are correct, no one app can use more than 2GB (or 3GB) of RAM, assuming it was using 100% of it's VA and it was 100% mapped into RAM.

However, the overhead just isn't worth it, plus the limitation of not being able to execute code there (even the driver would have to move the code down into a window inside the 4GB boundary to execute it like an app would).

[CoffeeFiend]

Ok, I grasp that virtual memory isn't addressed in addition to RAM then, as in, it's not an extension, just, what, temporary storage?

In layman's terms, no. When an app isn't active (like, minimized, not doing anything), then you can take the contents of whatever memory it fills, and write that to the hard drive. Then overwrite that physical RAM's content with other stuff you load from the hard drive, so you can execute it. When anything wants to interact with that program, it has to be loaded from disk first (which usually means writing another processes' memory to disk first). It allows more apps to share that same amount of RAM, but with a big performance hit.

So, a single program couldn't use more than 4GB of RAM, regardless of how much virtual memory you have allocated, correct?

On a standard 32 bit Windows version, a "typical" app can't access even near 4GB (there's less than 4GB available to the system in the first place, and then the kernel and such uses a large part of whatever is available). It's more like half that.

For multi-tasking purposes though, multiple programs could all make use of (not realistically, but hypothetically) all 4GB of RAM each, if you had enough virtual memory to page out the RAM for each program as they take turns using the RAM?

Nope. That same 2GB window of user mode RAM is shared by all apps. Virtual memory doesn't change anything to that. You want your apps to use 4GB or more, you have to move on to x64.

Regardless of how crude or elegant a hack it is to make a RAMDISK out of RAM beyond the 4GB limit, wouldn't this still provide much faster handling of virtual memory? Obviously it couldn't be as good as being able to use the RAM above 4GB as real RAM, but doesn't it still provide interesting potential?

Well, if the said hack BSODs your box, it's not that irrelevant... Sure, it would mean a faster page file, so faster paging, but that's about it. But with a x64 platform, your apps can actually use all of that RAM as they need (and not have to page nearly as much), and page whatever is unnecessary to disk if extra RAM is required, leaving even more free. Having that much RAM available makes some applications really fly (I had photoshop CS4 x64 up to 3.5GB recently on a pano shot, and it was still REAL fast). And that still doesn't address many of the other limitations (e.g. paged pool, non-paged pool, etc). And there's the page file size limit too... There's very few reasons not to move on to x64 anyways, so I don't really see the point of the ramdisk in the first place.