Jump to content

Puzzling Registry Size Issue


Dave-H

Recommended Posts

That's just bad news, isn't it? If there were a 9X version of the library, then the offline registry editor tool might be rebuilt and hopefully work OK with the 9X registry files. But as an offline registry editor works directly on the files, not the loaded registry via an API, how can an NT version be used, since 9X has two registry files, whereas NT has about half a dozen? In other words, if I were to (re)build NT registry files from exported 9X "reg" file(s), how would I merge the half dozen NT registry files into the two registry files that 9X requires?

It seems to me like you are still missing a point. :ph34r:

The Registry (online) is nothing but an assembly of 2 (win 9x) or 5 (NT) "databases" (or actually IMHO "filesystems") files.

See also:

http://www.911cd.net/forums//index.php?showtopic=24370&st=0&p=168043entry168043

Since you are seemingly old enough to remember good ol' DOS databases, you have to think to it as you would at a good ol DBaseIII (not that more modern databases are that much different ;)).

You had a number of tables containing DATA, that were assembled and accessed in an "assembled" way through a form, and the results (still coming form the assembly) were visualized through a report.

The normal online registry editor is nothing but a combined form/report "DBase" or "Clipper" app.

Of course, like you could use *any* dbase uility to access directly the tables, you can use *any* registry tool to access the single files composing the Registry (offline).

If you prefer, the Registry (online) as you are used to see it, is a building erected on-the-fly when booting with a number of bricks (and broken again into single elements when shutting down).

Nothing prevents you to access the bricks when they are not assembled all together.

jaclaz

Hi jaclaz,

I read this several times and gave it much thought. It is clear that there is something I don't understand, which to you is obvious. Perhaps it's this? :

The 9X and NT versions of 'regedit', when operating in offline mode, operate on 2/3 or 5 registry files simultaneously, placing each key/value/etc in the appropriate registry file, following some rules based on the particular hive or whatever. So when you create/update an offline registry, it is either 2/3 or 5 files, depending if you used the 9X or NT version of 'regedit'.

So in my mind, that's how I figured other offline tools would also work. So then the problem of converting 5 files into 2 ... could I simply use something like "copy /b a+b+c d"???

But I suspect now that other offline tools just work on a single registry file at a time, and it's up to the user to make sure it's the right one.

Have I got it now? :unsure:

Anyway, subsequent posts indicate more fundamental compatibility issues between 9X and NT registry files ...

AFAIK the structure/format of the registry database files on disk is different between 9x and NT OSes which is why offline registry editor apps for NT can't open 9x .dat files.

One utility which can apparently edit 9x registry dat files is the registered version of Regdat 1.3, however its website has disappeared and I am not sure whether it can be bought anymore and I don't know if it's better, besides being much more user friendly, than this russian editor running in a console which for editing is as limited as an hex editor.

Windows ME uses three files for the registry btw.

Thanks, loblo.

The descriptions for the two versions of RegDat here imply that you are correct (about the structure/format thing) : http://web.archive.org/web/20110202154630/http://regdat.com/

Joe.

Edited by jds
Link to comment
Share on other sites


@jds

I am sorry as I seemingly sent you after a red herring (re: 9x/Me vs. NT hives format) :blushing:

But the concept of the Registry being an assembly is important (and still valid) and you need to grasp it fully. :yes:

The Registry (as you see it in Regedit or actually how it is seen by the booted system) ONLY EXISTS when the system is booted (online).

What windows does when booting is to take 2 (9x), 3 (Me) or 5 (NT) "backing files" and assemble them in a "volatile" structure that is the Registry.

From the mouth of the wolf:

http://support.microsoft.com/kb/256986/en-us

Read attentively the explanation of HKEY_CLASSES_ROOT-

As another example (talking of NT which I am more familiar with) the hive

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet

does NOT exist (if not "online") as it is a redirection (or link, or "subst", whichever concept you like better) to one of the:

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet00n

(which do exist BOTH "online" and "offline"), to be more exact, the CurrentControlSet is a link to the ControlSet00n with n being the value of :

HKEY_LOCAL_MACHINE\SYSTEM\Select\Current

Try thinking of it as a common HTML page with frames, example:

http://www.tizag.com/htmlT/frames.php

In your browser you see a single page, but when you look at it in detail, what you see is not coming from a single file, but from a number of them that are "assembled" on the fly by the browser.

If you add to it some javascript that adds some "dynamic" contents on the fly, you have a reasonably accurate representation of the Registry as you see it in Regedit.... :unsure:

But I suspect now that other offline tools just work on a single registry file at a time, and it's up to the user to make sure it's the right one.

Yes and no.

Nothing prevents (as the Paul-Lee) thingy does, to make a program that does the same kind of "assembling" Windows does, so that you have a representation of the "offline" Registry similar to the one you would have for an "online" one.

jaclaz

Edited by jaclaz
Link to comment
Share on other sites

The descriptions for the two versions of RegDat here imply that you are correct (about the structure/format thing) : http://web.archive.org/web/20110202154630/http://regdat.com/

Make sure everyone grabs the two versions from the link posted above before they disappear. The HLP files are an important read and are vital to understand its capabilities. ... Download Links

The final known version is ( for NT ) is 4.3. I have just started looking at it myself and realized there is a lot to this program now. I have noticed it has several new functions, including the possibility of creating new hives. He also appears to have ironed out the security ACL's allowing direct editing AND saving. Unfortunately we will need to find the author, Henry Ulbrich, in order to get the thing registered to enable all the functions!

Anyway, this little tidbit jumps out of the HLP for v4.3 for NT based ( WinNT4/W2Kx/XP/Vista/W7 ) registry files:

"Repair Corrupt File

Instead of repairing the file, you can also try the binary export function ’Recreate Hive’ to export the hive into a new, non-corrupt registry file, including security information like SIDs and ACLs from the original file. This may be an even better solution when repairing the file causes too much loss of data."

The Win9x version does not have any binary output that I can see :realmad: which is a shame because this would solve many of the problems mentioned in this thread.

Edited by CharlotteTheHarlot
Link to comment
Share on other sites

As I understand it in Win 9x/Me Regedit displays 6 top keys which are:

HKEY_CLASSES_ROOT

HKEY_CURRENT_USER

HKEY_LOCAL_MACHINE

HKEY_USERS

HKEY_CURRENT_CONFIG

HKEY_DYN_DATA

HKEY_CLASSES_ROOT is a virtual key merely duplicating data from HKEY_LOCAL_MACHINE\SOFTWARE\CLASSES for the sake of convenience.

HKEY_CURRENT_USER is a virtual key duplicating data from HKEY_USERS for the currently logged in user.

HKEY_LOCAL_MACHINE includes everything that is in the system.dat file (9x) or everything that is in the file system.dat apart from the HKEY_LOCAL_MACHINE\SOFTWARE\CLASSES key which is in the file classes.dat (Me).

HKEY_USERS includes everything that is in the file user.dat.

HKEY_CURRENT_CONFIG is a virtual key duplicating data from a HKEY_LOCAL_MACHINE\Config subkey

HKEY_DYN_DATA is a completely volatile key which exist only in RAM during the time of the session and is not duplicated from any other key.

And btw, aside from Regdat, Henry Ulbrich also wrote RegExport which is freeware and is a must have utility IMO.

Edited by loblo
Link to comment
Share on other sites

Anyway, this little tidbit jumps out of the HLP for v4.3 for NT based ( WinNT4/W2Kx/XP/Vista/W7 ) registry files:

"Repair Corrupt File

Instead of repairing the file, you can also try the binary export function ’Recreate Hive’ to export the hive into a new, non-corrupt registry file, including security information like SIDs and ACLs from the original file. This may be an even better solution when repairing the file causes too much loss of data."

The Win9x version does not have any binary output that I can see :realmad: which is a shame because this would solve many of the problems mentioned in this thread.

Yes, however, I have no problem exporting from the registry using 'regedit', even when it's about 13MB+2MB. The difficulty is doing the reverse, either "live" or "offline", as 'regedit' crashes and other utilities seem to neglect an import capability.

As I understand it in Win 9x/Me Regedit displays 6 top keys which are:

HKEY_CLASSES_ROOT

HKEY_CURRENT_USER

HKEY_LOCAL_MACHINE

HKEY_USERS

HKEY_CURRENT_CONFIG

HKEY_DYN_DATA

HKEY_CLASSES_ROOT is a virtual key merely duplicating data from HKEY_LOCAL_MACHINE\SOFTWARE\CLASSES for the sake of convenience.

HKEY_CURRENT_USER is a virtual key duplicating data from HKEY_USERS for the currently logged in user.

HKEY_LOCAL_MACHINE includes everything that is in the system.dat file (9x) or everything that is in the file system.dat apart from the HKEY_LOCAL_MACHINE\SOFTWARE\CLASSES key which is in the file classes.dat (Me).

HKEY_USERS includes everything that is in the file user.dat.

HKEY_CURRENT_CONFIG is a virtual key duplicating data from a HKEY_LOCAL_MACHINE\Config subkey

HKEY_DYN_DATA is a completely volatile key which exist only in RAM during the time of the session and is not duplicated from any other key.

And btw, aside from Regdat, Henry Ulbrich also wrote RegExport which is freeware and is a must have utility IMO.

FWIW, I found this info somewhere on the web :

1. The HKEY_CLASSES_ROOT key provides a merged view of "HKEY_LOCAL_MACHINE\Software\Classes" (contains default settings that can apply to all users on the local computer) and "HKEY_CURRENT_USER\Software\Classes" (contains settings that apply only to the current interactive user).

2. The user-specific settings have priority over the default settings.

3. If you write values to a key under HKEY_CLASSES_ROOT, the values will go under:

* HKEY_CURRENT_USER\Software\Classes; if the key exists under HKEY_CURRENT_USER\Software\Classes.

* Otherwise, HKEY_LOCAL_MACHINE\Software\Classes.

As for RegExport, alas, I've seen a few other utilities with this capability, which as I've stated above, can be done quite satisfactorily with 'regedit'. Now, if he'd written an import utility, I'd consider that a "must have". :(

Joe.

Edited by jds
Link to comment
Share on other sites

As for RegExport, alas, I've seen a few other utilities with this capability, which as I've stated above, can be done quite satisfactorily with 'regedit'. Now, if he'd written an import utility, I'd consider that a "must have". :(

This program is just a reduced version of RegDat, a subset really, only consisting of the text export function to REG file, without the other bells and whistles, and the registration. Still, a definite must have!

Yes, however, I have no problem exporting from the registry using 'regedit', even when it's about 13MB+2MB. The difficulty is doing the reverse, either "live" or "offline", as 'regedit' crashes and other utilities seem to neglect an import capability.s.

Most likely, 'exporting' ( the ASCII output of the registry on a successfully booted computer ) will always work okay. I probably wasn't clear but what I meant was binary output from a registry tool that saves to a Win9x DAT on disk ( or a no-extension NT hive ) that is perfectly usable at a later bootstrap. Export is done while in protected-mode so by definition the registry was successfully read and loaded. At this point REGEDIT only has to write out the database. If you were to overload the registry with REG files, you could also export it out to a file. It's that next reboot which is scary.

My understanding is that there are two major but separate registry problems in the Win9x family ...

CREATION ... This occurs in DOS when the real-mode part of REGEDIT using /c cannot create one of the DAT files It is trying to rebuild a new registry ( output binary ) from a previously successful export ( input ASCII ). I discovered this way back in Win95 before the OSR updates corrected it ( temporarily ), when certain software moved all of its settings from private files into the registry and doubled the size. It was cat and mouse for the next few years but eventually even for Win98se large exports couldn't be /c created. Before WinME, the patching and backporting of a later REGEDIT version ( e.g., from 95osr to 95 ) allowed successful creation, but we ran out of luck when WinME changed REGEDIT ( it separated CLASSES out of the SYSTEM hive as a last ditch effort to make it manageable ). The reasons for this CREATION bug seem to be registry size, complexity, available RAM, ( and possibly CPU model, speed and cache ) so it is a chaotic and unpredictable problem, and pretty much unsolvable. Remember, this is the 16-bit universe and even before WinME Microsoft was already breaking things and leaving them broken. :realmad: Microsoft could have fixed this in the 32-bit portion instead, but I am guessing that they did not want to give REGEDIT an option to output binary DATs to an arbitrary selected folder which would be necessary since it cannot overwrite the current in-use registry files. A 32-bit registry tool that can do this ( output binary DATs to an arbitrary selected folder ) would be very desirable for Win9x, hence my previous comment.

LOADING ... BSOD protection error preventing successful bootstrap because one of the DATs cannot be successfully read or loaded. I had long thought that this was for all the same reasons, registry size, complexity, available RAM, etc, but here on MSFN RLoew has also discovered a connection to networking in some way. Consequently, a surprised and unprepared Win9x user is screwed at this point. However a well-prepared user has saved many DAT backups and thankfully in Win9x it is simple to change registries in F8 DOS command line. It is very possible while in Windows to absolutely overload the registry with imports ( or just install every SDK ) and it will appear fine as it fits within the protected-mode available memory space, but then it BSOD's on reboot. With great perseverance a person could manage this situation with lots of REG files used when in Windows and have REG 'deleters' to remove them before restart, and/or have fallback DATs ready to be copied in F8 DOS.

Interestingly enough, one program that I never saw choke on a large registry was RegCompact by Daniel Werner, a 32-bit protected mode app which near as I can tell reads the currently loaded registry in memory and writes it out sequentially, removes any fragmentation 'holes'. It uses WININIT.INI to do this by outputting the DATs as C:\WINDOWS\TEMP\RCxxxx.TMP that are swapped in on next reboot. In my experience it was always successful in this 'defragmentation, and, on the following reboot I don't recall any problems either. But if you stop and think about there shouldn't be any problems because what you are loading in these new' DATs is just a likely smaller, and purified version of the previous working registry. Theoretically it should become problematic when you start pumping in REG files just before you RegCompact.

The way I managed things on Win98se after the WinME registry change forked it up ...

  • Boot Win9x normally without BSOD
  • Backup DATs ( 'A' )
  • Import the required REG data in REGEDIT
  • Backup DATs ( 'B' )
  • Run RegCompact
  • Copy the newly created DATs cited in WININIT.INI, rename to DATs ( 'C' )
  • Now I have three sets of registry DATs, A, B, C

Then I would try booting C having A to fall back on. Note that B was saved just for informational purposes, because C is a purified and smaller version of B. If this concept is understandable and makes sense, then you should be able to do this successfully. Just remember that A, B, C represent three points along the timeline. I never did have a problem here but I never did push it for experimentation's sake. In other words I never imported huge amounts just to learn the limits of that specific computer ( once again assuming the BSOD is related to registry size, complexity, available RAM, Networking, CPU model/speed/cache, etc ).

Last registry size that I used on Win98se ...

SYSTEM.DAT ..... 12,873,760

USER.DAT ........ 3,665,952

ASCII Export ... 21,261,267

I just want to repeat - that registry was highly managed. I didn't just throw stuff into REG files to experiment and explore the outer limits! What I imported was highly error checked and optimized ( some might say obsessively :lol: ) , paths shortened, ~SFN~ removed, font and other paths, etc. And I had a habit of deleting large chunks of unnecessary information ( Shared DLLs, Windows setup info, etc ). If I had left that all in plus all the MSI installer garbage with the absolutely insane amounts of entries ( for every single file found in every SDK ), I bet that registry export might be be closer to 50 MB. The point is, those sizes shown DO NOT REPRESENT any kind of upper limits. Please, don't anyone assume those numbers should be used as guidelines!

NOTE: for RegCompact, there are four different versions that I have seen ...

73,728 ... 10-15-00 ... 4:36a ... Regcompact.exe_(2000-10-15)

73,728 ... 10-18-00 .. 12:39a ... Regcompact.exe_(2000-10-18)

73,728 ... 10-28-00 ... 5:16p ... Regcompact.exe_(2000-10-28)

73,728 ... 12-01-00 ... 8:33p ... Regcompact.exe_(2000-12-01) ... using this

What about this one (Commercial) :unsure: :

http://www.registrytool.com/features.htm

Can't say I ever saw it before. The only problem is it doesn't clearly mention the output of binary hives. Will have to check it out ... Someday.

EDIT: wording, spelling, mistakes

Edited by CharlotteTheHarlot
Link to comment
Share on other sites

The only problem is it doesn't clearly mention the output of binary hives. Will have to check it out ... Someday.

Well, no, it does mention them clearly (IMHO):

The editing commands, (add, modify, delete, Search and Delete, Search and Replace, Restore/Mass editing, and Send To), always target the Registry Tool Database. This feature enables you to edit registry backups and create custom or template Registry Tool Databases containing specific keys and values. It is an OPTION to have Registry Tool edit the active Windows registry. The Professional edition will target a remote machine. You may also elect to load ANY Dat/Hive Binary file, even from another machine/partition, and OPTIONALLY select the binary file as an editing target. You may edit the Dat/Hive binary file using one of the editing commands, or send keys and values from the Registry Tool Database into the Dat/Hive binary file. This feature enables you to fully edit any Dat/Hive binary file -- even create a new Dat/Hive binary file to replace a corrupt one..

BUt cannot say if it delivers what it states...:unsure:

jaclaz

Link to comment
Share on other sites

Somewhere (can't remember) I vaguely recollect using REG.EXE (from the Win98 Resource Kit) against the DAT files to "compress". There was some "method" to the madness... something about "creating" a new DAT file from the "exported" REG files? I could have SWORN I had used the method before...

(and I could be TOTALLY wrong, so don't be too harsh)

Edited by submix8c
Link to comment
Share on other sites

The only problem is it doesn't clearly mention the output of binary hives. Will have to check it out ... Someday.

Well, no, it does mention them clearly (IMHO):

The editing commands, (add, modify, delete, Search and Delete, Search and Replace, Restore/Mass editing, and Send To), always target the Registry Tool Database. This feature enables you to edit registry backups and create custom or template Registry Tool Databases containing specific keys and values. It is an OPTION to have Registry Tool edit the active Windows registry. The Professional edition will target a remote machine. You may also elect to load ANY Dat/Hive Binary file, even from another machine/partition, and OPTIONALLY select the binary file as an editing target. You may edit the Dat/Hive binary file using one of the editing commands, or send keys and values from the Registry Tool Database into the Dat/Hive binary file. This feature enables you to fully edit any Dat/Hive binary file -- even create a new Dat/Hive binary file to replace a corrupt one..

BUt cannot say if it delivers what it states...:unsure:

jaclaz

I stand corrected. Happily!

From what I can tell that feature ( binary DAT output ) is available in the free version, while the professional version adds networking functions. Yet another amzing find Jaclaz. :thumbup

EDIT: well there are several restrictions, particularly time :realmad: Click the button here ...

The Demo does not support the following Registry Tool Features:

(1) Validate File, which validates a filename and path, is inoperative.

(2) Registry Tool User Login and other internal security features are not supported.

(3) Tips and Notes do not recognize multiple logged in users for the user sections of the registry, (HKEY_USERS and its subkeys), thus creating new Tips and Notes for each user stored in the registry. The registered version disregards the currently logged in user and displays the Tips and Notes globally.

(4) The Demo is not networkable for Windows registry imports, restores, or edits.

(5) The Demo does not have a manual explaining the Windows registry sections, although it has a fully functional help file for the program itself.

(6) The Demo does not support the Search and Delete Command.

(7) The Demo expires after a set period of time, approximately 10 days.

(8) The Demo only imports directly from the Windows Registry 2 times.

(9) The Demo only restores to the Windows Registry 1 time.

(10) The dynamic Tips and Notes feature which displays default key and value Tips and Notes is only partially functional. For example, the hardware section isn't operational.

(11) The Demo only views NT 2000 XP 2003 security information, (Owner, SACL, DACL), it will not modify or restore security information.

Edited by CharlotteTheHarlot
Link to comment
Share on other sites

Somewhere (can't remember) I vaguely recollect using REG.EXE (from the Win98 Resource Kit) against the DAT files to "compress". There was some "method" to the madness... something about "creating" a new DAT file from the "exported" REG files? I could have SWORN I had used the method before...

(and I could be TOTALLY wrong, so don't be too harsh)

More than that, you could be opening a can of worms :w00t:

page__view__findpost__p__968052

;)

jaclaz

Link to comment
Share on other sites

As for RegExport, alas, I've seen a few other utilities with this capability, which as I've stated above, can be done quite satisfactorily with 'regedit'. Now, if he'd written an import utility, I'd consider that a "must have". :(

Joe.

You can't export reg files from a backed up registry dat file with regedit AFAIK.

RegExport is a great free tool and unique in its features. Besides letting you see differences between the live registry in ram with dat files or exporting from dat backups to selectively restore something you can use it to check out if you you've got a rootkit in the startup keys (run, runonce, etc...) for example.

I am not aware of an other tool that lets you do that besides its big brother RegDat and I am nearly 100% sure that no other free tool lets you do any of that so it's definitively a must have IMO.

Link to comment
Share on other sites

LOADING ... BSOD protection error preventing successful bootstrap because one of the DATs cannot be successfully read or loaded. I had long thought that this was for all the same reasons, registry size, complexity, available RAM, etc, but here on MSFN RLeow has also discovered a connection to video in some way ( driver, aperture, BIOS or something, I haven't had time to revisit this at all ). Consequently, a surprised and unprepared Win9x user is screwed at this point. However a well-prepared user has saved many DAT backups and thankfully in Win9x it is simple to change registries in F8 DOS command line. It is very possible while in Windows to absolutely overload the registry with imports ( or just install every SDK ) and it will appear fine as it fits within the protected-mode available memory space, but then it BSOD's on reboot. With great perseverance a person could manage this situation with lots of REG files used when in Windows and have REG 'deleters' to remove them before restart, and/or have fallback DATs ready to be copied in F8 DOS.

The problem I found isn't caused by the Registry not loading. It is caused by where and how much RAM is used by the loaded Registry. It is affected also by the Network Drivers, not by Video.

I added the /M Option to my RAM Limitation Patch to correct this problem. I was then able to load a Registry with an 18MB SYSTEM.DAT File.

Link to comment
Share on other sites

The problem I found isn't caused by the Registry not loading. It is caused by where and how much RAM is used by the loaded Registry. It is affected also by the Network Drivers, not by Video.

I added the /M Option to my RAM Limitation Patch to correct this problem. I was then able to load a Registry with an 18MB SYSTEM.DAT File.

Thanks, I am changing that sentence above in the original post ( and surname spelling, sorry ). Feel free to delete it from quoted my mistake.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...