How to debug out of memory issues in Windows 9x/ME?

[Goodmaneuver]

With your issue of not enough memory to run the program. Try running it from the run command. Click browse for the program and select it from run command window. After loading from run the programs may then be able to start from the desktop or start menu.

[Kahenraz]

It doesn't matter how you run the program. The memory has been consumed and isn't being released.

Unlike in Windows NT, I can't see any way of monitoring which process has allocated how much memory.

[Goodmaneuver]

It is probably a 16bit problem and that is reason to try replacing ati2cqag as it is the 2D driver. Codestuff Starter, HeapWalker and other NT SDK tools will work showing 16bit processes. You may need to use KernelEx's PSAPI.dll and KernelEx for some SDK programs.

[Kiriko Takemura]

On 7/31/2022 at 5:16 PM, Kahenraz said:

Does Windows 98 support JIT debugging? Maybe this can provide some more information as to what causes Notepad to crash.

Kind of, but you are in for the experience of a lifetime (but not in a good way). Debugging Win9x is not very easy if you are spoiled by modern debuggers. How good is your x86 asm skills? Mine are poor.

Whatever you do, basically you will need a second machine connected to the problematic machine using a null modem serial cable. Please, consider the pinout:

9 Pins
3  2  TX
2  3  RX
5  5  GND

25 Pins
2  3  TX
3  2  RX
7  7  GND

Where to find debugger:

• Windows 98: Win98ddk.exe (19987736 bytes)
• Windows 98SE: Win98ddk.exe (26377712 bytes)
• Windows Me: winmeddk.exe (1469056 bytes)

Where to find the files in 2022? No ideas. Not at Microsoft.com, that is for sure.

Anyway, Once you found DDK https://bikodbg.com/blog/2021/08/win98-ddk/ has a nice setup guide, but I replicate here because websites tend to disappear:

 

Quote

    A Windows 98 ISO.
        For instance, from WinWorld. Be sure to get "OEM Full" version.
    The DDK.
        For instance, this page has a bunch of useful downloads.
        Specifically, get the complete DDK with patches (that's what worked for me): https://www.mdgx.com/spx/98DDK.RAR.
    Visual C++ 5.0.
        Again, WinWorld to the rescue.


    Install Windows on a VM.
        WinWorld says that Windows 98 might have trouble running on a VM, so you may want to consider 86Box or PCem instead.
        In any case, make sure the VM has a serial port exposed to the host (through a pipe or some such).
    Install Visual C++.
        To be on the safe side, install everything.
    Install the DDK.
        Again, install everything.
        Important: increase the environment space by adding the following line to config.sys: shell=c:\command.com /p /e:4096
    Check the installation by building the samples.
        Open a "Checked Build Environment" command prompt from the start menu, go to C:\98DDK\src, and run build -cZ
        There will be a few errors, but most of the samples will compile. The binaries will be at C:\98DDK\lib\i386\checked.
    Check that debugging works.
        Reboot into DOS mode and run \98ddk\bin\wdeb386 /c:1 /r:19200 \windows\win.com
            This starts the debugger on COM port 1 (on the guest), with a baud rate of 19200, and instructs it to run the kernel at \windows\win.com.
        On the host, connect to the VM's serial port with the correct baud rate and press Ctrl-C.
            The debugger should break in (see screenshot below).
        Enter g to resume execution.
    Make sure the debugger starts on boot.
        Add the following line to config.sys: DEVICE=C:\98ddk\bin\wdeb386.exe /c:1 /r:19200
            Note: I'm not sure about the importance of the line order in this file. In my setup this line comes before the one that increases environment space (see screenshot below).
        Reboot and check that this works. On boot, the debugger should print a version string (see screenshot below).

Note: the debugger we're using here is WDEB386. There's a better debugger installed with the DDK — WDEB98 — but it requires the use of a special client — RTerm98 — which complicates matters. So that we can use PuTTY, we're sticking with the old debugger.

 

On another website http://www.massmind.org/techref/language/asm/debugs.htm there is the information that win 2000 SDK has a nice GUI for WDEB386. I never used it though.

Quote

 95,98,NT: The Windows 2000 Platform SDK has a (very nice) GUI and remote (for kernels) debugger for Windows 9x & NT available free at http://www.microsoft.com/msdownload/platformsdk/windbg.htm (cache)
The debugger tool provided on the Windows 98 DDK is the Windows System Debugger (Wdeb386.exe) (docs). It requires a remote computer connected via a cable to the target computer. Debug binaries and components are provided for Windows 98 in the \dbg_sym directory on the Windows 98 DDK distribution CD.

    Note that if you are viewing a range of memory starting before a section that is committed and ending inside the section that is committed, all the memory will show as being undefined ("??" in place of values). For example, if you VirtualAlloc some memory with MEM_COMMIT and then point edi at the first byte of it and view edi-20 (so you can later see what was stosb'd to edi) the entire view will fill with "??" If "Reevaluate Expression Always" is enabled for the view, when edi advances enough that then entire view is defined, the ?'s will suddenly dissappear.

 

Here is the manual for the debugger:

https://anonfiles.com/61jdP7D0y2/wdeb386-debugger_pdf

 

Someone, please mirror it because if it gets lost, the next person stopping by here will be sad.

 

If you do manage to debug the kernel far enough to find the root cause please let Raymond Chen at Microsoft know so you can get a commemorative plaque.

 

[pangoomis]

There's also the unfixable at the moment VCACHE error that appears on the newest motherboards:

https://msfn.org/board/topic/183200-z690-vcache-protection-error/

Maybe this info can help to troubleshoot this too...

[caroljames972022]

I have been facing the same issue in my system. I replaced my memory https://www.allhdd.com/samsung-m393b2g70bh0ck0-memory/ and tried again. It is working perfectly fine now.