jumper

I would open the file and map it as an array of dwords (using an existing function). If the data was presorted, a simple binary search would quickly find the count. Otherwise you're right, 73KB would still be too big for the PrivateProflle functions: Each line in an INI file would need one byte for the '=', one byte for the count, and one byte for the EOL marker. That leaves less than one byte for the name/hash/crc string! We might be able to get to 24 bits if we can: get below 16k functions by removing those with the most common count (1 or 2 maybe?); that count would become the default for functions not found group functions into sections and reclaim the '=' and <count> bytes. Early versions of ImportPatcher could batch process multiple files. When I added INI support, I found SE wasn't letting me access multiple PrivateProfile INI files. Once I picked one, I had to stick with it. So I dropped batch processing. Unless I can figure out how to easily work around this issue, even a single INI (in addition to the main #.ini) might not be an option. I don't think NtQueryInformationProcess can be static linked to--no import library for it. Though for completeness, a good idea to add it. Thanks.

Here is the aforementioned INI file with 18301 API parameters counts: APIParameterCounts.zip Last week I did try accessing it from IP using GetPrivateProfileString and it worked great--the first 64KB that is. The other 406KB - 64KB can't be accessed that way. I converted it to a REG file, hoping to be able to access it from the transient portion of the registry with RegQueryValue, but REG files can only add to the on-disk keys and that also takes several minutes. The current plan is to do a quick 27-bit hash or crc on the function names to reduce the data size (27+5 for the [0..23] count = 32). Or I might just split the data over seven INI files.

The trailing spaces are an indentation for a missing log message. This only happens when wsprintf encounters an error (usually an access violation in one of the parameters) and emits no text. I was able to reproduce this condition in a way that resembled divad's log file by intentionally using an invalid pointer for the delay-import ILT address. I then modified my PEfinder test app to search my local drives for apps with invalid delay-import ILT addresses. I found five. All had been UPX'ed. UPX compression abbreviates the import tables and corrupts the delay-import table (if any). This works because the system loader doesn't check the delay-import table and the UPX decompressor restores it for normal use later. Tools like Dependency Walker can only report that the address is invalid. Other compressors may also corrupt the delay-import table, but I found no such examples on my local drives. PEfinder also uncovered one app that stored the parallel ILT and IAT tables in different sections. In this case it wasn't a problem, but theoretically it could be. So I rewrote the rva-to-pointer routine to lookup every address in the section tables without making any assuptions, no matter how reasonable they might seem to be! Well, PEfinder only opens one file at a time, while ImportPatcher opens two (or more if walking) at a time. So the new rva-to-pointer routine was difficult to port and required lots of extra support code and modifications. However, IP.35 now seems to be working and will be released as soon I finish regression testing it.

I don't follow. The hypertrm.dll is from Winxp. I don't use kernelex. I only install it when testing the service pack. On December 19, I extracted all files from all four cabinets of the 'final' version of U98SESP3. These six files were in there. If you haven't removed HYPERTRM.DLL (and *.EXE, *.CHM, *.HLP) from SP3.CAB, then they are still in there. BTW, how did U98SESP3 go from "final" last month to "beta" this month? Shouldn't this beta be 3.1?

Yes, it's a great wish. I've quoted the entire discussion and replied on the 2012 Project Wish List thread in post #39.

The following discussion has Wish List written all over it. Would it be possible to limit the resources available to an app in any way other than to hook the various allocation and creation APIs to fail if the calling process has reached its limit?

These files all make delay-load calls to PrintDlgExA in COMDLG32.DLL which requires KernelEx (or a 2K version, not in this pack): HYPERTRM.DLL MFC70.DLL MFC70U.DLL MFC71.DLL MFC71U.DLL TWEAKUI.CPL

Have you tried FineSSE?

I use TaskInfo2000 v2.1 by Igor Arsenin Amongst many other things, for each process it lists: Data KB (total, in memory, in use) Code KB (total, in memory, in use) Handles count Windows It uses low resources itself, updates every two second, and shows most of what VWin32 reports (and much more).

As it turns out, the loader will append '.dll' to an import DLL name as needed, so 'msvrt.dll' can be replaced with 'msvcr70' without needing to rename it to 'msvcr7.dll'. Ignore the mention of ordinals for now. IP won't support replacement-by-ordinal until the next version. If you modify your Inkscape patch to use IPstub.dll, it should work. Otherwise try patching a copy of IP itself as I showed. IPstub.dll works the same as your dnsapi.dll/stub.dll, but with a wider variety of stubs and diagnostics functions to choose from. Function names are all two characters to ensure they're not too long: p1 p2 p3 p4 p5 p6 p7 p8 p9 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 o0 o1 o2 o3 o4 o5 o6 o7 o8 o9 t0 t1 t2 t3 t4 t5 t6 t7 t8 t9 yn op bp If the functions aren't being called, any function will work (just like in your dnsapi.dll). Otherwise choose a stub with the desired return code and parameter count. (A quick search of MSDN should yield these.) If you run IP.34 on Inkscape you should be able to make this replacement in the .ini file: [DLL substitutions] dnsapi.dll=IPstub.dll Then [ Retry ] and: [IPstub.dll] DnsQuery_A=f6 DnsRecordListFree=f2 [ Retry ] again to patch. IPstub.dll can be in the app folder, <system>, <windows>, or anywhere in the normal PATH. Just the type of feedback I love to hear! It was always optional (editable in .ini), but defaulted to 'Y' if walking and 'N' if not.

Searching for "windows api parameter count" lead me to these header resources: Jeremy Gordon's Go Tools site -> #headers Donkey's Stable -> GoAsm Headers -> HEADERS.ZIP Bryant Keller's header files: WIN32A.zip Expanding WIN32A.ZIP, I found WIN32P.INC and the following: _LocalEnroll() requires 23 arguments. _LocalEnrollNoDS() requires 23 arguments. These seem to be from CRYPTUI.dll I've used DOS FIND to extract the lines containing 'requires' and then the DevStudio '97 text replace function to create an .ini file with 17986 18300 API parameter counts: [ParameterCounts] CreateSecurityPage=1 EditSecurity=2 IID_ISecurityInformation=1 ADsBuildEnumerator=2 ...

Fortunately CreateFontA has been in GDI32 since Win32s so we don't need a stub for it. We can cross other bridges when we come to them. Do you know of any recent comprehensive lists of functions similar to the old WIN32API.CSV? I may need to bite the bullet and download a recent platform SDK, then look at the header files.

IPstub.dll is a library of 42 small functions that can be used to plug holes left by missing imports. There are four basic stub families and three debugging stubs. Basic stub families: n = 0..9 (0 to 9 32-bit parameters) pn (p1..p9): return <parameter 1> fn (f0..f9): return flast / 0 on (o0..o9): return one / 1 tn (t0..t9): return true / -1 Ordinal assignment (@1..@39): 4*n + { pn:0 | fn:1 | on:2 | tn:3 } (There is no p0: can't return 1 of 0 parameters, ordinals start at 1) Debugging stubs: yn @ 40 : Yes/No/Cancel messagebox [ Yes ] returns true [ No ] returns false [ Cancel ] calls ExitProcess(-1) - zero parameters op @ 41 : Cascading Yes/No/Cancel messageboxes [ Yes ] returns true [ No ] returns false [ Cancel ] invokes 2nd messagebox [ Yes ] returns <param1> [ No ] returns 1 [ Cancel ] calls ExitProcess(-1) - one parameter [*] bp @ 42 : calls MessageBeep (MB_ICONHAND), returns 0, zero parameters Tested with IP.7 (first to display usage MessageBox): [ImportPatcher.34] ;Edit parameters and replacement strings, then Retry or run again to patch. <= [Parameters] Walk dependencies=N Link to copies=N Unbind broken bindings=N Target OS=4.10 [DLL substitutions] USER32.dll=IPstub.dll [IPstub.dll] MessageBoxA=op wsprintfA=yn [Patch list] ip7.exe=DLLs, Functions Fun, fun! Did I mention it comes with source code?

After modding PEfinder to search for local files without ILTs, I discovered it's not just a few files from old linkers (as mentioned in one of the classic '90s PE guides). Many new apps (including a .NET installer!) suffer from this malady. So I quickly added support for missing ILTs and a related unbinding issue last night and posted IP.34 a few minutes ago. I don't think this was the problem, but uninitialized variables and bad pointers are leading causes of sporadic program behavior. In the course of this investigation, I also noticed that DW reports that COMCTL32.DLL and USER32.DLL both want to load at the same preferred base! That means every time the second one loads, there is a performance hit as it is relocated.

It looks like either your overclocked memory glitched or your copy of shell32.dll has been corrupted. ... Importing from module: 'SHELL32.dll' TimeDateStamp: 3c106ecb Target OS: 4.0 (300) Shell_NotifyIconA (288) ShellExecuteExA (224) SHGetSpecialFolderPathA Importing from module: 'KERNEL32.DLL' TimeDateStamp: 3caba233 Target OS: 4.0 Clues: These TimeDateStamps for Shell32 and Kernel32 match those in SP3. Kernel32 is processed higher in the log with no problems. I can successfully walk Shell32 directly with IP.33 and also indirectly via Explorer. Shell32 should be importing from GDI32 first, not Kernel32 (confirmed with DW and other sources). Please try analyzing Shell32 directly, with and without walking dependencies. Also see if you can analyze coretemp10rc2_1236.exe (Walk=N), then try to reproduce the error with Walk=Y. Meanwhile, I'll continue to investigate the twelve trailing spaces that don't seem to jive with the rest of the clues....

The ini and log files would have been nice to see, but looking at what we have: Image base is $400000 Code base is $401000 Map file says: 0001:0000038a _PatchFile 0040138a f ImportPatcher.obj so error is at b91 - 38a = 807 in PatchFile() Cod file says: ; 682 : wsprintf (szBuff, "%s\t(%d)\t* not found", pSearch, ByName->Hint); 00804 8b 45 e8 mov eax, DWORD PTR _ByName$17417[ebp] 00807 0f b7 00 movzx eax, WORD PTR [eax] A function search by name has just failed. Eax appears just a tad high, so must be pointing past end of file mapping. From the information here, it looks like the file is truncated. Or maybe it was produced by an early Borland linker without an ILT: Such files can't be bound, so evidence of binding in the log file would discount this theory.If the file works in IP.32, it is likely a problem with the delay-import data stuctures. A DW test would also be a good idea. Please post the text of the .ini and .log files (in spoilers if large). Thank you for testing!

IP.33 is finally done. VC6-style delay load imports took longer than expected. Some design changes were needed that affected a lot of the little details. After much testing and code clean-up, the only casualty was 'Link to copies.' Since this feature is only needed when walking and patching dependent DLLs, I decide not to hold up the release any longer. From ImportPatcher.c: // To do: // create DLL with families of stubs with various parameter counts and return values // fix 'Link to copies' // if bind mismatch, unbind by restoring parallel lists, then reprocess // if ILT null, set to IAT // list imports even if DLL missing or has no exports // ordinal support: replace, check, look up name // function substitution within a bound DLL implies unbinding! // display TimeDateStamp as words // custom file search path order w/o app folder, w/ KnownDLLs // batch process a folder of files: no-walk analyze only; first/last MB or progress window // dialog box interface // stub insertion // Future expansion: // create inf un/installer for patched file(s) // After patching, launch // ExportPatcher: add to a DLL's exports function forwards to a custom DLL

With R70 or R71 subbed for RT in <system>, DevStudio'97 / VC++5.0 gives me dual GPF's in first DM.DLL (Debuggee Module for WinDbg) and then in MFC42.DLL when I try to run a release build. At this point, using KnownDLLs to sub in and out SP3 modules on-the-fly became invaluable--great tip, loblo! Backing out the RT sub, I reloaded VC5, reloaded the project (IP.33), and clicked 'Run'; no problem. Subbed RT 6.10.9848.0, reloaded VC5; no problem. Subbed MSVCP50 version 5.00.7051 for MSVCP50; no problem. Subbed MFC42.DLL version 6.02.8081.0 for MFC42.DLL; Dual GPF's in DM and MFC42. I read yesterday about MSVC interdependencies and backwards compatibility at fighting-the-msvcrt-dll-hell (See great comment by Andrew on August 9). Remembering that, instead of backing out the MFC42 sub, I also subbed R71 back in...and it worked! (R70 also works.) (Note: this testing was all done with KernelEx 4.52 installed and defaulting to 'off', my normal setup.) So now I'm worried about M*10,20,30,40, etc. interacting with R70/71 subbed as RT. I haven't actually installed SP3, so this would be a great place for a Newby to jump in, install SP3 then sub in R70/71 for RT and begin testing with older apps. Building a test suit of small apps that exercise various legacy DLL's would be a great help. I'll add this to the Wish List.

About three weeks ago U98SESP3.EXE contained: 6.10.9848.0 7.00.9981.0 7.10.7031.4

> I am now using msvcr70 version 7.00.9955.0 as my default msvcrt in the sys dir and so far so good. Me too, only I'm using msvcr71. Now I've put 7.00.9981 in my sysdir and set: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs MSVCRT=MSVCR70 After a restart, MSVCR70 is indeed being used by apps instead of MSVCRT (7.1). So KnownDLLs can be used to redirect modules! I was able to confirm Neither msvcr70 or msvcr71 cover all of the NT Msvcrt exports. msvcr70 is missing: _get_heap_handle msvcr71 is missing: _ctype but not void * bad_cast::`scalar deleting destructor'(unsigned int) void * bad_cast::`vector deleting destructor'(unsigned int) void * bad_typeid::`scalar deleting destructor'(unsigned int) void * bad_typeid::`vector deleting destructor'(unsigned int) void * exception::`scalar deleting destructor'(unsigned int) void * exception::`vector deleting destructor'(unsigned int) void * __non_rtti_object::`scalar deleting destructor'(unsigned int) void * __non_rtti_object::`vector deleting destructor'(unsigned int) Both 70 and 71 have ??0bad_cast... and so forth so I don't see how they differ. What tool did you use to interpret the encoded function names (and presumably find the above missing from 71)? If I make a patch to add a _get_heap_handle stub to msvcr70, we may have a nice solution. Otherwise we can try to get the real NT msvcrt itself working...!

I've copied the msvcr71.dll from SP3 into my system folder. I'm shutting SE down for the night, so I'll exit to DOS and make the switch with msvcrt.dll. In the morning I'll boot with the 'updated' msvcrt.dll and exercize a bunch of apps. If all goes well, I'll try modifying HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs to make the redirection (in the system folder). Perhaps this key can be used to make module subs on-the-fly without needing help from ImportPatcher. That would be great for redirecting delay-load modules. ImportPatcher.32 is now posted. I spent a couple of hours Saturday creating PEfinder, a tool that scans all files in a given folder and all sub-folders for valid PE headers. A stripped-down version of ImportPatcher, it filters on any DOS/PE/Optional/Directory field I specify (in the src) and outputs a log file with the filename and field value of the matches. (No UI or run-time options.) Compared to the dozen or so files I had been testing IP with, I was able to test with thousands of files in just a few minutes. And I found three problems: Empty files caused CreateFileMapping to fail (like dencorso reported) 16-bit apps without full DOS header lacked a valid e_lfanew field leading to GPF if random value pointed past EOF 64-bit modules are PE32+ with longer Optional header, moving Directory table back 16 bytes Fixes for these issues were ported back to ImportPatcher.32. I also added the bonus character for even-length string names as recently discussed.

Those using ImportPatcher on 2K, XP, or later might find some registry left-overs in: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\IniFileMapping The PrivateProfileString functions I use to manage .ini files don't use the registry in 9x, but supposedly do in NT. I plan to move away from the use of .ini files in the near future, so hopefully this problem for NT users will go away soon. -jumper

So msvcr70 and msvcr71 seems to be good replacements for msvcrt. Another way to do the substitution is to put a copy renamed to msvcrt directly in the folder the the app and its dll's. And with more backwards compatibility testing, perhaps msvcr71 can be used as a direct update for msvcrt in the <system> folder. (Oops...Did I just volunteer again )

Wow, great work loblo! Is KernelEx helping at all? And how did you determine what code to put into the stubs? GTK is a popular toolkit. Qt is another. If we can get them and .Net working, a lot of apps will start to work.

Joe, KernelEx v4.5 Beta 1 added a stub for HeapSetInformation, so you can leave that function substitution blank. As for msvcrt.dll, try subbing 71 instead of 90: I searched my HDD for "msvc*.dll" and came up with 36 hits (including dups). Refining the search to files containing text: "__uncaught_exception" reduced that count to 18 and these seven unique files: msvcr70.dll msvcr71.dll msvcr90.dll msvcp70.dll msvcp71.dll msvcp90.dll msvcm90.dll r = C Run-time (CRT) m = managed (.Net) p = C++ Run-time??? (Bonus points to first responder) All seven of these missing functions appear to have been introduced in MSVC++ 7.0: [msvcrt.dll] __uncaught_exception ___lc_handle_func ___lc_codepage_func ___mb_cur_max_func __pctype_func __iob_func __crtLCMapStringW __crtGetStringTypeW These 8 "functions" (variables probably) can be found in msvc 4 through 7.1, but were removed by 9: [msvcr9.dll] ??0exception@@QAE@ABQBD@Z= ?what@exception@@UBEPBDXZ= ??1exception@@UAE@XZ= mktime= ??0exception@@QAE@ABV0@@Z= ??0exception@@QAE@XZ= ??1bad_cast@@UAE@XZ= ??0bad_cast@@QAE@ABV0@@Z= I don't have "8" on my system...maybe someone else can check it for these exports. Good move--I forgot to count when I suggested it. Because table entries are word-aligned, there should actually be an extra byte available to even-lengthed strings (NULL terminator makes them odd). An additional byte or two can (usually) be stolen from the word-sized hint of the following hint-string pair. (At least one linker out there has a bug that does overlap entries half the time, making optimizing hints impossible without completely bulding the table!) Since "msvcrt.dll" is even, look for the next beta to support subbing up to: ((length&-2)+1) p.s. I hope everyone is having as much fun as I am. Thanks everyone!