jumper

As it turns out, the loader will append '.dll' to an import DLL name as needed, so 'msvrt.dll' can be replaced with 'msvcr70' without needing to rename it to 'msvcr7.dll'. Ignore the mention of ordinals for now. IP won't support replacement-by-ordinal until the next version. If you modify your Inkscape patch to use IPstub.dll, it should work. Otherwise try patching a copy of IP itself as I showed. IPstub.dll works the same as your dnsapi.dll/stub.dll, but with a wider variety of stubs and diagnostics functions to choose from. Function names are all two characters to ensure they're not too long: p1 p2 p3 p4 p5 p6 p7 p8 p9 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 o0 o1 o2 o3 o4 o5 o6 o7 o8 o9 t0 t1 t2 t3 t4 t5 t6 t7 t8 t9 yn op bp If the functions aren't being called, any function will work (just like in your dnsapi.dll). Otherwise choose a stub with the desired return code and parameter count. (A quick search of MSDN should yield these.) If you run IP.34 on Inkscape you should be able to make this replacement in the .ini file: [DLL substitutions] dnsapi.dll=IPstub.dll Then [ Retry ] and: [IPstub.dll] DnsQuery_A=f6 DnsRecordListFree=f2 [ Retry ] again to patch. IPstub.dll can be in the app folder, <system>, <windows>, or anywhere in the normal PATH. Just the type of feedback I love to hear! It was always optional (editable in .ini), but defaulted to 'Y' if walking and 'N' if not.

Searching for "windows api parameter count" lead me to these header resources: Jeremy Gordon's Go Tools site -> #headers Donkey's Stable -> GoAsm Headers -> HEADERS.ZIP Bryant Keller's header files: WIN32A.zip Expanding WIN32A.ZIP, I found WIN32P.INC and the following: _LocalEnroll() requires 23 arguments. _LocalEnrollNoDS() requires 23 arguments. These seem to be from CRYPTUI.dll I've used DOS FIND to extract the lines containing 'requires' and then the DevStudio '97 text replace function to create an .ini file with 17986 18300 API parameter counts: [ParameterCounts] CreateSecurityPage=1 EditSecurity=2 IID_ISecurityInformation=1 ADsBuildEnumerator=2 ...

Fortunately CreateFontA has been in GDI32 since Win32s so we don't need a stub for it. We can cross other bridges when we come to them. Do you know of any recent comprehensive lists of functions similar to the old WIN32API.CSV? I may need to bite the bullet and download a recent platform SDK, then look at the header files.

IPstub.dll is a library of 42 small functions that can be used to plug holes left by missing imports. There are four basic stub families and three debugging stubs. Basic stub families: n = 0..9 (0 to 9 32-bit parameters) pn (p1..p9): return <parameter 1> fn (f0..f9): return flast / 0 on (o0..o9): return one / 1 tn (t0..t9): return true / -1 Ordinal assignment (@1..@39): 4*n + { pn:0 | fn:1 | on:2 | tn:3 } (There is no p0: can't return 1 of 0 parameters, ordinals start at 1) Debugging stubs: yn @ 40 : Yes/No/Cancel messagebox [ Yes ] returns true [ No ] returns false [ Cancel ] calls ExitProcess(-1) - zero parameters op @ 41 : Cascading Yes/No/Cancel messageboxes [ Yes ] returns true [ No ] returns false [ Cancel ] invokes 2nd messagebox [ Yes ] returns <param1> [ No ] returns 1 [ Cancel ] calls ExitProcess(-1) - one parameter [*] bp @ 42 : calls MessageBeep (MB_ICONHAND), returns 0, zero parameters Tested with IP.7 (first to display usage MessageBox): [ImportPatcher.34] ;Edit parameters and replacement strings, then Retry or run again to patch. <= [Parameters] Walk dependencies=N Link to copies=N Unbind broken bindings=N Target OS=4.10 [DLL substitutions] USER32.dll=IPstub.dll [IPstub.dll] MessageBoxA=op wsprintfA=yn [Patch list] ip7.exe=DLLs, Functions Fun, fun! Did I mention it comes with source code?

After modding PEfinder to search for local files without ILTs, I discovered it's not just a few files from old linkers (as mentioned in one of the classic '90s PE guides). Many new apps (including a .NET installer!) suffer from this malady. So I quickly added support for missing ILTs and a related unbinding issue last night and posted IP.34 a few minutes ago. I don't think this was the problem, but uninitialized variables and bad pointers are leading causes of sporadic program behavior. In the course of this investigation, I also noticed that DW reports that COMCTL32.DLL and USER32.DLL both want to load at the same preferred base! That means every time the second one loads, there is a performance hit as it is relocated.

It looks like either your overclocked memory glitched or your copy of shell32.dll has been corrupted. ... Importing from module: 'SHELL32.dll' TimeDateStamp: 3c106ecb Target OS: 4.0 (300) Shell_NotifyIconA (288) ShellExecuteExA (224) SHGetSpecialFolderPathA Importing from module: 'KERNEL32.DLL' TimeDateStamp: 3caba233 Target OS: 4.0 Clues: These TimeDateStamps for Shell32 and Kernel32 match those in SP3. Kernel32 is processed higher in the log with no problems. I can successfully walk Shell32 directly with IP.33 and also indirectly via Explorer. Shell32 should be importing from GDI32 first, not Kernel32 (confirmed with DW and other sources). Please try analyzing Shell32 directly, with and without walking dependencies. Also see if you can analyze coretemp10rc2_1236.exe (Walk=N), then try to reproduce the error with Walk=Y. Meanwhile, I'll continue to investigate the twelve trailing spaces that don't seem to jive with the rest of the clues....

The ini and log files would have been nice to see, but looking at what we have: Image base is $400000 Code base is $401000 Map file says: 0001:0000038a _PatchFile 0040138a f ImportPatcher.obj so error is at b91 - 38a = 807 in PatchFile() Cod file says: ; 682 : wsprintf (szBuff, "%s\t(%d)\t* not found", pSearch, ByName->Hint); 00804 8b 45 e8 mov eax, DWORD PTR _ByName$17417[ebp] 00807 0f b7 00 movzx eax, WORD PTR [eax] A function search by name has just failed. Eax appears just a tad high, so must be pointing past end of file mapping. From the information here, it looks like the file is truncated. Or maybe it was produced by an early Borland linker without an ILT: Such files can't be bound, so evidence of binding in the log file would discount this theory.If the file works in IP.32, it is likely a problem with the delay-import data stuctures. A DW test would also be a good idea. Please post the text of the .ini and .log files (in spoilers if large). Thank you for testing!

IP.33 is finally done. VC6-style delay load imports took longer than expected. Some design changes were needed that affected a lot of the little details. After much testing and code clean-up, the only casualty was 'Link to copies.' Since this feature is only needed when walking and patching dependent DLLs, I decide not to hold up the release any longer. From ImportPatcher.c: // To do: // create DLL with families of stubs with various parameter counts and return values // fix 'Link to copies' // if bind mismatch, unbind by restoring parallel lists, then reprocess // if ILT null, set to IAT // list imports even if DLL missing or has no exports // ordinal support: replace, check, look up name // function substitution within a bound DLL implies unbinding! // display TimeDateStamp as words // custom file search path order w/o app folder, w/ KnownDLLs // batch process a folder of files: no-walk analyze only; first/last MB or progress window // dialog box interface // stub insertion // Future expansion: // create inf un/installer for patched file(s) // After patching, launch // ExportPatcher: add to a DLL's exports function forwards to a custom DLL

With R70 or R71 subbed for RT in <system>, DevStudio'97 / VC++5.0 gives me dual GPF's in first DM.DLL (Debuggee Module for WinDbg) and then in MFC42.DLL when I try to run a release build. At this point, using KnownDLLs to sub in and out SP3 modules on-the-fly became invaluable--great tip, loblo! Backing out the RT sub, I reloaded VC5, reloaded the project (IP.33), and clicked 'Run'; no problem. Subbed RT 6.10.9848.0, reloaded VC5; no problem. Subbed MSVCP50 version 5.00.7051 for MSVCP50; no problem. Subbed MFC42.DLL version 6.02.8081.0 for MFC42.DLL; Dual GPF's in DM and MFC42. I read yesterday about MSVC interdependencies and backwards compatibility at fighting-the-msvcrt-dll-hell (See great comment by Andrew on August 9). Remembering that, instead of backing out the MFC42 sub, I also subbed R71 back in...and it worked! (R70 also works.) (Note: this testing was all done with KernelEx 4.52 installed and defaulting to 'off', my normal setup.) So now I'm worried about M*10,20,30,40, etc. interacting with R70/71 subbed as RT. I haven't actually installed SP3, so this would be a great place for a Newby to jump in, install SP3 then sub in R70/71 for RT and begin testing with older apps. Building a test suit of small apps that exercise various legacy DLL's would be a great help. I'll add this to the Wish List.

About three weeks ago U98SESP3.EXE contained: 6.10.9848.0 7.00.9981.0 7.10.7031.4

> I am now using msvcr70 version 7.00.9955.0 as my default msvcrt in the sys dir and so far so good. Me too, only I'm using msvcr71. Now I've put 7.00.9981 in my sysdir and set: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs MSVCRT=MSVCR70 After a restart, MSVCR70 is indeed being used by apps instead of MSVCRT (7.1). So KnownDLLs can be used to redirect modules! I was able to confirm Neither msvcr70 or msvcr71 cover all of the NT Msvcrt exports. msvcr70 is missing: _get_heap_handle msvcr71 is missing: _ctype but not void * bad_cast::`scalar deleting destructor'(unsigned int) void * bad_cast::`vector deleting destructor'(unsigned int) void * bad_typeid::`scalar deleting destructor'(unsigned int) void * bad_typeid::`vector deleting destructor'(unsigned int) void * exception::`scalar deleting destructor'(unsigned int) void * exception::`vector deleting destructor'(unsigned int) void * __non_rtti_object::`scalar deleting destructor'(unsigned int) void * __non_rtti_object::`vector deleting destructor'(unsigned int) Both 70 and 71 have ??0bad_cast... and so forth so I don't see how they differ. What tool did you use to interpret the encoded function names (and presumably find the above missing from 71)? If I make a patch to add a _get_heap_handle stub to msvcr70, we may have a nice solution. Otherwise we can try to get the real NT msvcrt itself working...!

I've copied the msvcr71.dll from SP3 into my system folder. I'm shutting SE down for the night, so I'll exit to DOS and make the switch with msvcrt.dll. In the morning I'll boot with the 'updated' msvcrt.dll and exercize a bunch of apps. If all goes well, I'll try modifying HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs to make the redirection (in the system folder). Perhaps this key can be used to make module subs on-the-fly without needing help from ImportPatcher. That would be great for redirecting delay-load modules. ImportPatcher.32 is now posted. I spent a couple of hours Saturday creating PEfinder, a tool that scans all files in a given folder and all sub-folders for valid PE headers. A stripped-down version of ImportPatcher, it filters on any DOS/PE/Optional/Directory field I specify (in the src) and outputs a log file with the filename and field value of the matches. (No UI or run-time options.) Compared to the dozen or so files I had been testing IP with, I was able to test with thousands of files in just a few minutes. And I found three problems: Empty files caused CreateFileMapping to fail (like dencorso reported) 16-bit apps without full DOS header lacked a valid e_lfanew field leading to GPF if random value pointed past EOF 64-bit modules are PE32+ with longer Optional header, moving Directory table back 16 bytes Fixes for these issues were ported back to ImportPatcher.32. I also added the bonus character for even-length string names as recently discussed.

Those using ImportPatcher on 2K, XP, or later might find some registry left-overs in: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\IniFileMapping The PrivateProfileString functions I use to manage .ini files don't use the registry in 9x, but supposedly do in NT. I plan to move away from the use of .ini files in the near future, so hopefully this problem for NT users will go away soon. -jumper

So msvcr70 and msvcr71 seems to be good replacements for msvcrt. Another way to do the substitution is to put a copy renamed to msvcrt directly in the folder the the app and its dll's. And with more backwards compatibility testing, perhaps msvcr71 can be used as a direct update for msvcrt in the <system> folder. (Oops...Did I just volunteer again )

Wow, great work loblo! Is KernelEx helping at all? And how did you determine what code to put into the stubs? GTK is a popular toolkit. Qt is another. If we can get them and .Net working, a lot of apps will start to work.

Joe, KernelEx v4.5 Beta 1 added a stub for HeapSetInformation, so you can leave that function substitution blank. As for msvcrt.dll, try subbing 71 instead of 90: I searched my HDD for "msvc*.dll" and came up with 36 hits (including dups). Refining the search to files containing text: "__uncaught_exception" reduced that count to 18 and these seven unique files: msvcr70.dll msvcr71.dll msvcr90.dll msvcp70.dll msvcp71.dll msvcp90.dll msvcm90.dll r = C Run-time (CRT) m = managed (.Net) p = C++ Run-time??? (Bonus points to first responder) All seven of these missing functions appear to have been introduced in MSVC++ 7.0: [msvcrt.dll] __uncaught_exception ___lc_handle_func ___lc_codepage_func ___mb_cur_max_func __pctype_func __iob_func __crtLCMapStringW __crtGetStringTypeW These 8 "functions" (variables probably) can be found in msvc 4 through 7.1, but were removed by 9: [msvcr9.dll] ??0exception@@QAE@ABQBD@Z= ?what@exception@@UBEPBDXZ= ??1exception@@UAE@XZ= mktime= ??0exception@@QAE@ABV0@@Z= ??0exception@@QAE@XZ= ??1bad_cast@@UAE@XZ= ??0bad_cast@@QAE@ABV0@@Z= I don't have "8" on my system...maybe someone else can check it for these exports. Good move--I forgot to count when I suggested it. Because table entries are word-aligned, there should actually be an extra byte available to even-lengthed strings (NULL terminator makes them odd). An additional byte or two can (usually) be stolen from the word-sized hint of the following hint-string pair. (At least one linker out there has a bug that does overlap entries half the time, making optimizing hints impossible without completely bulding the table!) Since "msvcrt.dll" is even, look for the next beta to support subbing up to: ((length&-2)+1) p.s. I hope everyone is having as much fun as I am. Thanks everyone!

Probably a delay-load import. IP doesn't handle those yet. Try Dependency Walker--it should find it and report if it is a standard or delay load type. Sounds like a wide-char mapi library. Try doing a preemptive module substitution with mapi32.dll or tmapi.dll; then look at what functions are reported missing.

ImportPatcher.31 is now posted. It has the fixes I promised dencorso. I've found enough discrepancies concerning function hints between the documentation, various guides, and what Dependency Walker reports that I've decided to disable the hints and binds features as a needless distraction at this time. The code is still in there, so it'll come back in time. The [Missing modules] section that only appeared if a module couldn't be found is now named [Module substitutions] and is always present. Substitutions for module and function names are now made before the dependency is checked, allowing for more creative patching options. There is still a constraint, however, that substituted names must not be longer than the name they replace. As for a more compelling new feature, I was able to alter the ending message box to provide a 'Retry' option. Now we can check the log file, edit the ini file, then click on 'Retry' to start another pass without having to reinvoke IP. (We can still click 'Cancel' to exit and then reinvoke later if we want.) The next step is now obvious: a dialog box interface with checkboxes for the Yes/No parameters and edit boxes for the text fields, and buttons that can be labelled beyond MessageBox contraints. No more hunting for the ini or log files! After that, I'll fully parse the headers and essentially relink much of the file if necessary to allow substitution of longer names and even additional modules. This will allow individual function calls to be redirected to stub libraries or even custom DLL's.

The previous version was dotNet4.0--even more recent. Because KernelEx won't always be up-to-date with the latest demands of new software, it would be nice to know if ImportPatcher can help fill the void. To that end, it would be great if you could test the dotNet4.0 version with the msvcrt->msvcr90 replacement I proposed. This might also really help out those who don't use KernelEx. TIA, jumper.

I'll try, but I don't have any OS later than SE to test on. Good error reporting like you provided here will be important. My WinMainCRTStartup function simply returned without calling exit or ExitProcess. This works in SE; apparently not in 2K+. I've added ExitProcess now. The Debug message is mine and indicates that CreateFileMapping (part of the file-mapping sequence of calls) failed. I have located and fixed a minor (SE didn't mind) error in one of the protection flags. I'll also add GetLastCallError support to the error reporting. Despite forcing CreateFileMapping to fail when in patch mode, I was unable to reproduce the error loop. I'm testing IP.31 builds now and much code has been cleaned up since IP.29. I'll trace the old code in my best simulator (sleep on it) in a few minutes.... ImportPatcher is currently designed to function recursively. A stack overflow is the expected result of a runaway loop. The slow speed is likely the result of DW managing a huge amount of text in the log window. A (hitherto) undocumented feature of ImportPatcher is that the text of all message boxes, log file entries, and any error messages are also passed to OutputDebugMessage(). Running IP in a debug environment such as DW allows viewing of these messages. If IP is looping endlessly (until the stack overflows) the DW log window should be filling will huge amounts of text.

Well, I found a version "7.0.6002.18005 (lh_sp2rtm.090410-1830)" on a Vista machine, dated 2009/4/11. However, although this version only reported "[Need patching? ... msvcrt.dll=Y (OS subsystem)" in ImportPatcher (with 'Walk dependencies=N'), after being patched for the OS subsystem, it looked like a descent into DLL dependency hell. All seven of those functions are supported in MSVCR90.dll in the package VC_R_9X.EXE at MDGX. If (anyone is) not running KernelEx, patch MSVCR90.dll with this function replacement: [KERNEL32.dll] GetSystemWindowsDirectoryW=GetWindowsDirectoryW Put MSVCR90.dll in the same folder as signtool.exe or in <windows> or <system>. Then add to signtoo#.ini: [Missing modules] msvcrt.dll=MSVCR90.dll msvcrt.dll=MSVCR9#.dll ;or this if you don't rename after patching This should fix the MSVCRT.DLL issues. If signtool has futher dependency problems, post the full .ini file this time (in a 'spoiler' box if large). * Note: ImportPatcher.29 and .30 syntax (may change in other versions) *

Great ideas! KernelEx is crucial to most of our wishes, so I'll add it as a separate category. It is already accumulating many wishes of its own. Java is one I had thought of but completely forgot. Printer/printing is another--hardware and software issue here. I'll review all discussion and update the Wish List tonight! -jumper

Since writing the COMCTL32.DLL example the other day, the patched COMCTL32.DLL has been running on my system with no problems. >It's quite amazing, really! Thanks, but it's really just an exercise in learning how to parse the various header structures in the Portable Executable file format. Documentation and guides are hard to find and incomplete, but I keep stumbling onto more of them each week! IP.30 is undergoing final testing and includes unbinding of broken links. On the drawing board for function substitution is redirection to another module: [uSER32.dll] _missing=KERNEL32.SetLastError and possibly module insertion: [uSER32.dll] _missing=stubs.T16 >Substituting another function or stub for '__uncaught_exception' might {not} be acceptable to the calling app. If not, ... I could have written "might or might not be", but chose to simplify and wrote "might be". When dealing in fuzzy logic, "not" sometimes becomes optional or even meaningless! >>Also, a *#.* copy of every file that is walked is created, including system DLLs (only useful if you're trying to fragment your HDD!). >Does this relate to the "Link to copies=Y/N" option in the INI file? Would this also require "Walk dependencies=Y"? No and Yes! IP.27 would open for R/W a copy of every file it analyzed (whether walking dep's or not) so that it could analyze and patch in one pass. Unfortunately, it didn't delete unneeded copies. Copying every file also made it slow (and loud). "Link to copies=Y/N" determines whether an app or dll references the original or patched dependency. Naming this option to something understandable has been problematic! Y = patch reference to refer to patched copy of dependency needed if dependencies are patched and the (patched) app is to be directly executable N = continue to refer to original needed if patched files are intended to be installed over originals For patched system files an installer is needed (or the file must be copies by hand in DOS). Creation of an .inf will also be tied to this option some time soon! >>>1. Is there a way to pass command line parameters to "signtool.exe" when using the Import Patcher? >>IP.28 reads parameters from an .ini file that can be edited between passes. >I think that ImportPatcher.27 gave the impression that it would load and execute a file, while satisfying missing dependencies. Since that doesn't seem to be the case, my earlier question was null and void. I misread the question about command line parameters, but now understand. Executing the patched app is a possible future feature and parameter passing would be an important design issue. Perhaps a "[Parameters] App command line parameters=" line in the .ini?

That's great news! Now folks using KernelEx will soon have the option to start creating files larger than 4GB... ...errr, does this mean I have to start adding large file support to all my programs? ...

I made better progress today trying to build some of the sub-projects with VC5 than I previously did several months ago. ApiLog and Core are both close to building after numerous small mods to paths, pragmas, link options, pointer casts, etc. Use of ApiHook in a special build is not what I had in mind, but might work. I'm looking for a method that will allow non-programmers to add stubs to the standard version of KernelEx by modifying an .ini file. For instance, Core.ini contains the section: [WIN98.names] KERNEL32.GetVersion=kexbases.1 KERNEL32.GetVersionExA=kexbases.2 KERNEL32.GetVersionExW=kexbases.2 I'd like to be able to add: MYDLL.SomeFunction=stubs.5 to allow SomeFunction in MYDLL.dll to be replaced with function ordinal 5 in Windows\KernelEx\stubs.dll (stubs.dll is an actual dll built ten days ago containing a family of stubs along the lines of rloew's suggestions). Well, that's it for now. I'll resume trying to get more of KernelEx to build tomorrow.