Jump to content

ImportPatcher.41 - Find and fix dependency problems


jumper

Recommended Posts

:rolleyes: ImportPatcher.31 is now posted.

It has the fixes I promised dencorso.

I've found enough discrepancies concerning function hints between the documentation, various guides, and what Dependency Walker reports that I've decided to disable the hints and binds features as a needless distraction at this time. The code is still in there, so it'll come back in time.

The [Missing modules] section that only appeared if a module couldn't be found is now named [Module substitutions] and is always present. Substitutions for module and function names are now made before the dependency is checked, allowing for more creative patching options. There is still a constraint, however, that substituted names must not be longer than the name they replace.

As for a more compelling new feature, I was able to alter the ending message box to provide a 'Retry' option. Now we can check the log file, edit the ini file, then click on 'Retry' to start another pass without having to reinvoke IP. (We can still click 'Cancel' to exit and then reinvoke later if we want.)

The next step is now obvious: a dialog box interface with checkboxes for the Yes/No parameters and edit boxes for the text fields, and buttons that can be labelled beyond MessageBox contraints. No more hunting for the ini or log files! :D

After that, I'll fully parse the headers and essentially relink much of the file if necessary to allow substitution of longer names and even additional modules. This will allow individual function calls to be redirected to stub libraries or even custom DLL's.

Link to comment
Share on other sites


The future of Import Patcher looks very promising! :thumbup

Anyway, I recently encountered what appears to be a bug/quirk with IP.29. I wanted to try the RW utility here : http://rweverything.myweb.hinet.net/

What I found was that, for versions 1.4.5 and newer of RW, there is a missing dependency on 'dwmapi.dll', a Vista DLL. However, IP didn't detect this missing dependency.

Joe.

Link to comment
Share on other sites

...for versions 1.4.5 and newer of RW, there is a missing dependency on 'dwmapi.dll', a Vista DLL. However, IP didn't detect this missing dependency.

Probably a delay-load import. IP doesn't handle those yet. Try Dependency Walker--it should find it and report if it is a standard or delay load type.

Sounds like a wide-char mapi library. Try doing a preemptive module substitution with mapi32.dll or tmapi.dll; then look at what functions are reported missing.

Link to comment
Share on other sites

The previous version was dotNet4.0--even more recent. Because KernelEx won't always be up-to-date with the latest demands of new software, it would be nice to know if ImportPatcher can help fill the void. To that end, it would be great if you could test the dotNet4.0 version with the msvcrt->msvcr90 replacement I proposed. This might also really help out those who don't use KernelEx.

TIA, jumper.

Sure, but I won't get a chance to try this until Monday.

Since I do use/rely on KernelEx, for the purposes of this experiment, I presume the following will be sufficient :

* In "signtoo#.ini", I'll add :


[Missing modules]
msvcrt.dll=MSVCR90.dll

Joe.

Well, my first attempt was thwarted because the name "MSVCR90.dll" was longer than the original "msvcrt.dll". So I copied "MSVCR90.dll" locally as "MSVCR9.dll" and did the module substitution using that name.

However, I can't get this to work. Here are the generated INI files in chronological sequence (the initial one generated by IP, my modified version, the subsequent one generated by IP) :


[ImportPatcher.31]
;Edit parameters and replacement strings, then Retry or run again to patch. <=

[Parameters]
Walk dependencies=N
Link to copies=N
Target OS=4.10

[Module substitutions]

[KERNEL32.dll]
HeapSetInformation=

[msvcrt.dll]
__uncaught_exception=
___lc_handle_func=
___lc_codepage_func=
___mb_cur_max_func=
__pctype_func=
__iob_func=
__crtLCMapStringW=
__crtGetStringTypeW=

[Patch list]
E:\Winstall\Development\Vista\710_buildtools_signtool\SignTool.exe=OS subsystem, function names


[ImportPatcher.31]
;Edit parameters and replacement strings, then Retry or run again to patch. <=

[Parameters]
Walk dependencies=N
Link to copies=N
Target OS=4.10

[Module substitutions]
msvcrt.dll=msvcr9.dll

[KERNEL32.dll]
HeapSetInformation=HeapSetInformation

[Patch list]
E:\Winstall\Development\Vista\710_buildtools_signtool\SignTool.exe=OS subsystem, function names


[ImportPatcher.31]
;Edit parameters and replacement strings, then Retry or run again to patch. <=

[Parameters]
Walk dependencies=N
Link to copies=N
Target OS=4.10

[Module substitutions]
msvcrt.dll=msvcr9.dll

[KERNEL32.dll]
HeapSetInformation=HeapSetInformation * not found

[Patch list]
E:\Winstall\Development\Vista\710_buildtools_signtool\SignTool.exe=OS subsystem, function names, submodule names
msvcr9.dll=OS subsystem

[msvcr9.dll]
??0exception@@QAE@ABQBD@Z=
?what@exception@@UBEPBDXZ=
??1exception@@UAE@XZ=
mktime=
??0exception@@QAE@ABV0@@Z=
??0exception@@QAE@XZ=
??1bad_cast@@UAE@XZ=
??0bad_cast@@QAE@ABV0@@Z=

...for versions 1.4.5 and newer of RW, there is a missing dependency on 'dwmapi.dll', a Vista DLL. However, IP didn't detect this missing dependency.

Probably a delay-load import. IP doesn't handle those yet. Try Dependency Walker--it should find it and report if it is a standard or delay load type.

Sounds like a wide-char mapi library. Try doing a preemptive module substitution with mapi32.dll or tmapi.dll; then look at what functions are reported missing.

Well, the good news is that substituting "mapi32.dll" for "dwmapi.dll" fixes that particular dependency.

Unfortunately, after that there's a dependency on "WindowsCodecs.dll" that's more intractable. I can get this DLL online or from the dotNet 3 package, however, this has many missing function dependencies, from several other DLL's.

Joe.

Link to comment
Share on other sites

Give it any dll that you rename to windowscodecs.dll. ;) For as long as the application that requires this dll doesn't actually need it for loading and saving image files, this should be OK (succesfully tested on RW build 1.4.9.7) B)

And btw I think I have just nicely solved the problem of the missing dnsapi.dll required by recent versions of libgio-2.0-0.dll used by GTK apps by making a dummy dnsapi.dll from the sample dll project in VC6 plus a bit of hexing. More about that later after testing a bit more, only (succesfully) tried it on a couple of unofficial Gimp 2.7.4 so far. B)

Link to comment
Share on other sites

By doing a substitution of msvcrt.dll for msvcr70.dll (renamed and substituted as msvcr7.dll), msvcr90 or 80 wouldn't work, on two dlls and using my dummy dnsapi.dll, I could get the latest svn build of Inkscape (uploaded just 15 hours ago at the time of this post) to run perfectly. :thumbup

https://skydrive.live.com/?cid=09706D11303FA52A&id=9706D11303FA52A%21128

Attached are the blank stub.dll with 4 blank export functions and the dnsapi.dll dummy made from it for GTK apps. :hello:

stub.7z

Edited by loblo
Link to comment
Share on other sites

Joe,

KernelEx v4.5 Beta 1 added a stub for HeapSetInformation, so you can leave that function substitution blank.

As for msvcrt.dll, try subbing 71 instead of 90:

I searched my HDD for "msvc*.dll" and came up with 36 hits (including dups). Refining the search to files containing text: "__uncaught_exception" reduced that count to 18 and these seven unique files:


msvcr70.dll
msvcr71.dll
msvcr90.dll
msvcp70.dll
msvcp71.dll
msvcp90.dll
msvcm90.dll

r = C Run-time (CRT)
m = managed (.Net)
p = C++ Run-time??? (Bonus points to first responder)

All seven of these missing functions appear to have been introduced in MSVC++ 7.0:


[msvcrt.dll]
__uncaught_exception
___lc_handle_func
___lc_codepage_func
___mb_cur_max_func
__pctype_func
__iob_func
__crtLCMapStringW
__crtGetStringTypeW

These 8 "functions" (variables probably) can be found in msvc 4 through 7.1, but were removed by 9:


[msvcr9.dll]
??0exception@@QAE@ABQBD@Z=
?what@exception@@UBEPBDXZ=
??1exception@@UAE@XZ=
mktime=
??0exception@@QAE@ABV0@@Z=
??0exception@@QAE@XZ=
??1bad_cast@@UAE@XZ=
??0bad_cast@@QAE@ABV0@@Z=

I don't have "8" on my system...maybe someone else can check it for these exports.

Well, my first attempt was thwarted because the name "MSVCR90.dll" was longer than the original "msvcrt.dll". So I copied "MSVCR90.dll" locally as "MSVCR9.dll" and did the module substitution using that name.

Good move--I forgot to count when I suggested it. Because table entries are word-aligned, there should actually be an extra byte available to even-lengthed strings (NULL terminator makes them odd). An additional byte or two can (usually) be stolen from the word-sized hint of the following hint-string pair. (At least one linker out there has a bug that does overlap entries half the time, making optimizing hints impossible without completely bulding the table!)

Since "msvcrt.dll" is even, look for the next beta to support subbing up to: ((length&-2)+1)

p.s. I hope everyone is having as much fun as I am. :w00t: Thanks everyone! :thumbup

Edited by jumper
Link to comment
Share on other sites

By doing a substitution of msvcrt.dll for msvcr70.dll (renamed and substituted as msvcr7.dll), msvcr90 or 80 wouldn't work, on two dlls and using my dummy dnsapi.dll, I could get the latest svn build of Inkscape (uploaded just 15 hours ago at the time of this post) to run perfectly. :thumbup

https://skydrive.live.com/?cid=09706D11303FA52A&id=9706D11303FA52A%21128

Attached are the blank stub.dll with 4 blank export functions and the dnsapi.dll dummy made from it for GTK apps. :hello:

Thanks, loblo. I'm sure that stub will come in handy!

So, you substituted msvcrt.dll with msvrt70.dll (renamed as msvrt7.dll), because the same trick didn't work with msvrt80 or msvcr90. Then you also created a dummy dnsapi.dll. Gotcha.

As for msvcrt.dll, try subbing 71 instead of 90:

I searched my HDD for "msvc*.dll" and came up with 36 hits (including dups). Refining the search to files containing text: "__uncaught_exception" reduced that count to 18 and these seven unique files:


msvcr70.dll
msvcr71.dll
msvcr90.dll
msvcp70.dll
msvcp71.dll
msvcp90.dll
msvcm90.dll

r = C Run-time (CRT)
m = managed (.Net)
p = C++ Run-time??? (Bonus points to first responder)

All seven of these missing functions appear to have been introduced in MSVC++ 7.0:


[msvcrt.dll]
__uncaught_exception
___lc_handle_func
___lc_codepage_func
___mb_cur_max_func
__pctype_func
__iob_func
__crtLCMapStringW
__crtGetStringTypeW

These 8 "functions" (variables probably) can be found in msvc 4 through 7.1, but were removed by 9:


[msvcr9.dll]
??0exception@@QAE@ABQBD@Z=
?what@exception@@UBEPBDXZ=
??1exception@@UAE@XZ=
mktime=
??0exception@@QAE@ABV0@@Z=
??0exception@@QAE@XZ=
??1bad_cast@@UAE@XZ=
??0bad_cast@@QAE@ABV0@@Z=

I don't have "8" on my system...maybe someone else can check it for these exports.

Good call. I tried msvcr71.dll, renamed as msvcr7.dll, and it worked! :)

As for msvcr80.dll, that doesn't have those strange-looking identifiers. (Actually, when I initially saw them, I thought they were just garbage.)

Joe.

Link to comment
Share on other sites

By doing a substitution of msvcrt.dll for msvcr70.dll (renamed and substituted as msvcr7.dll), msvcr90 or 80 wouldn't work, on two dlls and using my dummy dnsapi.dll, I could get the latest svn build of Inkscape (uploaded just 15 hours ago at the time of this post) to run perfectly. :thumbup

https://skydrive.live.com/?cid=09706D11303FA52A&id=9706D11303FA52A%21128

Attached are the blank stub.dll with 4 blank export functions and the dnsapi.dll dummy made from it for GTK apps. :hello:

Wow, great work loblo!

Is KernelEx helping at all? And how did you determine what code to put into the stubs?

GTK is a popular toolkit. Qt is another. If we can get them and .Net working, a lot of apps will start to work.

Link to comment
Share on other sites

By doing a substitution of msvcrt.dll for msvcr70.dll (renamed and substituted as msvcr7.dll), msvcr90 or 80 wouldn't work, on two dlls and using my dummy dnsapi.dll, I could get the latest svn build of Inkscape (uploaded just 15 hours ago at the time of this post) to run perfectly.

Good call. I tried msvcr71.dll, renamed as msvcr7.dll, and it worked! :)

So msvcr70 and msvcr71 seems to be good replacements for msvcrt. Another way to do the substitution is to put a copy renamed to msvcrt directly in the folder the the app and its dll's. And with more backwards compatibility testing, perhaps msvcr71 can be used as a direct update for msvcrt in the <system> folder. (Oops...Did I just volunteer again :blink:)

Link to comment
Share on other sites

Those using ImportPatcher on 2K, XP, or later might find some registry left-overs in:


  • HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\IniFileMapping

The PrivateProfileString functions I use to manage .ini files don't use the registry in 9x, but supposedly do in NT.

I plan to move away from the use of .ini files in the near future, so hopefully this problem for NT users will go away soon.

-jumper

Edited by jumper
Link to comment
Share on other sites

By doing a substitution of msvcrt.dll for msvcr70.dll (renamed and substituted as msvcr7.dll), msvcr90 or 80 wouldn't work, on two dlls and using my dummy dnsapi.dll, I could get the latest svn build of Inkscape (uploaded just 15 hours ago at the time of this post) to run perfectly. :thumbup

https://skydrive.live.com/?cid=09706D11303FA52A&id=9706D11303FA52A%21128

Attached are the blank stub.dll with 4 blank export functions and the dnsapi.dll dummy made from it for GTK apps. :hello:

Wow, great work loblo!

Is KernelEx helping at all? And how did you determine what code to put into the stubs?

GTK is a popular toolkit. Qt is another. If we can get them and .Net working, a lot of apps will start to work.

Thanks Jumper.

Of course KernelEx in Windows 2000 compatibility mode is mandatory for running GTK apps.

I did not code anything, I just fired VC6 which I have since years but have never used as I am not a coder, I haven't got a clue about coding, and choose to create a new dll project and I selected the one from the wizard that offered to create some empty skeleton export functions and then I just compiled that. Once this was done I opened the dll with dependency walker and saw there was 4 export function with super weird and rather long names which I hexed to stub1, 2 3 and 4 and this is is the stub.dll I uploaded. After that I just renamed it to dnsapi.dll and hexed two of the export functions to the name of the two functions libgio-2.0-0.dll wants to import from dnsapi and that's all I did and it works. :w00t:

It was possible and in some case still is possible to replace libgio-2.0-0.dll with an older version from around 2009 which doesn't rely on dnsapi.dll but it is becoming increasingly incompatible with other GTK runtime files it is interdependent with, forcing to replace more dlls with older ones to satisfy it to the point it was hardly workable anymore and in some cases not at all so I am glad I found that little hack as it should give a bit of headroom, along with your msvcrx for mscvrt substitution, for running newer GTK apps.

Link to comment
Share on other sites

By doing a substitution of msvcrt.dll for msvcr70.dll (renamed and substituted as msvcr7.dll), msvcr90 or 80 wouldn't work, on two dlls and using my dummy dnsapi.dll, I could get the latest svn build of Inkscape (uploaded just 15 hours ago at the time of this post) to run perfectly.

Good call. I tried msvcr71.dll, renamed as msvcr7.dll, and it worked! :)

So msvcr70 and msvcr71 seems to be good replacements for msvcrt. Another way to do the substitution is to put a copy renamed to msvcrt directly in the folder the the app and its dll's. And with more backwards compatibility testing, perhaps msvcr71 can be used as a direct update for msvcrt in the <system> folder. (Oops...Did I just volunteer again :blink:)

If I put msvcrt.dll with an application folder it doesn't get used by it, it always uses the one in the system folder. It seems that's because it is listed under the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs registry key. I think this key forces windows to use the listed dlls only from the location that is specified for them and which is the system dir if no full path is specified. I am not sure it is a good idea to override that by deleting some of those entries as to run core system files or quasi core system files such as msvcrt from other locations as when I experimented a bit with that I recall it led to weird problems with standard win9x programs refusing to start. (Well it was with files from Reactos and Wine I tried so maybe it might work with less or no problems with Microsoft files).

Edited by loblo
Link to comment
Share on other sites

If I put msvcrt.dll with an application folder it doesn't get used by it, it always uses the one in the system folder. It seems that's because it is listed under the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs registry key. I think this key forces windows to use the listed dlls only from the location that is specified for them and which is the system dir if no full path is specified. I am not sure it is a good idea to override that by deleting some of those entries as to run core system files or quasi core system files such as msvcrt from other locations as when I experimented a bit with that I recall it led to weird problems with standard win9x programs refusing to start. (Well it was with files from Reactos and Wine I tried so maybe it might work with less or no problems with Microsoft files).

I've copied the msvcr71.dll from SP3 into my system folder. I'm shutting SE down for the night, so I'll exit to DOS and make the switch with msvcrt.dll.

In the morning I'll boot with the 'updated' msvcrt.dll and exercize a bunch of apps. If all goes well, I'll try modifying HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs to make the redirection (in the system folder). Perhaps this key can be used to make module subs on-the-fly without needing help from ImportPatcher. That would be great for redirecting delay-load modules.


ImportPatcher.32 is now posted.

I spent a couple of hours Saturday creating PEfinder, a tool that scans all files in a given folder and all sub-folders for valid PE headers. A stripped-down version of ImportPatcher, it filters on any DOS/PE/Optional/Directory field I specify (in the src) and outputs a log file with the filename and field value of the matches. (No UI or run-time options.)

Compared to the dozen or so files I had been testing IP with, I was able to test with thousands of files in just a few minutes. And I found three problems:

  1. Empty files caused CreateFileMapping to fail (like dencorso reported)
  2. 16-bit apps without full DOS header lacked a valid e_lfanew field leading to GPF if random value pointed past EOF
  3. 64-bit modules are PE32+ with longer Optional header, moving Directory table back 16 bytes

Fixes for these issues were ported back to ImportPatcher.32. I also added the bonus character for even-length string names as recently discussed.

Link to comment
Share on other sites

By substituting msvcrt for msvcr70 in the Gimp plugin G'Mic I could get rid of the only systematic crash that occurs with it here when using the apply button which is meant to apply the filter on the main image and leave the plugin window open after that. With msvcrt it would crash after application of the filter and now it doesn't anymore which is very cool.

I also tried to make such a substitution for the plugin mathmap which crashes instantly on clicking on some of its scripts/presets entries but in this case it only made matters worst with systematic instant crash in kexbases on lauching the plugin but since it crashes in kexbases it might just well be a KernelEx bug.

Also I think that perrhaps msvcr70 is most compatible with msvcrt as in one instance of the substitutions I made, msvcr71 was missing a function which msvcr70 had.

Edited by loblo
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...