Jump to content

ImportPatcher.41 - Find and fix dependency problems


jumper

Recommended Posts


You will need to cover more than 9 parameters. CreateFontA uses 14 parameters. There probably are larger ones elsewhere.

Fortunately CreateFontA has been in GDI32 since Win32s so we don't need a stub for it. We can cross other bridges when we come to them.

i knew it didn't need a stub. It was an example of what is out there. I had redirected USER32.DLL and GDI.DLL through a logging DLL I was experimenting with, so I had a list of the APIs and their parameter counts. It was the largest.

Do you know of any recent comprehensive lists of functions similar to the old WIN32API.CSV? I may need to bite the bullet and download a recent platform SDK, then look at the header files.

I don't have a list. I wrote a tool that extracts the APIs from the header files and helps build source code for the loggers I mentioned above.

Link to comment
Share on other sites

I don't have a list. I wrote a tool that extracts the APIs from the header files and helps build source code for the loggers I mentioned above.

Searching for "windows api parameter count" lead me to these header resources:

Expanding WIN32A.ZIP, I found WIN32P.INC and the following:

  • _LocalEnroll() requires 23 arguments.
  • _LocalEnrollNoDS() requires 23 arguments.

These seem to be from CRYPTUI.dll

I've used DOS FIND to extract the lines containing 'requires' and then the DevStudio '97 text replace function to create an .ini file with 17986 18300 API parameter counts:

[ParameterCounts]
CreateSecurityPage=1
EditSecurity=2
IID_ISecurityInformation=1
ADsBuildEnumerator=2
...

Edited by jumper
Link to comment
Share on other sites

By doing a substitution of msvcrt.dll for msvcr70.dll (renamed and substituted as msvcr7.dll), msvcr90 or 80 wouldn't work, on two dlls and using my dummy dnsapi.dll, I could get the latest svn build of Inkscape (uploaded just 15 hours ago at the time of this post) to run perfectly. :thumbup

https://skydrive.live.com/?cid=09706D11303FA52A&id=9706D11303FA52A%21128

Attached are the blank stub.dll with 4 blank export functions and the dnsapi.dll dummy made from it for GTK apps. :hello:

As it turns out, the loader will append '.dll' to an import DLL name as needed, so 'msvrt.dll' can be replaced with 'msvcr70' without needing to rename it to 'msvcr7.dll'.

Jumper, as it's a bit complex, please post any working example you might have for use of ipstub.dll.

...

Ignore the mention of ordinals for now. IP won't support replacement-by-ordinal until the next version.

If you modify your Inkscape patch to use IPstub.dll, it should work. Otherwise try patching a copy of IP itself as I showed.

IPstub.dll works the same as your dnsapi.dll/stub.dll, but with a wider variety of stubs and diagnostics functions to choose from. Function names are all two characters to ensure they're not too long:


p1 p2 p3 p4 p5 p6 p7 p8 p9
f0 f1 f2 f3 f4 f5 f6 f7 f8 f9
o0 o1 o2 o3 o4 o5 o6 o7 o8 o9
t0 t1 t2 t3 t4 t5 t6 t7 t8 t9
yn op bp

If the functions aren't being called, any function will work (just like in your dnsapi.dll). Otherwise choose a stub with the desired return code and parameter count. (A quick search of MSDN should yield these.)

If you run IP.34 on Inkscape you should be able to make this replacement in the .ini file:


[DLL substitutions]
dnsapi.dll=IPstub.dll

Then [ Retry ] and:


[IPstub.dll]
DnsQuery_A=f6
DnsRecordListFree=f2

[ Retry ] again to patch. IPstub.dll can be in the app folder, <system>, <windows>, or anywhere in the normal PATH.

...

And btw I prefer that importpatcher does not link to copies of dependencies anymore.

:wub: Just the type of feedback I love to hear! :wub:

It was always optional (editable in .ini), but defaulted to 'Y' if walking and 'N' if not.

Link to comment
Share on other sites

Meanwhile, I'll continue to investigate the twelve trailing spaces that don't seem to jive with the rest of the clues....

The trailing spaces are an indentation for a missing log message. This only happens when wsprintf encounters an error (usually an access violation in one of the parameters) and emits no text.

I was able to reproduce this condition in a way that resembled divad's log file by intentionally using an invalid pointer for the delay-import ILT address. I then modified my PEfinder test app to search my local drives for apps with invalid delay-import ILT addresses. I found five. All had been UPX'ed.

UPX compression abbreviates the import tables and corrupts the delay-import table (if any). This works because the system loader doesn't check the delay-import table and the UPX decompressor restores it for normal use later. Tools like Dependency Walker can only report that the address is invalid. Other compressors may also corrupt the delay-import table, but I found no such examples on my local drives.

PEfinder also uncovered one app that stored the parallel ILT and IAT tables in different sections. In this case it wasn't a problem, but theoretically it could be. So I rewrote the rva-to-pointer routine to lookup every address in the section tables without making any assuptions, no matter how reasonable they might seem to be!

Well, PEfinder only opens one file at a time, while ImportPatcher opens two (or more if walking) at a time. So the new rva-to-pointer routine was difficult to port and required lots of extra support code and modifications. However, IP.35 now seems to be working and will be released as soon I finish regression testing it.

Link to comment
Share on other sites

Do you know of any recent comprehensive lists of functions similar to the old WIN32API.CSV? I may need to bite the bullet and download a recent platform SDK, then look at the header files.

I don't have a list. I wrote a tool that extracts the APIs from the header files and helps build source code for the loggers I mentioned above.

Jumper, is this something that could be implemented (without too much pain) in a future version of IP?

Joe.

Link to comment
Share on other sites

Jumper, is this something that could be implemented (without too much pain) in a future version of IP?

Here is the aforementioned INI file with 18301 API parameters counts: APIParameterCounts.zip

Last week I did try accessing it from IP using GetPrivateProfileString and it worked great--the first 64KB that is. The other 406KB - 64KB can't be accessed that way.

I converted it to a REG file, hoping to be able to access it from the transient portion of the registry with RegQueryValue, but REG files can only add to the on-disk keys and that also takes several minutes.

The current plan is to do a quick 27-bit hash or crc on the function names to reduce the data size (27+5 for the [0..23] count = 32). Or I might just split the data over seven INI files. :sneaky:

Link to comment
Share on other sites

Jumper, is this something that could be implemented (without too much pain) in a future version of IP?

Here is the aforementioned INI file with 18301 API parameters counts: APIParameterCounts.zip

Last week I did try accessing it from IP using GetPrivateProfileString and it worked great--the first 64KB that is. The other 406KB - 64KB can't be accessed that way.

I converted it to a REG file, hoping to be able to access it from the transient portion of the registry with RegQueryValue, but REG files can only add to the on-disk keys and that also takes several minutes.

The current plan is to do a quick 27-bit hash or crc on the function names to reduce the data size (27+5 for the [0..23] count = 32). Or I might just split the data over seven INI files. :sneaky:

If I understand you correctly, the 27-bit hash will shrink the INI file down to 73K at best, so that's not going to do the trick (by itself). However, seven or more INI files would work, and be easier to edit :

NtQueryInformationProcess=5 (from 'ntdll.dll') --- now it's 18302 entries!

Joe.

Link to comment
Share on other sites

If I understand you correctly, the 27-bit hash will shrink the INI file down to 73K at best, so that's not going to do the trick (by itself).

I would open the file and map it as an array of dwords (using an existing function). If the data was presorted, a simple binary search would quickly find the count.

Otherwise you're right, 73KB would still be too big for the PrivateProflle functions:

Each line in an INI file would need one byte for the '=', one byte for the count, and one byte for the EOL marker. That leaves less than one byte for the name/hash/crc string!

We might be able to get to 24 bits if we can:

  1. get below 16k functions by removing those with the most common count (1 or 2 maybe?); that count would become the default for functions not found
  2. group functions into sections and reclaim the '=' and <count> bytes.

However, seven or more INI files would work, and be easier to edit :

Early versions of ImportPatcher could batch process multiple files. When I added INI support, I found SE wasn't letting me access multiple PrivateProfile INI files. Once I picked one, I had to stick with it. So I dropped batch processing. Unless I can figure out how to easily work around this issue, even a single INI (in addition to the main #.ini) might not be an option. :(

NtQueryInformationProcess=5 (from 'ntdll.dll') --- now it's 18302 entries!

I don't think NtQueryInformationProcess can be static linked to--no import library for it. Though for completeness, a good idea to add it. Thanks. :thumbup

Link to comment
Share on other sites

If I understand you correctly, the 27-bit hash will shrink the INI file down to 73K at best, so that's not going to do the trick (by itself).

I would open the file and map it as an array of dwords (using an existing function). If the data was presorted, a simple binary search would quickly find the count.

Otherwise you're right, 73KB would still be too big for the PrivateProflle functions:

Each line in an INI file would need one byte for the '=', one byte for the count, and one byte for the EOL marker. That leaves less than one byte for the name/hash/crc string!

We might be able to get to 24 bits if we can:

  1. get below 16k functions by removing those with the most common count (1 or 2 maybe?); that count would become the default for functions not found
  2. group functions into sections and reclaim the '=' and <count> bytes.

However, seven or more INI files would work, and be easier to edit :

Early versions of ImportPatcher could batch process multiple files. When I added INI support, I found SE wasn't letting me access multiple PrivateProfile INI files. Once I picked one, I had to stick with it. So I dropped batch processing. Unless I can figure out how to easily work around this issue, even a single INI (in addition to the main #.ini) might not be an option. :(

Perhaps this is the point at which this potential feature exceeds the pain threshold? Since we have a nice big list for reference, we can look this up ourselves anyway, no big deal.

The only other idea I've had, is that for IP's purposes, only those functions that don't exist in W9X are needed. So we can have a full list for reference, and an abridged list used by IP.

NtQueryInformationProcess=5 (from 'ntdll.dll') --- now it's 18302 entries!

I don't think NtQueryInformationProcess can be static linked to--no import library for it. Though for completeness, a good idea to add it. Thanks. :thumbup

OK, now I'm confused. Here's the INI file for 'msi.dll' version 3.0.3790.2180 :

[importPatcher.34]

;Edit parameters and replacement strings, then Retry or run again to patch. <=

[Parameters]

Walk dependencies=Y

Link to copies=N

Unbind broken bindings=N

Target OS=4.10

[DLL substitutions]

USERENV.dll=

[ntdll.dll]

NtQueryInformationProcess=

[ADVAPI32.dll]

ConvertSidToStringSidW=

[KERNEL32.dll]

GetFileSizeEx=

[Patch list]

msi.dll=DLLs, Functions, Unbind

There's obviously something here I'm not quite understanding.

Joe.

Link to comment
Share on other sites

MS intended NtQueryInformationProcess to be an internal function for OS use only. Their tools don't let us static link to it, but they do.

NtQueryInformationProcess provides process and thread details that cannot all be obtained through other APIs. As NtQueryInformationProcess is not available on 9x (and can't be simulated) any DLLs (or apps) that call it should be avoided! :ph34r:

In your case it appears to be the version of ntdll.dll you are using that is the one to be avoided. If you really must use this version for a given app, unregister it from "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SessionManager\KnownDLLs" and put a copy of the dangerous version directly into the folder of any app that really needs it. The version in <system> should be a 9x-safe version!

Also, KernelEx comes with a version of USERENV.dll; try copying it from <windir>\KernelEx to <system> to resolve that missing dependency issue.

Link to comment
Share on other sites

KernelEx's got its own KnownDlls key at HKEY_LOCAL_MACHINE\SOFTWARE\KernelEx\KnownDLLs. It works the same way the MS one does and Userenv.dll should get loaded from there if the executable that needs it has KernelEx enabled.

In previous versions KernelEx used the MS KnownDlls key for the files now listed under its own key btw.

Link to comment
Share on other sites

jumper, loblo,

Thanks for the input, I do need to understand KernelEx in more detail when looking into practical use of IP.

As regards 'msi.dll', I was just checking to see if this could be adapted to W9X, for the purpose of processing the main MSI file from LibreOffice 3.4.5. However, from what I've learnt from Joe of JSWare, the problem may be more fundamental than 'msi.dll'. Apparently MSI files are structured like a file system, known as Microsoft Compound Document File Format and, unlike every other MSI file I've got lying around, the LibreOffice one uses 4K sectors instead of 512 byte sectors.

Now as far as the NtQueryInformationProcess function used by the v3 'msi.dll', frankly this doesn't sound like a legitimate need of 'msi.dll', especially as v2 doesn't need such a thing. Hence, a dummy function may be sufficient, IMHO. But it's all too hard at the moment. I'll have to look at some way to convert 4K-sector MSI files into 512-byte sectors, if/when time permits.

Joe.

Link to comment
Share on other sites

  • 1 month later...

Next experiment ...

Well, I've just tried to apply "ImportPatcher.34" with "IPStub.dll" to the Altium Designer Viewer : http://downloads.altium.com/altiumdesigner/AltiumDesignerViewerBuild9.3.0.19153.zip

After downloading and extracting the ZIP file, it is necessary to edit 'Setup\Setup.msi' in Orca and delete the "NOT Version9X" row in "LaunchCondition", then "Save" it (avoid using "Save As").

Running 'Setup.exe' stalls during the "Deleting backup files" phase. However, running it again and selecting "Repair" is successful.

Now that Altium Viewer is installed, it is necessary to patch 'dxp.exe' for two dependencies from 'Netapi32.dll'. Since nothing else is used from the W9X version of 'Netapi32.dll', we substitute the DLL with 'IPStub.dll' and select the 'o1' and 'o3' functions, per the following 'dx#.ini' file :


[ImportPatcher.34]
;Edit parameters and replacement strings, then Retry or run again to patch. <=

[Parameters]
Walk dependencies=N
Link to copies=N
Unbind broken bindings=N
Target OS=4.10

[DLL substitutions]
Netapi32.dll=IPStub.dll

[user32.dll]
UpdateLayeredWindow=

[IPStub.dll]
NetApiBufferFree=o1
NetWkstaGetInfo=o3

[Patch list]
dxp.exe=DLLs, Functions

[Need patching? (do not edit)]
C:\Program Files\Altium Designer S09 Viewer\dxp.exe=Y (function name)

So far, so good. With the additional assistance of KernelEx 4.5.2 (default setting is fine), we are able to launch Altium Viewer.

Unfortunately however, for me, it crashes :

DXP caused an invalid page fault in

module KERNEL32.DLL at 0187:bff9e0b7.

Registers:

EAX=00000000 CS=0187 EIP=bff9e0b7 EFLGS=00000a83

EBX=81db5144 SS=018f ESP=00ccf910 EBP=00000000

ECX=0000018f DS=018f ESI=00000001 FS=276f

EDX=00ccfab0 ES=018f EDI=bffce060 GS=0000

Bytes at CS:EIP:

cc a1 e0 dc fc bf 8b 00 66 64 f7 05 1c 00 00 00

Stack dump:

08c10000 81db5188 81deba2c 81deb998 81deb9ac e242f970 00000000 8348200c 83528e30 bff7a3bc 00ccf978 00057f90 8352902c 00000048 bff7a3a0 83482000

DXP caused an invalid page fault in

module KERNEL32.DLL at 0187:bff9e0b7.

Registers:

EAX=00000000 CS=0187 EIP=bff9e0b7 EFLGS=00000a83

EBX=81db5144 SS=018f ESP=00ccf8b0 EBP=00000000

ECX=0000018f DS=018f ESI=00000001 FS=276f

EDX=00ccfa50 ES=018f EDI=bffce060 GS=0000

Bytes at CS:EIP:

cc a1 e0 dc fc bf 8b 00 66 64 f7 05 1c 00 00 00

Stack dump:

08c10000 81db5188 81deba60 81deb9cc 81deb9e0 e242f970 00000000 8348200c 83528e30 bff7a3bc 00ccf918 00057f90 8352902c 00000048 bff7a3a0 83482000

Substituting the 'IPStub.dll' debugging functions for 'Netapi32.dll' (instead of 'o1' and 'o3') did not produce any pop-ups, so the crash occurs before either of the substituted functions is called. In other words, the import patching doesn't seem responsible for the crash.

So ... Is some KernelEx code failing? If this software works on XP or whatever, and all its required functions (AFAIK) are being supplied on W98, why should it crash in this way? Any ideas?

Joe.

Edited by jds
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...