Python 3.8.13 for Windows XP SP3

[cmalex]

On 3/1/2023 at 9:48 PM, A1ternativE said:

Hello, I can't get Pillow to work on your assembly.

Error:

from . import _imaging as core
ImportError: DLL load failed: The specified procedure could not be found.
Any ideas?
Thank you

Greetings.

Please check lib\site-packages\PIL\_imaging.cp38-win32.pyd for "kernel32.dll" strings.

If there are any - replace them with "kernelxp.dll"

Best regards.

[Zorba the Geek]

I want to thank cmalex for all the effort he must have made in producing this backported build of Python 3.8.13.  I have succeeded in compiling an XP compatible yt-dlp which is very fortunate because I doubt if there are any other video downloaders currently being developed that support XP.

My problem with installing psutils was caused by my use of an unsuitable copy of pyconfig.h which lacked the line  #define NT_THREADS.  I used the pyconfig.h from version 3.4 which seemed to work. I am including it as an attachment.  I have revised my package of this backported Python 3.8.13 in a number of ways and you can download it from my 4Shared account here.  The SHA1 hash is bfe13847982d0d75f48b750f8bd700decd72ac87.Here is a list of changes to the package:

• I have included the pyconfig.h file from the MSI installer of Python 3.8.13, and changed the following lines

  /* set the version macros for the windows headers */
  /* Python 3.5+ requires Windows Vista or greater */
  #define Py_WINVER 0x0600 /* _WIN32_WINNT_VISTA */
  #define Py_NTDDI NTDDI_VISTA

  to this 

  /* set the version macros for the windows headers */
  /* Python 3.4+ requires Windows XP or greater */
  #define Py_WINVER 0x0501 /* _WIN32_WINNT_WINXP */
  #define Py_NTDDI NTDDI_WINXP

  There is still the line #define PYTHONPATH L".\\DLLs;.\\lib" which may need revising because there is no DLLs folder in this distribution.

• I have copied the contents of Python.zip to the .\Lib folder because pyinstaller seems to have a problem reading the contents of archives.  Now it works like a charm.

• I have updated batch files that install and uninstall Python to the a folder named Python38 in the root of the system drive.  i am not clever enough to make an interactive batch file allowing you to choose the name and location of the Python folder.  Maybe later.

• My batch file installs shortcuts for the Python console, the IDLE interactive envirironment, the Python help file, and the pydoc server.

• The Python path variables are set in the Environment registry key, and a PYTHONPATH value is created under the same key.

• It might be possible to install pip and setuptools with the bundled wheels using

  python -m ensurepip --default-pip

  but I have included get-pip.py to enable the download and install of the latest pip, setuptools and wheel/

pyconfig.h

Edited March 12, 2023 by Zorba the Geek
Wrong HASH value

[Zorba the Geek]

On 3/3/2023 at 5:18 PM, cmalex said:

Greetings.

Please check lib\site-packages\PIL\_imaging.cp38-win32.pyd for "kernel32.dll" strings.

If there are any - replace them with "kernelxp.dll"

Best regards.

I think what you mean is that _imaging.cp38-win32.pyd should be opened with CFFExplorer and the import directory edited so that the module named KERNEL32.DLL is renamed to kernelXP.dll.  I assume that kernelXP.dll supplied in the External folder of your distribution should be placed in the same folder as _imaging.cp38-win32.pyd.  The missing imports InitializeConditionVariable, InitializeCriticalSectionEx, SleepConditionVariableCS and WakeConditionVariable from kernel32.dll are actually supplied by kerneXP.dll so this might work.

I wonder if we can rely modules from the Xompie project like kernelXP.dll to allow the use of up to date Python modules with C extensions incompatible with XP as long as they supply the required functions to be imported.

[nicolaasjan]

11 hours ago, Zorba the Geek said:

I have succeeded in compiling an XP compatible yt-dlp which is very fortunate because I doubt if there are any other video downloaders currently being developed that support XP.

Well, there is mine.

https://github.com/nicolaasjan/yt-dlp/releases/latest/download/yt-dlp_x86_Windows-XP.zip

(manually uploaded; not part of the GitHub CI workflow)

 

That said, I ran into the same problems as you did.

Grabbed 'pyconfig.h' from a Py3.7 install and didn't have to change anything in it (there were no errors any more while compiling).

Disclaimer: I'm not a programmer.

[cmalex]

Greetings.

On 3/12/2023 at 7:59 PM, Zorba the Geek said:

I think what you mean is that _imaging.cp38-win32.pyd should be opened with CFFExplorer and the import directory edited so that the module named KERNEL32.DLL is renamed to kernelXP.dll

May be. I'd never used CFFExplorer. HxD or GSAR  utility(General Search And Replace - https://gnuwin32.sourceforge.net/packages/gsar.htm ).

On 3/12/2023 at 7:59 PM, Zorba the Geek said:

I assume that kernelXP.dll supplied in the External folder of your distribution should be placed in the same folder as _imaging.cp38-win32.pyd. 

Not sure. KernelXP.dll must exist in %PATH% or near python.exe

On 3/12/2023 at 7:31 PM, Zorba the Geek said:

unsuitable copy of pyconfig.h which lacked the line  #define NT_THREADS

I need to check it once more.  Did You surely used  pyconfig.h from DEV\Python3.8_dev_ddfdeced29.7z ?

On 3/13/2023 at 7:30 AM, nicolaasjan said:

That said, I ran into the same problems as you did.

Sorry, i did not understood - why psutils  easy compiles in my system - but throw errors on Your's with same configs...

Best regards.

Edited March 14, 2023 by cmalex

[nicolaasjan]

1 hour ago, cmalex said:

Sorry, i did not understood - why psutils  easy compiles in my system - but throw errors on Your's with same configs...

No, I was not referring to compiling psutils, but compiling yt-dlp (missing 'pyconfig.h').

I looked again now inside your Mega archive and only now saw it in './PythonXP/Dev/Python3.8_dev_ddfdeced29.7z/include/'.

But, as I said above:

Quote

Grabbed 'pyconfig.h' from a Py3.7 install and didn't have to change anything in it (there were no errors any more while compiling).

 

Edited March 14, 2023 by nicolaasjan

[Zorba the Geek]

 

cmalex thanks for taking the time to comment on the issues we have raised.  I need help compiling LOKi the IOC scanner which requires Python 3.8, but is not supposed to support XP.  I know that this should be raised as an issue at the LOKI Github site, but I am afraid they may laugh at me when I tell them I am trying to compile it for XP.

After version 0.45 was compiled on a Windows 8.1 machine LOKI ran fine with the following Python modules chosen for compatibility with XP:

pyinstaller-4.10. psutils-3.4.2, and pywin32-304

To run it on an XP machine I have to install Microsoft TCP/IP version 6 required by Brotli, and an XP compatible bcrypt.dll required by yara-python.  When first run under XP I receive this error message:

Linux System - deactivating process memory check ...
Traceback (most recent call last):
  File "loki.py", line 1515, in <module>
  File "loki.py", line 1490, in main
  File "lib\helpers.py", line 284, in getHostname
AttributeError: module 'os' has no attribute 'uname'

After some Googling I remedied this by editing this block of code in Loki-0.45.0\Lib\helpers.py

297    # Computername
298    if os_platform == "linux" or os_platform == "macos":
299        return os.uname()[1]
300    else:
301        return os.environ['COMPUTERNAME']

by changing os.uname() [1] to platform.uname() [1]

This resulted in a new error message thus:

Traceback (most recent call last):
  File "loki.py", line 1535, in <module>
  File "loki.py", line 142, in __init__
  File "loki.py", line 1413, in updateLoki
  File "subprocess.py", line 858, in __init__
  File "subprocess.py", line 1311, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified

After some Googling I remedied this by altering the following block of code in Python38\Lib\subprocess.py

736    def __init__(self, args, bufsize=-1, executable=None,
737                 stdin=None, stdout=None, stderr=None,
738                 preexec_fn=None, close_fds=True,
739                 shell=False, cwd=None, env=None, universal_newlines=None,
740                 startupinfo=None, creationflags=0,
741                 restore_signals=True, start_new_session=False,
742                 pass_fds=(), *, encoding=None, errors=None, text=None):

and changed shell-False to shell=True.  This then resulted in another error message thus:

Python path configuration:
  PYTHONHOME = 'D:\DOCUME~1\\LOCALS~1\Temp\_MEI32882'
  PYTHONPATH = (not set)
  program name = 'D:\Loki-0.45.0\dist\loki.exe'
  isolated = 0
  environment = 0
  user site = 0
  import site = 0
  sys._base_executable = 'D:\\Loki-0.45.0\\dist\\loki.exe'
  sys.base_prefix = ''
  sys.base_exec_prefix = ''
  sys.executable = 'D:\\Loki-0.45.0\\dist\\loki.exe'
  sys.prefix = ''
  sys.exec_prefix = ''
  sys.path = [
    'D:\\DOCUME~1\\USERPR~1\\LOCALS~1\\Temp\\_MEI32882\\base_library.zip',
    'D:\\DOCUME~1\\USERPR~1\\LOCALS~1\\Temp\\_MEI32882\\lib-dynload',
    'D:\\DOCUME~1\\USERPR~1\\LOCALS~1\\Temp\\_MEI32882',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
LookupError: unknown encoding: utf-8

At this point I am at my wits end because utf_8.py is present in the Python encodings folder.  I am trying to compare the code between version 0.45 and version 0.33 the last XP compatible version looking for clues.  This is complicated by the fact that version 0.33 was written for Python 2.7.

Edited April 30, 2023 by Zorba the Geek

[cmalex]

Greetings.

Sorry, currently didn't have access to Windows comp. I'll check it in nearest time.

On 3/18/2023 at 8:15 PM, Zorba the Geek said:

After version 0.45 was compiled on a Windows 8.1 machine

https://github.com/Neo23x0/Loki ? Looks like it didn't require compiling. Try to install it on target XP machine from source folder in editable (developer) mode:

pip install -e d:\LokiSourceFolder

or

cd /d d:\LokiSourceFolder

pip install -e .

On 3/18/2023 at 8:15 PM, Zorba the Geek said:

utf_8.py is present in the Python encodings folder

PyInstaller binaries prefer to use their own packages and folders. So the system-wide python can just be ignored. Please use "pip install -e" ;-)

On 3/18/2023 at 8:15 PM, Zorba the Geek said:

Linux System - deactivating process memory check ...

It's a first error - with pywin32 (loki.py, line 69). Please execute manually commands

        import wmi
        import win32api
        from win32com.shell import shell
        import win32file

On 3/18/2023 at 8:15 PM, Zorba the Geek said:

Python38\Lib\subprocess.py

changed shell-False to shell=True

AFAIK this is a big problem... Subprocesses will start to be executed in visible cmd windows - to grab their output. I hope that i'm wrong.

On 3/18/2023 at 8:15 PM, Zorba the Geek said:

I am trying to compare the code between version 0.45 and version 0.33 the last XP compatible version looking for clues. 

It's a very big distance between versions. Did You tried to run 0.40.0 BETA - first Python3 release? Git themself gave a handy instrument to find problems - "git bisect"

Best regards.

UPD:

1. Last compatible pywin32-300

2. pip -e didn't work. You can manually add path to unpacked source (for example "C:\Loki-0.45.0" ) to files "Lib\site-packages\loki.egg-path" and "Lib\site-packages\easy-install.pth". Then run with "python -m loki"

UPD2:

0. loki-upgrader.py must be in PATH... I'd create bat in Loki source folder:

Quote

cd /d "%~dp0"
set "PATH=%~dp0;C:\Python3.8\;%PATH%"
python -m loki
pause

1. pe-sieve32.exe work after replace "GetTickCount64" with "GetTickCount\0\0"

2. At least my version of python3.8 gave error with psutils 5.6.7, compiled by me during "connections()" loop. loki.py line 873.  Temporarly replace "p.connections():" with "[]:"... (UPD: Look like lack of update. On my main system with same Python folder psutils 5.6.7 works fine. )

Can You please give to me Yours build of psutils-3.4.2 ?

PS. Looks like it work.

Edited March 19, 2023 by cmalex

[cmalex]

Greetings.

Yet another "mess of things" ;-) https://mega.nz/folder/CgkRVQZA#d6J2CHt65J-vAEhwb0CmFg

Python 3.9.13 with some wheels.

Custom variables -

PYTHONCOMPILER = "mingw"/"mingw32"/"msvc" to force set compiler of distutils

SETUPTOOLS_USE_DISTUTILS = "stdlib" / "local" - to disallow (""stdlib") or allow ("local") of use bundled into setuptools distutils.

PS

On 3/18/2023 at 8:15 PM, Zorba the Geek said:

LOKI

Loki 0.45.0 https://mega.nz/file/nwkGAa7C#MZ3seTUo3zzHproGQXqNsrGvo2VUHppzTpiukJjeJsc
Best regards.

[Zorba the Geek]

See my next post

Edited March 28, 2023 by Zorba the Geek

[Zorba the Geek]

I have successfully compiled an XP compatible version of the latest release from Github which is 0.45 and now it runs perfectly with no error messages.  You can try it out by downloading loki-0.45.0-xp.7z from my 4Shared account.  You will also require this limited implementation of bcrypt.dll intended for running ffmpeg under Windows XP.  It supplies the BCryptGenRandom function required by yara.cp35-win32.pyd.  It should be placed somewhere in your path like the System32 folder.  Also the Microsoft TCP/IP version 6 network protocol is required by netaddr (I need to work on this).

If you want to compile it yourself you need to install the following Python packages:

colorama-0.4.6, future-0.18.3, netaddr-0.8.0, psutil-3.4.2, rfc5424-logging-handler-1.43, pywin32-300, wmi-1.5.0, yara-python-4.3.0, and pyinstaller-4.10.

There is no distribution of psutil-3.4.2 for Python 3.8. so I have compiled a wheel for psutil-3.4.2 which you can download here.

You will require the build of pe-sieve32.exe that does not require the function GetTickCount64 made available by cmalex here.

Loki seems to work OK with psutil-3.4.2, but the yara-python developers recommend version =>5.6.7.  It is possible to compile psutil-5.5.1 for Windows XP with reduced functionality, because _psutil_windows.c includes options for VISTA+ and XP depending on your operating system.  These are typically shown as

#if (_WIN32_WINNT >= 0x0600) // Windows Vista and above
Lines of code
#else // Windows XP
Lines of code
#endif

With a Python 3.8 installation this can only be done with VC14 under VISTA+ which means that the XP only options cannot be implemented.  Therefore it ought to be possible to edit _psutil_windows.c to remove the winnt>=6.0 specific code and compile an XP compatible build using VC14.  Also it is necessary to delete these lines in setup.py

111    if sys.getwindowsversion()[0] < 6:
112        msg = "this Windows version is too old (< Windows Vista); "
113        msg += "psutil 3.4.2 is the latest version which supports Windows "
114        msg += "2000, XP and 2003 server"
115        raise RuntimeError(msg)

 

 

Edited April 4, 2023 by Zorba the Geek

[Zorba the Geek]

I have compiled Windows XP compatible builds of psutil-5.5.1-cp38-cp38-win32.whl and psutil-5.5.1-cp27-cp27m-win32.whl which you can obtain as direct downloads.  I have tested _psutil_windows.cp38-win32.pyd and _psutil_windows.pyd with Dependency Walker and there are no missing imports under XP.  I have further tested them by installing them into the Python 3.8.13 supplied by cmalex and then rebuilding Loki, and everything works fine when Loki is run.

To create these Windows XP compatible builds of psutil-5.5.1 I first had to delete these lines in setup.py:

111    if sys.getwindowsversion()[0] < 6:
112        msg = "this Windows version is too old (< Windows Vista); "
113        msg += "psutil 3.4.2 is the latest version which supports Windows "
114        msg += "2000, XP and 2003 server"
115        raise RuntimeError(msg)

Then I added these lines after line 10 in _psutil_windows.c

#define WINVER 0x0501
#define _WIN32_WINNT 0x0501

These #define statements instruct the compiler to select the Windows XP options in the code suitable for the target OS.

These XP compatible builds have reduced functionality due to the functions ConvertLengthToIpv4Mask and GetIfEntry2 not being available in iphlpapi.dll version 5.1.2600.5512.  GetTickCount64 has been replaced with GetTickCount.

The Return NICs addresses section has reduced functionality due to the absence of ConvertLengthToIpv4Mask.

The Return path's disk total and free as a Python tuple section has reduced functionality due to the absence of GetIfEntry2.

The Return the number of physical CPU cores section was removed for Windows XP and VISTA after version 3.4.2 because they decided to use GetLogicalProcessorInformationEx rather than GetLogicalProcessorInformation.  I tried to copy this section of code from version 3.4.2 to version 5.5.1, but naturally compilation failed with lots of undefined variables messages.

It could be possible to make Windows XP compatible versions of psutil up to version 5.6.7 if you are prepared to rewrite the code in _psutil_windows.c and possibly elsewhere which I am not qualified to do.

Edited April 27, 2023 by Zorba the Geek

[greenhand]

Is there anyone else under this thread now? Many thanks to someone who was able to write python 3.8 to run on xp, but I'm having some problems with it, I'm currently trying to run QT and PYQT5 on xp, I'm able to run python 3.8 and QT now, but PYQT is having problems loading the DLL, here's my problem

ImportError: DLL load failed while importing QtWidgets

How do I debug it? Thanks a lot!

Edited April 12, 2023 by greenhand

[cmalex]

On 4/12/2023 at 12:41 PM, greenhand said:

trying to run QT and PYQT5 on xp

Greetings.

PyQt-5.7.1 compiled by qt-opensource-windows-x86-mingw530-5.7.1.exe . Needs SSE2 CPU (as QT>5)

Unpack to Python folder (because of sip.exe) https://mega.nz/file/708kWLSI#iPKglikgZZHGZdA7vVev4EnCiByzorsBXjlaZ7o7YdI

Best regards.

[genieautravail]

On 3/28/2023 at 11:17 AM, Zorba the Geek said:

I have successfully compiled an XP compatible version of the latest release from Github which is 0.45 and now it runs perfectly with no error messages. 

@Zorba the Geek

Your compiled version of LOKI doesn't works on my computer:

Any idea of what happen ?

Regards