Jump to content

Error while compiling Python 3.4.10 with latest WinXP compatible Cygwin

Reino

By Reino
in Windows XP

 Share

Recommended Posts

Reino

Reino

Posted
Posted (edited)

Hello,

Some of you might know me from FFMpeg Windows XP Updated Builds by CoRoNe,or rather Windows XP compatible non-SSE2 FFmpeg binaries. Every 4 months or so I try to compile FFmpeg binaries for and on WinXP. Some external libraries for FFmpeg now require Python 3, which the latest WinXP compatible Cygwin install doesn't have (python 2.7.10). So I'm trying to compile Python 3.4.10 (if I'm correct, the latest WinXP compatible release) myself, but I'm getting some errors. 

[...]
gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse  -Werror=declaration-after-statement   -I. -IInclude -I./Include    -DPy_BUILD_CORE  -c ./Modules/signalmodule.c -o Modules/signalmodule.o
In file included from Include/Python.h:85:0,
                 from ./Modules/signalmodule.c:6:
./Modules/signalmodule.c: In function `fill_siginfo':
./Modules/signalmodule.c:746:60: error: `siginfo_t {aka struct <anonymous>}' has no member named `si_band'
     PyStructSequence_SET_ITEM(result, 6, PyLong_FromLong(si->si_band));
                                                            ^
Include/tupleobject.h:62:75: note: in definition of macro `PyTuple_SET_ITEM'
 #define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject *)(op))->ob_item[i] = v)
                                                                           ^
./Modules/signalmodule.c:746:5: note: in expansion of macro `PyStructSequence_SET_ITEM'
     PyStructSequence_SET_ITEM(result, 6, PyLong_FromLong(si->si_band));
     ^
make: *** [Makefile:1648: Modules/signalmodule.o] Error 1

This seems to be a known problem. After applying '3.4-issue21085-struct_siginfo.patch' the compilation-process continues, but then stops at: 

[...]
gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse  -Werror=declaration-after-statement   -I. -IInclude -I./Include    -DPy_BUILD_CORE  -c ./Modules/xxsubtype.c -o Modules/xxsubtype.o
gcc -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse  -Werror=declaration-after-statement   -I. -IInclude -I./Include    -DPy_BUILD_CORE \
      -DHGVERSION="\"`LC_ALL=C `\"" \
      -DHGTAG="\"`LC_ALL=C `\"" \
      -DHGBRANCH="\"`LC_ALL=C `\"" \
      -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
gcc -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse -O2 -march=pentium3 -mtune=athlon-xp -mfpmath=sse -msse  -Werror=declaration-after-statement   -I. -IInclude -I./Include    -DPy_BUILD_CORE -o Python/frozen.o Python/frozen.c
rm -f libpython3.4m.a
ar rc libpython3.4m.a Modules/getbuildinfo.o
ar rc libpython3.4m.a Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o
ar rc libpython3.4m.a Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/capsule.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o
ar rc libpython3.4m.a Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/random.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o   Python/thread.o Python/frozen.o
ar rc libpython3.4m.a Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o
ar rc libpython3.4m.a Modules/_threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/_functoolsmodule.o  Modules/_operator.o  Modules/_collectionsmodule.o  Modules/itertoolsmodule.o  Modules/atexitmodule.o  Modules/_stat.o  Modules/_localemodule.o  Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/zipimport.o  Modules/faulthandler.o  Modules/_tracemalloc.o Modules/hashtable.o  Modules/symtablemodule.o  Modules/xxsubtype.o
ranlib libpython3.4m.a
make: *** No rule to make target 'libpython3.4m.dll.a', needed by 'python.exe'.  Stop.

I don't understand how such a rule is missing from the 'Makefile'. Does anyone know what I'm missing here?

Edited by Reino
1
Link to comment
Share on other sites
  • Reino changed the title to Error while compiling Python 3.4.10 with latest WinXP compatible Cygwin

Reino

Reino

Posted
  • Author
Posted
On 1/2/2023 at 3:11 PM, user57 said:

was it renamed or deleted ?

I don't know Cygwin's changelog history, but I guess it has never provided that function.
But, as mentioned earlier, this issue was solved because of the patch I've applied. It's the other error I'm still having problems with.

On 1/2/2023 at 5:44 PM, R1600 said:

I don't have knowledge about this, but AFAIK the latest Windows XP compatible Python release is 3.4.4.

Looking at https://www.python.org/downloads/windows/ you'll see it doesn't mention "Note that Python 3.4.10 cannot be used on Windows XP or earlier.", so I guess it's the latest official Windows XP compatible release.

On 1/2/2023 at 5:44 PM, R1600 said:

not sure this would be helpful - Python 3.8.13 for Windows XP SP3

Hardly "official" and rather experimental. Also as far as I can see, no source-files, only a pre-compiled release.

1
Link to comment
Share on other sites
VistaLover

VistaLover

Posted
Posted

Someone else, sometime ago, did succeed into compiling py3.4.10_x86:

http://matejhorvat.si/en/windows/python/index.htm

I have been successfully running that in my Vista SP2 32-bit machine for over two years now :P ... 

 Python 3.4.10 (default, Jul 14 2019, 14:41:03) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

It is WinXP SP3 compatible, however I'm not sure it'll run on a non-SSE2 CPU :dubbio:...

BTW, many thanks :wub: for those FFmpeg builds...

 

Link to comment
Share on other sites
  • 4 months later...
cmalex

cmalex

Posted
Posted (edited)

Greetings.

Thanks.

On 1/1/2023 at 6:56 PM, Reino said:

Some external libraries for FFmpeg now require Python 3, which the latest WinXP compatible Cygwin install doesn't have (python 2.7.10).

Package "python3" is a python 3.4.3 with a bunch of patches (look at source package). http://cygwinxp.cathedral-networks.org/x86/release/python3/

On 5/12/2023 at 5:22 PM, Reino said:

Python-3.4.10_config-files.7z

At first look - strange problem in Makefile

Lines in Makefile

028: VERSION=    3.4

043: LDVERSION=    $(VERSION)$(ABIFLAGS)

196: LDLIBRARY=      libpython$(LDVERSION).dll.a

199: DLLLIBRARY=    libpython$(LDVERSION).dll

But

655: $(DLLLIBRARY) libpython$(VERSION).dll.a: $(LIBRARY_OBJS)

So it will build "libpython3.4.dll.a", but "libpython3.4m.dll" and search for linking with "libpython3.4m.dll.a"

Please try to replace "$(VERSION)" with "$(LDVERSION)" in 655 string of Makefile or 641 of Makefile.pre.in

PS. Sorry, i need some more time to understood Cygwin.

UPD: 3.4-distutils-soname.patch:21 !

UPD2: With patches from 3.4.3 (last - python3-3.4.3-1.src.patch - must be applued with "-f") 3.4.10 compiles and works under Cygwin.

Best regards.

Edited by cmalex
3
Link to comment
Share on other sites
Reino

Reino

Posted
  • Author
Posted

Hello cmalex,

On 5/12/2023 at 6:42 PM, cmalex said:

At first look - strange problem in Makefile
[...]
Please try to replace "$(VERSION)" with "$(LDVERSION)" in 655 string of Makefile or 641 of Makefile.pre.in

Thanks. That solves that issue, but the next issue is that it fails to build ALL modules: 

building '...' extension
[...]
/usr/lib/gcc/i686-pc-cygwin/5.4.0/../../../../i686-pc-cygwin/bin/ld: cannot find -lpython3.4
collect2: error: ld returned 1 exit status

[...]
Failed to build these modules:
_bisect               _codecs_cn            _codecs_hk
_codecs_iso2022       _codecs_jp            _codecs_kr
_codecs_tw            _crypt                _csv
_ctypes               _ctypes_test          _datetime
_decimal              _elementtree          _heapq
_json                 _lsprof               _md5
_multibytecodec       _multiprocessing      _opcode
_pickle               _posixsubprocess      _random
_sha1                 _sha256               _sha512
_socket               _struct               _testbuffer
_testcapi             _testimportmultiple   array
audioop               binascii              cmath
fcntl                 grp                   math
mmap                  parser                pyexpat
resource              select                syslog
termios               time                  unicodedata
xxlimited             zlib

 

On 5/12/2023 at 6:42 PM, cmalex said:

Package "python3" is a python 3.4.3 with a bunch of patches (look at source package). http://cygwinxp.cathedral-networks.org/x86/release/python3/

Wow, great find! Seeing so many patches are necessary for Cygwin, it's no wonder I'm having so much difficulty. I would've never been able to solve this on my own.

At the moment I have this working procedure in my (local) script: 

build_python() {
  download_and_unpack_file http://cygwinxp.cathedral-networks.org/x86/release/python3/python3-3.4.3-1-src.tar.xz python3-3.4.3-1.src
  cd python3-3.4.3-1.src
    #cygport python3.cygport all   # cygport is not available with Cygwin 2.874.
    if [ ! -f "Python-3.4.3_unpacked.successfully" ]; then
      echo -e "\e[1;33mUnpacking 'Python-3.4.3.tar.xz'.\e[0m"
      tar -xf "Python-3.4.3.tar.xz" || exit 1
      touch "Python-3.4.3_unpacked.successfully" || exit 1
      rm "Python-3.4.3.tar.xz" || exit 1
    fi
    if [ ! -f "patch_path-correction.done" ]; then
      sed -r -i "s/(orig)?src\/Python-[0-9\.]*\///" *.patch
      touch "patch_path-correction.done" || exit 1
    fi
    cd Python-3.4.3
      apply_patch ../python3-3.4.3-1.src.patch
      apply_patch ../3.4-dbm-cygwin.patch
      apply_patch ../3.1-enable-new-dtags.patch
      apply_patch ../3.4-tkinter-cygwin.patch
      apply_patch ../3.4-ctypes-cygwin.patch
      apply_patch ../3.1-PATH_MAX.patch
      apply_patch ../3.1-ncurses-abi6.patch
      apply_patch ../3.2-export-PySignal_SetWakeupFd.patch
      apply_patch ../3.4-distutils-soname.patch
      apply_patch ../3.2-distutils-shlibext.patch
      apply_patch ../3.4-pep3149-cygwin.patch
      apply_patch ../3.4-thread-cygwin64.patch
      ac_cv_func_bind_textdomain_codeset=yes do_configure --prefix=/usr --enable-shared --enable-ipv6 --with-dbmliborder=gdbm --with-libc= --with-libm= --with-system-expat --with-system-ffi --without-ensurepip   # 'configure'-options from '../python3.cygport'.
      do_make install
    cd ..
  cd ..
}

At first I saw this 'python3.cygport' file. After some reading I realized that with an internal tool called "cygport" I could automatically compile and install the whole Python release, BUT I soon found out that Cygwin 2.874 (the latest WinXP compatible version) doesn't even have this tool. For a website that hosts this latest WinXP compatible Cygwin release I find that really strange. Guess I'll have to do it myself after all.

I've had a look at these patches (I really don't know what they're for, but they're probably necessary) and found out that you won't need '3.2-getpath-exe-extension.patch' and '3.4-select-cygwin.patch' if you're already starting off with 'python3-3.4.3-1.src.patch'.
Next I've added the 'configure'-options from 'python3.cygport'. In the end this produced a working Python3 Cygwin install! :cheerleader:

On 5/12/2023 at 6:42 PM, cmalex said:

UPD2: With patches from 3.4.3 (last - python3-3.4.3-1.src.patch - must be applued with "-f") 3.4.10 compiles and works under Cygwin.

`patch --force` you mean? Nice to see you got it working for v3.4.10. I'll have a look at it soon.
What 'configure'-options did you use?

1
Link to comment
Share on other sites
cmalex

cmalex

Posted
Posted

Greetings

7 hours ago, Reino said: 
cannot find -lpython3.4

Because modules still tries to link with -lpython$(VERSION)... Sorry, I didn't know.

7 hours ago, Reino said:

After some reading I realized that with an internal tool called "cygport" I could automatically compile and install the whole Python release, BUT I soon found out that Cygwin 2.874 (the latest WinXP compatible version) doesn't even have this tool.

How did You checking the presence of packages? It's exists - http://cygwinxp.cathedral-networks.org/noarch/release/cygport/

I'd install Cygwin using setup-x86-2.874.exe via runme.bat from  http://cygwinxp.cathedral-networks.org/cathedral/

And to add packages i just rerun runme.bat. It must show list of available packages with search by name.

7 hours ago, Reino said:

`patch --force` you mean?

Yes. And cygport build fails because of

7 hours ago, Reino said:

you won't need '3.2-getpath-exe-extension.patch' and '3.4-select-cygwin.patch' if you're already starting off with 'python3-3.4.3-1.src.patch'.

7 hours ago, Reino said:

Nice to see you got it working for v3.4.10. I'll have a look at it soon.
What 'configure'-options did you use?

Options from python3.cygport . It require many *-devel packages from Cygwin setup but as result only two or three failed internal modules (ossaudio). You don't need to reconfigure - just install the dependency and run make again

Externals must be installed via pip.

Offtopic - Do You have in plans to adding brotli, zstd and nghttp2/nghttp3 support to curl?

Best regards.

Clipboard01.png

1
Link to comment
Share on other sites
cmalex

cmalex

Posted
Posted (edited)

Greetings

Under Cygwin for XP even 3.9.16 work fine (but i didn't try without SSE2! Only in virtual system).

Part 1 

# Download, patch and build
wget https://mirrors.163.com/cygwin/x86_64/release/python39/python39-3.9.16-1-src.tar.xz
tar --strip-components=1 -xf python39-3.9.16-1-src.tar.xz
tar -xf Python-3.9.16.tar.xz
cd Python-3.9.16
cat ../3*.patch | patch --strip 2
./configure --enable-shared --enable-ipv6 --with-libc= --with-libm= --with-system-expat --with-system-ffi --enable-loadable-sqlite-extensions ac_cv_func_bind_textdomain_codeset=yes
make 
# Install to default - /usr/local
make install
# Add *.a to LD search path
cp libpython*.a /lib

Part 2 Cygwin rebaseall. Prepare files for Cygwin rebase 

find /bin /lib /usr -iname '*.so' > /tmp/to_rebase.lst
find /bin /lib /usr -iname '*.dll' >> /tmp/to_rebase.lst

Close all Cygwin terminals and programs and run as Admin Cygwin/bin/ash.exe 

/bin/rebaseall -T /tmp/to_rebase.lst

Part 3 From Cygwin console install pip and yt-dlp (for example) 

python3.9 -m ensurepip && python3.9 -m pip install wheel
pip3.9 install yt-dlp

If errors like "child_info_fork::abort: unable to remap" appeared - it's time to rebaseall.

Best regards.

Edited by cmalex
Fix --with-libc= --with-libm=
Link to comment
Share on other sites
Reino

Reino

Posted
  • Author
Posted (edited)
On 5/14/2023 at 9:54 AM, cmalex said:

How did You checking the presence of packages? It's exists - http://cygwinxp.cathedral-networks.org/noarch/release/cygport/

I'd install Cygwin using setup-x86-2.874.exe via runme.bat from  http://cygwinxp.cathedral-networks.org/cathedral/

Oh, I understand now. Back when I cloned rdp's repo I've left the cygwin installer command practically untouched, which only installs a portion of all the packages. So no wonder it can't find cygport.

On 5/15/2023 at 8:56 PM, cmalex said:

Under Cygwin for XP even 3.9.16 work fine (but i didn't try without SSE2! Only in virtual system).

Thanks for you effort, but I've settled with v3.4.10, which is good enough for me. I've bunched all the necessary patches into one and ultimately these patches proved to be the solution for this thread to get me working Python3 Cygwin install.

On 5/15/2023 at 8:56 PM, cmalex said:

Part 2 Cygwin rebaseall. Prepare files for Cygwin rebase 

find /bin /lib /usr -iname '*.so' > /tmp/to_rebase.lst
find /bin /lib /usr -iname '*.dll' >> /tmp/to_rebase.lst

Close all Cygwin terminals and programs and run as Admin Cygwin/bin/ash.exe 

/bin/rebaseall -T /tmp/to_rebase.lst

I'm unfamiliar with the term "rebase" in this case. What does it do and why is it needed?

On 5/14/2023 at 9:54 AM, cmalex said:

Offtopic - Do You have in plans to adding brotli, zstd and nghttp2/nghttp3 support to curl?

No. Honoustly, I can't even remember why I started compiling and uploading Curl binaries in the first place. If I want to download something, then I'll use the Palemoon browser and if I want to scrape / extract something, then I'll use Xidel.

Edited by Reino
Link to comment
Share on other sites
cmalex

cmalex

Posted
Posted (edited)

Greetings.

12 hours ago, Reino said:

I'm unfamiliar with the term "rebase" in this case. What does it do and why is it needed?

Sorry, i didn't have enough knowledge. Below is how i understand it.

MSYS2 and Cygwin have some problems between fetching actual dll location in memory (default is called "base address") by Windows LoadLibrary()/GetModuleHandle()/FreeLibrary() functions (address may change between process launches) and Linux fork() function (which create full clone of existing process - including dll handles from parent).

"rebaseall" command creates a list of fixed addresses in memory to place dll's.

Theoretically, there is no need for "rebaseall" - only to "rebase" new dll's ( for example - python modules, that was created by pip). Something like 

find /usr/local/lib/python3.4/ -iname "*.dll" > rebase.lst
rebase -T rebase.lst

https://social.msdn.microsoft.com/Forums/ie/en-US/e65e5454-a35c-4dfd-9532-9284460310e1/dll-rebase?forum=windowssdk

https://web.archive.org/web/20090221070123/http://msdn.microsoft.com:80/en-us/library/ms810432.aspx

Best regards.

Edited by cmalex
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...