ProxHTTPSProxy and HTTPSProxy in Windows XP for future use

[AstroSkipper]

On 11/29/2022 at 7:10 PM, RainyShadow said:

I found this small tool and was planning to just dump it here in case anyone needs it... 

I didn't check it in XP, but since the source is provided, i guess it could be recompiled easily. 

 

P.S. make sure to check the other projects on that site too.

Hello @RainyShadow! Thanks for your hint! Frankly, I'd know exactly what I would have to do to port my program package ProxHTTPSProxy's PopMenu TLS 1.3 3V3 to Windows 7. For checking or changing the system's proxy status, this OS has also corresponding, native commands which probably could be easily implemented in my program modules. But at the moment, it's not an issue for me. 

Edited December 17, 2022 by AstroSkipper

[WinFX]

Why ProxHTTPSProxy 1.5 doesn't work in IE6 but works in IE8?

Edited December 21, 2022 by WinFX

[AstroSkipper]

10 hours ago, WinFX said:

Why ProxHTTPSProxy 1.5 doesn't work in IE6 but works in IE8?

It's a very long time ago, I used IE6. When IE7 came out, I upgraded to that version and later to IE8. Anyway! As far as I can remember, it has something to do with the signing process of certificates (SHA-1 vs. SHA256), but I have to check that once again if more time is available. Try instead ProxHTTPSProxy 1.3a! You can find it in the section 11.1. Archived Downloads (obsolete) in the first post of this thread. The more recent proxies such as ProxHTTPSProxy 1.5 and up are too "modern" for the old IE6. But actually, you should upgrade to IE8 in any case, only if possible, of course.

Edited December 21, 2022 by AstroSkipper
Update of content

[AstroSkipper]

Interesting news!

ProxHTTPSProxy can only be used by programs which offer an option to use the IE proxy settings as for example browsers or use system components if the proxy has been set system-wide. From now on, we have the possibility to connect any program with a server or the internet via the local proxy ProxHTTPSProxy if necessary, of course including the latest protocol TLS 1.3. The solution is SocksCap64. I have already tested this tool, and it works perfectly with ProxHTTPSProxy.
Link: https://www.sockscap64.com/homepage/

Cheers, AstroSkipper

[Snowshoe]

On 1/4/2023 at 5:19 PM, AstroSkipper said:

Interesting news!

ProxHTTPSProxy can only be used by programs which offer an option to use the IE proxy settings as for example browsers or use system components if the proxy has been set system-wide. From now on, we have the possibility to connect any program with a server or the internet via the local proxy ProxHTTPSProxy if necessary, of course including the latest protocol TLS 1.3. The solution is SocksCap64. I have already tested this tool, and it works perfectly with ProxHTTPSProxy.
Link: https://www.sockscap64.com/homepage/

Cheers, AstroSkipper

Can you demonstrate how to get this working with ProxHTTPSProxy? Or show your settings inside SocksCap64? It doesn't have a HTTPS proxy setting, only HTTP, which doesn't work for me.

I am trying to use programs that do use HTTPS but don't use the system proxy server (including programs that use libcurl.dll, or have curl static compiled into the executable). Simply updating root certificates and enabling TLS 1.2 in IE is not enough, I get a mis-matched hostname certificate error when trying to run programs that use curl (or go on certain sites with IE when ProxHTTPSProxy isn't enabled, for that matter). Also, redirecting requests from curl to plain HTTP is not always possible because the server does not serve plain HTTP and only uses a 301 redirect to HTTPS regardless of the URL.

Edited January 27, 2023 by Snowshoe

[cmalex]

On 1/27/2023 at 3:09 PM, Snowshoe said:

I am trying to use programs that do use HTTPS but don't use the system proxy server (including programs that use libcurl.dll, or have curl static compiled into the executable). Simply updating root certificates and enabling TLS 1.2 in IE is not enough, I get a mis-matched hostname certificate error when trying to run programs that use curl

Greetings.

Certificates and proxy settings from IE are not always respected by curl and libcurl.

Did You tried system variables, used by libcurl - "https_proxy" for example https://curl.se/libcurl/c/libcurl-env.html

If You know exactly which SSL backend is used - You can use it's own enviroment variables for CA certificates.

For example, OpenSSL has been using "SSL_CERT_FILE" and "SSL_CERT_DIR" since the dawn of time : https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_default_verify_paths.html   https://www.openssl.org/docs/man3.0/man7/openssl-env.html

Best regards.

[Snowshoe]

On 1/29/2023 at 8:08 AM, cmalex said:

Greetings.

Certificates and proxy settings from IE are not always respected by curl and libcurl.

Did You tried system variables, used by libcurl - "https_proxy" for example https://curl.se/libcurl/c/libcurl-env.html

If You know exactly which SSL backend is used - You can use it's own enviroment variables for CA certificates.

For example, OpenSSL has been using "SSL_CERT_FILE" and "SSL_CERT_DIR" since the dawn of time : https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_default_verify_paths.html   https://www.openssl.org/docs/man3.0/man7/openssl-env.html

Best regards.

While setting the https_proxy environment variable to localhost:8079 forces it to use the HTTPS proxy, it seems to abort the connection at the certificate revocation check. I've looked around and I don't think cert verification in curl (at least with OpenSSL) can be disabled with an environment variable, but I could be wrong.

* Uses proxy env variable https_proxy == 'localhost:8079'
*   Trying 127.0.0.1:8079...
* Connected to localhost (127.0.0.1) port 8079 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to ms.kartkrew.org:443
> CONNECT ms.kartkrew.org:443 HTTP/1.1
Host: ms.kartkrew.org:443
Proxy-Connection: Keep-Alive
< HTTP/1.1 200 Connection established
< Proxy-agent: ProxHTTPSProxyMII FrontProxy/v1.5 Python/3.7.1
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* CONNECT phase completed!
* CONNECT phase completed!
* schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection 0
* schannel: shutting down SSL/TLS connection with ms.kartkrew.org port 443

[cmalex]

15 hours ago, Snowshoe said:

cert verification in curl (at least with OpenSSL) can be disabled with an environment variable, but I could be wrong.

Greetings.

At least with some version of curl (and OpenSSL) it works.

Best regards.

test_curl.zip

[AstroSkipper]

On 1/27/2023 at 2:09 PM, Snowshoe said:

On 1/5/2023 at 1:19 AM, AstroSkipper said:

Interesting news!

ProxHTTPSProxy can only be used by programs which offer an option to use the IE proxy settings as for example browsers or use system components if the proxy has been set system-wide. From now on, we have the possibility to connect any program with a server or the internet via the local proxy ProxHTTPSProxy if necessary, of course including the latest protocol TLS 1.3. The solution is SocksCap64. I have already tested this tool, and it works perfectly with ProxHTTPSProxy.
Link: https://www.sockscap64.com/homepage/

Cheers, AstroSkipper

Can you demonstrate how to get this working with ProxHTTPSProxy? Or show your settings inside SocksCap64? It doesn't have a HTTPS proxy setting, only HTTP, which doesn't work for me.

I am trying to use programs that do use HTTPS but don't use the system proxy server (including programs that use libcurl.dll, or have curl static compiled into the executable). Simply updating root certificates and enabling TLS 1.2 in IE is not enough, I get a mis-matched hostname certificate error when trying to run programs that use curl (or go on certain sites with IE when ProxHTTPSProxy isn't enabled, for that matter). Also, redirecting requests from curl to plain HTTP is not always possible because the server does not serve plain HTTP and only uses a 301 redirect to HTTPS regardless of the URL.

Hello @Snowshoe! I tested SocksCap64 with some browsers and it worked perfectly. You can try any other application but there might be some which won't work properly with SocksCap64. It's always trial and error. Anyway! Next time I'm at my desktop computer, I'll take some screenshots of my settings and post it here for you.

Edited January 31, 2023 by AstroSkipper
correction

[AstroSkipper]

3 hours ago, cmalex said:

Greetings.

At least with some version of curl (and OpenSSL) it works.

Best regards.

test_curl.zip 4.77 kB · 1 download

Hello @cmalex! Welcome back! I hope everything is OK with you. I send you my warmest greetings! And thanks for helping here! Always appreciated!

Cheers, AstroSkipper

Edited January 31, 2023 by AstroSkipper

[AstroSkipper]

Hello @Snowshoe! Here are two screenshots with the settings of my SocksCap64 installation:





If you have any further questions, do not hesitate to ask here! 

Cheers, AstroSkipper 

Edited January 31, 2023 by AstroSkipper
Update of content

[AstroSkipper]

On 1/30/2023 at 6:58 PM, Snowshoe said:

While setting the https_proxy environment variable to localhost:8079 forces it to use the HTTPS proxy, it seems to abort the connection at the certificate revocation check. I've looked around and I don't think cert verification in curl (at least with OpenSSL) can be disabled with an environment variable, but I could be wrong.

* Uses proxy env variable https_proxy == 'localhost:8079'
*   Trying 127.0.0.1:8079...
* Connected to localhost (127.0.0.1) port 8079 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to ms.kartkrew.org:443
> CONNECT ms.kartkrew.org:443 HTTP/1.1
Host: ms.kartkrew.org:443
Proxy-Connection: Keep-Alive
< HTTP/1.1 200 Connection established
< Proxy-agent: ProxHTTPSProxyMII FrontProxy/v1.5 Python/3.7.1
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* CONNECT phase completed!
* CONNECT phase completed!
* schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection 0
* schannel: shutting down SSL/TLS connection with ms.kartkrew.org port 443

@Snowshoe! I have checked your log file. Try following command line option with curl: --ssl-no-revoke. This disables cert revocation checks (Schannel) which most probably caused the error 0x80092012. You should do that only if you trust the server you want to connect. I had a similar problem with wget in the past, and disabling cert revocation was the way to solve it. 

Cheers, AstroSkipper 

Edited February 3, 2023 by AstroSkipper

[cmalex]

Greetings.

On 1/31/2023 at 3:13 PM, AstroSkipper said:

Welcome back! 

Thank you for Your attention and my best wishes to You.

17 hours ago, AstroSkipper said:

Try following command line option with curl: --ssl-no-revoke.

"These aren't the droids you're looking for." (c)

Problem is to force ProxiMII certificate as trusted for program, that didn't allow this from settings and didn't use Windows certificate store.

I can't figure out how an abstract program works with https - i need to investigate this specimen :-).

curl mostly uses ".curlrc" (in variaty of names for customs builds) and "cacert.pem" (idem)

OpenSSL is a Unix things - it's wide use config files and environment variables. But which settings was forced during compiling time? Is it was linked statically or dynamically? Does it use one of openssl.cnf, cacert.pem, SSL_DIR_PATH, SSL_FILE_PATH?

On 1/27/2023 at 3:09 PM, Snowshoe said:

I am trying to use programs that do use HTTPS but don't use the system proxy server (including programs that use libcurl.dll, or have curl static compiled into the executable)

On 1/30/2023 at 7:58 PM, Snowshoe said:

cert verification in curl (at least with OpenSSL) can be disabled with an environment variable,

Best regards.

[AstroSkipper]

4 hours ago, cmalex said:

Greetings.

On 1/31/2023 at 2:13 PM, AstroSkipper said:

Welcome back! 

Thank you for Your attention and my best wishes to You.

You're welcome! And dito! 

4 hours ago, cmalex said:

"These aren't the droids you're looking for." (c)

I love it! 

[AstroSkipper]

18 hours ago, cmalex said:

Problem is to force ProxiMII certificate as trusted for program, that didn't allow this from settings and didn't use Windows certificate store.

I can't figure out how an abstract program works with https - i need to investigate this specimen :-).

curl mostly uses ".curlrc" (in variaty of names for customs builds) and "cacert.pem" (idem)

OpenSSL is a Unix things - it's wide use config files and environment variables. But which settings was forced during compiling time? Is it was linked statically or dynamically? Does it use one of openssl.cnf, cacert.pem, SSL_DIR_PATH, SSL_FILE_PATH?

On 1/27/2023 at 2:09 PM, Snowshoe said:

I am trying to use programs that do use HTTPS but don't use the system proxy server (including programs that use libcurl.dll, or have curl static compiled into the executable)

On 1/30/2023 at 6:58 PM, Snowshoe said:

cert verification in curl (at least with OpenSSL) can be disabled with an environment variable,

Best regards.

Back on topic!
A few days ago, I had a similar problem with wget. Among other things, my self-created batch file should download a certain file from a server which leads to an error. I was able to fix the problem by adding an option to wget that disables the server certificate check against the available certificate authorities. With the option --no-check-certificate, wget was able to download the desired file again without any problems. Curl is a similar program which is able to download files from servers. It uses SSL/TLS protocols which can be controlled by different options as for example --sslv3, --tlsv1.2, --tlsv1.3, --tls-max and so on. Furthermore, curl has a lot of proxy options: 

--proxy [protocol://]host[:port] Use this proxy
--proxy-anyauth Pick any proxy authentication method
--proxy-basic   Use Basic authentication on the proxy
--proxy-cacert <file> CA certificate to verify peer against for proxy
--proxy-capath <dir> CA directory to verify peer against for proxy
--proxy-cert <cert[:passwd]> Set client certificate for proxy
--proxy-cert-type <type> Client certificate type for HTTPS proxy
--proxy-ciphers <list> SSL ciphers to use for proxy
--proxy-crlfile <file> Set a CRL list for proxy
--proxy-digest  Use Digest authentication on the proxy
--proxy-header <header/@file> Pass custom header(s) to proxy
--proxy-insecure Do HTTPS proxy connections without verifying the proxy
--proxy-key <key> Private key for HTTPS proxy
--proxy-key-type <type> Private key file type for proxy
--proxy-negotiate Use HTTP Negotiate (SPNEGO) authentication on the proxy
--proxy-ntlm    Use NTLM authentication on the proxy
--proxy-pass <phrase> Pass phrase for the private key for HTTPS proxy
--proxy-pinnedpubkey <hashes> FILE/HASHES public key to verify proxy with
--proxy-service-name <name> SPNEGO proxy service name
--proxy-ssl-allow-beast Allow security flaw for interop for HTTPS proxy
--proxy-tls13-ciphers <ciphersuite list> TLS 1.3 proxy cipher suites
--proxy-tlsauthtype <type> TLS authentication type for HTTPS proxy
--proxy-tlspassword <string> TLS password for HTTPS proxy
--proxy-tlsuser <name> TLS username for HTTPS proxy
--proxy-tlsv1   Use TLSv1 for HTTPS proxy
--proxy-user <user:password> Proxy user and password
--proxy1.0 <host[:port]> Use HTTP/1.0 proxy on given port
--proxytunnel   Operate through an HTTP proxy tunnel (using CONNECT)

Frankly, I don't really see the need to use curl with ProxHTTPSProxy. It should also do its job without this proxy in Windows XP. About programs with curl embedded, I can't say anything detailed. These programs would have to be specified more clearly. Maybe, @Snowshoe can explain more detailed what kind of programs he referred to, what he actually wants to achieve with these curl programs and why he needs a connection established by ProxHTTPSProxy, including the command line input or other execution commands which leads to the error. In any case, the curl option --ssl-no-revoke prevents the cert revocation check with the server to be connected.

Are these the droids you're looking for?  Probably not! 

Greetings from Germany, AstroSkipper 

Edited February 4, 2023 by AstroSkipper
Update of content