Microsoft ISA Server and Windows XP

[desrtfx]

Hi everybody!

I hope that this post fits in this category:

In the compan I work for we use Microsoft ISA Server as Internet Proxy. The company Network extends over various Locations which are connected via satellite links with Cisco gateways.

As far as I can see, all proxies and gateways are configured correctly, but on certain Windows XP pCs I cannot retrieve any Internet page.

The strange thing is that the problem does not appear on all Windows XP machines, it seems to appear only on newer (less than a year old) high performance and up-to date machines. Funnily enough, using Microsoft's virtual PC, or VMware to create and run a virtual machine connects that one without any problems even though it basically goes through the same network connection.

Does anybody have similar experiences or any ideas how to solve this problem without resorting to virtual machines.

Regards,

desrtfx (Georg)

[tain]

Are all of those new machines being deployed from the same master image? Has that master image been tested for this problem? Is it a special homebrew image that has been nLited or some other such wizardry?

Assuming you have already checked the gateway address and DNS addresses on the affected machines...is it a static or DHCP network? Could be an IP conflict.

funnily

I use that word sometimes because it doesn't sound like a real word and my gf gets annoyed at me

[desrtfx]

No, the new machines are not deployed from the same master image as they are contractor machines of different make, type, language, etc.

The problem stays the same with or without DHCP enabled and I can exclude IP conflicts as I am the one assigning IPs to the computers.

What still confuses me with this is that the virtual system accesses the same NIC without any problems. To me it seems that there is a Windows Patch common to all languages that causes the problem.

Even when I traceroute (with tracert, ant, solarwinds, etc.) I always reach all gateways on the way, but stop at the ISA server.

I know that the ISA is configured to work without authentication. Unfortunately, I cannot access any details of the ISA installation since it is done by the head office EDP and I am working in a site office which is connected through satellite link. I don't want to raise the issue with the head office since I never got even a close reply from there.

Funnily:

English

Adverb

funnily

1. in a funny manner

2. in a strange manner

acc to Wiktionary

Edited July 11, 2006 by desrtfx

[tain]

Some brainstorming is in order so here comes a hail of bullets in the dark

Is there something else that these machines have in common? Subnet? Domain/workgroup? Group membership? You've tried different users, I'm sure.

When using DHCP, do the machines get valid addresses? Shot in the dark, but could help pinpoint the problem depending on where the DHCP server resides.

You said that activity stops at the ISA server, can these machines ping it? What error do you get when you try to access the web through it? The proxy settings for the browser is OK?

What still confuses me with this is that the virtual system accesses the same NIC without any problems. To me it seems that there is a Windows Patch common to all languages that causes the problem.

The working VMs at least tells us that it isn't a physical problem. Is there a patch or some other software that is unique to these machines?

--

re: funnily, yeah...I said it doesn't sound like a real word B)

[desrtfx]

Well, with that amount of bullets, good thing I wear a Kevlar jacket

Ok, the light streams in through the bulletholes:

1. Yes, the machines are all on the same subnet: 255.255.248.0 (assigned from company)
   
2. I've tried different users on different user levels
   
3. The machines all get valid DHCP addresses since the DHCP is (for now) managed locally
   
4. The proxy settings for the browsers (IE, Firefox 1.5, Opera) are all OK
   
5. All machines can ping the ISA server and the DNS server (both at the central site) but nothing can be reached after the ISA
   
6. All machines have only
   • Windows Networking
     
   • File and Printer Sharing
     
   • TCP/IP networking
     

enabled

[*]Some machines are on the domain, some are not (has as far as I've seen no effect at all)

[*]Workgroup settings have no effect at all (tried joining the main domain, different workgroups, etc.)

[*]The errors I get when trying to reach a webpage differ from Browser to Browser

• IE gives "Server not found"
  
• Firefox gives "Connection to server reset"
  
• Opera 9 keeps displaying a blank page forever
  

The only thing that I can think of being unique to these machines is that all of them are quite up-to-date with Microsoft patches.

Not even the OS (Windows XP) is common to the machines since some use XP Pro, some use XP Home, in UK & US English and in German versions. (Actually, the machines are private Laptops that need to be used in the work location and I don't really have much influence on the OS used.)

The Antivirus and Firewall solutions are different (Bitdefender, AVP, AVG, Symantec)

I already thought about installing the IPv6 protocol and seeing what will happen then, but I can't think that this will solve the problem.

Also one more thing: I found that some programs (other than webbrowsers) find their home servers (when I allow them to connect and set the proxy) - so - what do all webbrowsers have in common? Winsock?

Before I forget: Physical problems with the NICs can be totally excluded since all machines are working perfectly well on other networks - of course with changed IPs, Subnets, DNS, gateways, etc. - Actually, I tried to use WLAN on one net, and LAN on the other (swapped) - still same result - so it cannot be anything with the NICs either.

I know that this is a tough one and appreciate all help I can get.

Edited July 12, 2006 by desrtfx

[tain]

Awesomely detailed post!

That is one crazy environment you are running there. Sounds like a nightmare to administer...

Looks like a browser proxy setting issue for these reasons:

Can ping the proxy

VMWare can get through the proxy

Other apps can get through the proxy

The environment seems diverse enough to rule out authentication

But then this issue conflicts with that notion:

All machines can ping the ISA server and the DNS server (both at the central site) but nothing can be reached after the ISA

Does this include simple pings? Try pinging a remote host outside of the proxy.

Another shot in the dark: how is VMWare networking setup?

How do you manually setup the proxy settings on the machines, other apps and virtual machines? The secret probably lies within there

The only thing that I can think of being unique to these machines is that all of them are quite up-to-date with Microsoft patches.

I know you keep asserting this theory but it really sounds like a long shot to me. A patch that breaks networking at layer 4 would be big news I think. You can try to troubleshoot the patches...on a machine that works fine, make yourself a list of what patches it doesn't have (when compared to a machine that is broken) and then install the the patches one-by-one, testing browser access to internet after each patch + reboot.

[desrtfx]

Thanks for the compliment!

I think that "nightmare" is an understatement

Up to a few months ago this network was only a small, island network which the company then decided to join via satellite to the main office and therefore the main central network. Here the problem started. They wanted us to become a child domain of theirs but the "specialist" failed in doing so (he had never seen Windows 2003 Advanced Server before and the main net is running on Windows 2000 Servers). He also wanted to install an Exchange Server which I could just stop before it was too late. I postponed the Exchange installation until the child domain would be running.

Now the network became patchy - we are using static IP addresses assigned according to our addresspace (which we got from the main office). The gateways are connected to a "VSat" satellite link system which joins the networks via 2MBits/s lines (shared with 5 other sites).

In my network I currently use only Cat5e and WLAN connections (on Netgear, 3com & Cisco equipment - All gateway routers are Cisco, all Switches are Netgear and all WLAN devices are also Netgear, only two 3com Hubs are currently in the system.)

The local domain server is a quite new HP Proliant running 2003 Advanced Server, but that one does not count for the current problem. DHCP is managed from one of the Netgear WLAN devices for now (because I am currently not using DHCP - it is just there to serve "Wildpluggers" and protect the network from IP conflicts)

The WLAN is not encrypted since the site is so remote that there is literally no chance of anybody "Wardriving"

I did a tracert today to reach a server that I know in Atlanta, Georgia (I have hosted domains there), but unfortunately, during the tracert the Internet connection after the ISA failed, so I was not able to get a detailed result. I added two new Laptops to the net this morning of which one immediately connected to the internet where the other one failed. On the failed one I did not even reach a response from the ISA Gateway.

The virtual machines I know working are running on Microsoft Virtual PC 2004 with default settings (I think the virtual PC defaults to NAT - isn't it?)

I also would like to take a shot in the dark - or better ask a really queer question (I am an absolute ISA newbie - never needed to work with it): Could it be, that if a PC registers itself with two addresses (e.g. LAN and WLAN on same subnet, but with different IPs), the ISA gateway blocks that PC? Could that be a possible cause? But on the other hand I don't think so, since when I only use one type of connection (LAN or WLAN) I still get the same results unless the gateway caches?

EDIT: Upon reflecting about this post another thought came up - Could it help to bridge the LAN/WLAN connections on the affected computers?

Regards,

desrtfx (Georg)

Edited July 12, 2006 by desrtfx

[tain]

DHCP is managed from one of the Netgear WLAN devices for now (because I am currently not using DHCP - it is just there to serve "Wildpluggers" and protect the network from IP conflicts)

You probably already know to make sure the scope is deconflicted with the IP range that you normally work with...

I did a tracert today to reach a server that I know in Atlanta, Georgia (I have hosted domains there), but unfortunately, during the tracert the Internet connection after the ISA failed, so I was not able to get a detailed result.

Is this true for all of the workstations in question? None of them can ping/traceroute past the ISA? But all of the known good machines can do so? FYI, the proxy might only filter port 80 or could limit other traffic altogether...

The virtual machines I know working are running on Microsoft Virtual PC 2004 with default settings (I think the virtual PC defaults to NAT - isn't it?)

I usually use VMWare, but I think that VPC uses the host adapter addresses. Check your VMs to see how they are networking and if they are getting separate IPs.

Could it be, that if a PC registers itself with two addresses (e.g. LAN and WLAN on same subnet, but with different IPs), the ISA gateway blocks that PC?

Not likely. From an IP accounting perspective it will look like two different machines (IPs/MACs).

Upon reflecting about this post another thought came up - Could it help to bridge the LAN/WLAN connections on the affected computers?

Probably not, but in the interest of troubleshooting you should disable the WLAN so we can eventually/finally pinpoint your problem.

Looks like a browser proxy setting issue for these reasons:

Have you ruled this out?

How do you manually setup the proxy settings on the machines, other apps and virtual machines? The secret probably lies within there

???

You can try to troubleshoot the patches...on a machine that works fine, make yourself a list of what patches it doesn't have (when compared to a machine that is broken) and then install the the patches one-by-one, testing browser access to internet after each patch + reboot.

Do you intend to try this?

BTW, your new avatar is cool

[desrtfx]

Ok, here we are again, sorry for the late reply, but I had a horrible day...

First of all, today, the first Windows 2000 Pro Laptop showed the same symptoms.

Apparently, I have been looking in the wrong direction. I dedicated loads of time to the problem on the Windows 2000 Pro machine today and found that there is no DNS response coming through - no matter what I set the DNS to (local DNS at my site, DNS at the main site, a couple of open DNS servers that I know of on the net) the problem stays the same - confused me even more.

Pinging is not possible even if I extend the Reply time over half a minute, I don't get a reply

Disabling one of the NICs didn't change the situation at all.

DHCP issues can be ruled out - I am 100% sure, that I have no interferences since all the PCs with static IPs are on 192.168.35.x and the DHCP is on 192.168.33.x both with Subnet mask 255.255.248.0 - so there cannot be any conflicts

In between I had a little brainstorming here with a colleague (hobby computer crack) who told me that on another laptop, he removed all Microsoft security updates and the problem was cured - so there's something we can build up on. (unfortunately, this worked so far only for one PC - lets see what the future brings...)

I think investigating in direction of the security updates gets me to the target...

So far, thanks for all the help.

I will keep posting news.

Regards,

desrtfx (georg) - really from the Sahara desert (my Avatar is NOT A JOKE )

[tain]

You are having issues with Win2k machines now? I understood that the problems were isolated to cutting-edge XP home/pro machines...

That might actually help you find an offending patch since now you are looking for patches that apply to BOTH systems.

On your DNS issue, have you tried running wireshark (ethereal) on another machine in the subnet to see if the DNS reponse is coming back but getting dropped/ignored by the requesting machine?

[desrtfx]

Well, up to yesterday, I also shard the opinion that only cutting edge WinXP computers would be affected, but reality proved me wrong.

So far I haven't actually tried ethereal - simply didn't have the time to test this. After all I cannot dedicate much time and efforts to this problem since it's only private Laptops (partly used for company business) that are affected.

I will keep testing one or the other approaches, but that will have to wait until I manage to get my PSU for my personal notebook, this then gives me more time.

Regards,

desrtfx (Georg)

[RogueSpear]

Given the following:

• Computer A cannot web browse
  
• A virtual computer installed on Computer A can web browse
  
• Non-browser type applications can find their home server
  
• DNS resolution does not appear to be working
  

I would say there is an issue with a service. Make sure that the TCP/IP NetBIOS Helper service is running, even though you probably do not use WINS on your network. Make sure that the UPnP and SSDP Discovery Service are disabled. Make sure the DNS Client service is running and if it is, double check any group policies to see if there is a custom DNS option configured. Are you able to resolve DNS for your local intranet?

I have next to no experience when it comes to ISA Server, but it sounds as though you have it configured in a fairly relaxes pass thru kind of mode, so that does not sound like the problem to me - especially given how some other clients are blowing right through it without any issues.

[tain]

Thanks for jumping in, RogueSpear. You are quite the networking ninja and I always learn a lot from your posts

[desrtfx]

Hi Rogue,

This post just nicely sums a long conversation - yes, the issues that you listed are my problems.

Yet: I can exclude any groupp policy issues since it happens even for "vanilla" machines which come with a clean install and only the Windows updates - no hand has been laid on any group policies, DNS settings other than the ones in the Network Property sheets.

All the listed services are what they should be (I actually mimicked the services on a machine which could connect and tested with the same result - no reply from DNS)

DNS resolution on the local intranet works perfectly fine, no problems there.

The more I reflect upon this matter, the more convinced I am that there is a security update which causes that effect. I just want to wait now until one of my colleagues has tested his machine (without the security updates).

One more thing that is worth mentioning: If I re-configure the machines for another network without the queer setup of the ISA gateway, etc. all machines work just fine!

The gateway is basically setup as a firewall/proxy/download prefetcher/virus scan - but the open ports are extremely restricted (not even POP and SMTP). I, for my own part can't claim any experience with the ISA Server since I haven't yet setup or configured any - and I'm not entirely convinced that I will need to do that with our current network configuration.

Regards,

desrtfx (Georg)

[RogueSpear]

If DNS resolution works for local things, and you sure would know if it weren't since AD would fail miserably, yet DNS does not resolve anything to the outside, then there is a DNS issue. Unfortunately with the setup you have, I would be terribly unqualified to give you much advice.

Either DNS itself has a config issue, or ISA for whatever reason is blocking incoming DNS traffic to selected hosts. While I have not experienced this particular problem before, I certainly feel your pain.

EDIT: btw the reason I come to this conclusion is that your non-browser apps seem to be communicating fine. Most of the time these smaller apps tend to have an IP hardwired in the code, so DNS is not needed. Furthermore, they all tend to go over port 80 since almost everyone leaves that open.

Edited July 14, 2006 by RogueSpear