Only 1 site slow. Can't isolate where problem is.

[ddudenhoeffer]

Hello,

I'm a desktop/network support person at a company called Data Source Inc in KC. We use an online application called ESPOnline supplied to us by ASICentral. This application has its own browser built in and is strictly online. For the past 1-2 weeks, it has been extremely slow and much of the functions in it time out because it is not getting the data fast enough.

I talked with ASICentral (www.asicentral.com) and at first they recommended uninstalling and reinstalling the app. They sent me a link to download the app from www.asicentral.com/support/update. It is just a 6 meg exe file. However, that download was going slow too. It was going to take over 2 hours to get it. I decided to wait and let it download, but the download never finishes. Matter of fact, it does say it finished, but it only downloaded about half of it.

Now, here is where it gets interesting. In my troubleshooting, I noticed our internet connection going every where else is fine... it is fast as can be. I downloaded a 7 meg file from download.com in about 1 minute. Next, I went home and downloaded the file from www.asicentral.com/support/update in less than a minute. So, for some reason at work, we have a slow connection to ASICentral. ASI said it is not a problem on their end because no one else is having a problem... they said they have been walking people through downloading the file and it normally takes around 3 minutes. I called our ISP and they can't find anything wrong either.

I downloaded VisualRoute and noticed we get 90-100% packet loss about 3-4 hops before the final destination to asicentral.com, but my ISP told me that is not a problem because the router probably doesn't respond to icmp.

We have looked at our firewall, surfcontrol, isa server, etc and can't find anything wrong.

Any ideas would greatly be appreciated. Please let me know what info you might need to help me out!

Thanks,

Doug Dudenhoeffer

Edited March 1, 2007 by ddudenhoeffer

[cluberti]

A network trace from your machine, and at your border router, simultaneously, will at least show how fast the traffic is passing on your network (likely you'll see a request from the client immediately leave from your border router), and how long it takes to get back (you'll also likely see the delay after the packets have left your border router, but when they do return, the client gets the packets almost immediately after that). It won't show you what the problem is, but you can at least be sure it isn't on your network at this point. If you're getting 100% packet loss (or anything like that), it is definitely a routing issue somewhere between you and the destination server(s). Assuming no one else is having a problem accessing those servers from outside of your ISP's network........ .

[ddudenhoeffer]

How do I do the simultaneous trace? I would like to know at least if it is an issue on our network or some other routing issue. Problem is, if it is a routing issue, I don't know where to go...

Take a look at this visual route screen shot in the attached word doc.

Thanks,

DougVisualRoute2.doc

[Drugwash]

Dunno if it's gonna be of much help: 3D Traceroute.

[cluberti]

Use WireShark or Netmon to gather the trace from the workstation. As to the network device, you usually attach a machine to the admin port of the switch/router, or use a hub between the switch / router port and the network, with a machine attached to the hub in promiscuous mode running wireshark or netmon.

[ddudenhoeffer]

Still working on this issue.

Here is what I think, but need someone with more knowledge to tell me if I'm right.

I think I'm seeing packet loss 3-4 hops before the final destination. It is on a router owned by Telcove Inc. When I use Visual Route and do a trace to ASICentral.com, I always have 90-100% loss. Now, I am reaching ASICentral in 70-80ms most of the time, but this packet loss I believe is what's slowing down the application as well as the download from asicentral.com/support/update.

How do I handle a problem like this? I've talked to Telcove before and they tried to tell me it is not their problem and they really couldn't help me because I'm not a customer of theirs. How do I get this fixed when it looks like it is a problem with the internet. Our ISP stated they can't do anything.

Could it be a problem with our network why we are getting packet loss near the end of the destination? Thing is it used to work fine and we never made any changes to the network.

I'm lost on where to go on this one, but it is severely hendering the performance of an entire department in my company. Unfortunately we rely on asicentral's product for much of our business.

Any more suggestions or help would greatly be appreciated.

Thanks,

Doug

[cluberti]

I think I'm seeing packet loss 3-4 hops before the final destination. It is on a router owned by Telcove Inc. When I use Visual Route and do a trace to ASICentral.com, I always have 90-100% loss. Now, I am reaching ASICentral in 70-80ms most of the time, but this packet loss I believe is what's slowing down the application as well as the download from asicentral.com/support/update.

Well, it could be the router you're hitting at Telcove, or it could be the hop before it that's misconfigured. You really should get with your ISP to trace this through their network - i.e., a trace by the ISP on your switch port and their border router before you hit telcove (good luck, some do, some don't provide this kind of help). You could at least see the packets on your ISPs network to know if it's your ISP or telcove.

How do I handle a problem like this? I've talked to Telcove before and they tried to tell me it is not their problem and they really couldn't help me because I'm not a customer of theirs. How do I get this fixed when it looks like it is a problem with the internet. Our ISP stated they can't do anything.

If that's the case, and it really is telcove, your only real options are switching ISPs so you don't go through telcove to hit this site, or having your ISP route your packets destined for that site through a different route (again, good luck). An ISP with fewer hops to a core backbone is preferable, but you will pay for the privilege (and it is worth it, if you're a heavy user).

Could it be a problem with our network why we are getting packet loss near the end of the destination? Thing is it used to work fine and we never made any changes to the network.

I'm lost on where to go on this one, but it is severely hendering the performance of an entire department in my company. Unfortunately we rely on asicentral's product for much of our business.

It's not your problem, and the likelihood that it's your ISP is also very slim. Again, if telcove won't give you (or your ISP, if you can get them behind you) any help, switch ISPs so you don't ever go through telcove to hit that site. Others obviously don't, so it should at least be a possibility .